2024-02-19T15:56:39.470 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-19T15:56:39.470 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} --owner scheduled_leonidus@teuthology --name leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488 --verbose -- /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/orig.config.yaml 2024-02-19T15:56:39.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T15:56:39.614 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488 branch: wip-lusov-qdb description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7566488' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee owner: scheduled_leonidus@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2280 sha1: 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi008.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6YLnbOWLAMMgd9y9+tWj00Ho2BFWYD4iRiKLHWDtiZ0W/ccp4KKfErA9licffVLsl1U+QT5YURnNVG80ilreo= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmO4OhXuNsz4g5x92SzAmoBxhaK2ReqT7bjDc5YDCRAi6G39ay7K8y7QhKaGxS7rlc1YwnBammnbEsi+sd24/o= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHbjSsnTtHJUjPF9JBs62HxTGDNA7R5A4lfvXf5YGv9HA9LovsrHODQDoRmMW2QLrpghe3v74AgEx64XX7d/ojM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-19_14:34:49 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-19T15:56:39.614 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa; will attempt to use it 2024-02-19T15:56:39.615 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks 2024-02-19T15:56:39.615 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-19T15:56:39.617 INFO:teuthology.task.internal:Checking packages... 2024-02-19T15:56:39.640 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee' 2024-02-19T15:56:39.640 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-19T15:56:39.640 INFO:teuthology.packaging:ref: None 2024-02-19T15:56:39.640 INFO:teuthology.packaging:tag: None 2024-02-19T15:56:39.640 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-02-19T15:56:39.640 INFO:teuthology.packaging:sha1: 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T15:56:39.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-02-19T15:56:39.853 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1451.g9f8908ee 2024-02-19T15:56:39.854 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-19T15:56:39.861 INFO:teuthology.task.internal:no buildpackages task found 2024-02-19T15:56:39.861 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-19T15:56:39.896 INFO:teuthology.task.internal:Saving configuration 2024-02-19T15:56:39.915 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-19T15:56:39.921 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T15:56:39.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 15:51:39.462193', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6YLnbOWLAMMgd9y9+tWj00Ho2BFWYD4iRiKLHWDtiZ0W/ccp4KKfErA9licffVLsl1U+QT5YURnNVG80ilreo='} 2024-02-19T15:56:39.970 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 15:51:39.463922', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmO4OhXuNsz4g5x92SzAmoBxhaK2ReqT7bjDc5YDCRAi6G39ay7K8y7QhKaGxS7rlc1YwnBammnbEsi+sd24/o='} 2024-02-19T15:56:39.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 15:51:39.465061', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHbjSsnTtHJUjPF9JBs62HxTGDNA7R5A4lfvXf5YGv9HA9LovsrHODQDoRmMW2QLrpghe3v74AgEx64XX7d/ojM='} 2024-02-19T15:56:39.992 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-19T15:56:39.998 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-19T15:56:39.998 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-19T15:56:39.998 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-19T15:56:39.998 INFO:teuthology.run_tasks:Running task console_log... 2024-02-19T15:56:40.093 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe7faa9c0d0>, signals=[15]) 2024-02-19T15:56:40.093 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-19T15:56:40.099 INFO:teuthology.task.internal:Opening connections... 2024-02-19T15:56:40.099 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T15:56:40.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T15:56:40.203 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-02-19T15:56:40.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T15:56:40.290 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-02-19T15:56:40.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T15:56:40.380 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-19T15:56:40.387 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2024-02-19T15:56:40.407 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2024-02-19T15:56:40.408 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="9" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="9" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el9" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:LOGO="fedora-logo-icon" 2024-02-19T15:56:40.466 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T15:56:40.467 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2024-02-19T15:56:40.467 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T15:56:40.467 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T15:56:40.467 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T15:56:40.467 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-02-19T15:56:40.495 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-02-19T15:56:40.513 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-02-19T15:56:40.514 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2024-02-19T15:56:40.571 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T15:56:40.572 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2024-02-19T15:56:40.572 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T15:56:40.572 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T15:56:40.572 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T15:56:40.572 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-02-19T15:56:40.595 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-02-19T15:56:40.613 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-02-19T15:56:40.614 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-02-19T15:56:40.670 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-02-19T15:56:40.670 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-02-19T15:56:40.670 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T15:56:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T15:56:40.671 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-02-19T15:56:40.695 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-19T15:56:40.705 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-19T15:56:40.711 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-19T15:56:40.711 DEBUG:teuthology.orchestra.run.smithi008:> test '!' -e /home/ubuntu/cephtest 2024-02-19T15:56:40.714 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-02-19T15:56:40.716 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-02-19T15:56:40.731 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-19T15:56:40.738 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-19T15:56:40.738 DEBUG:teuthology.orchestra.run.smithi008:> test -z $(ls -A /var/lib/ceph) 2024-02-19T15:56:40.771 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-02-19T15:56:40.774 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-02-19T15:56:41.104 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-19T15:56:41.177 INFO:teuthology.run_tasks:Running task kernel... 2024-02-19T15:56:41.191 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-02-19T15:56:41.191 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-02-19T15:56:41.192 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-02-19T15:56:41.192 DEBUG:teuthology.orchestra.run.smithi008:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-19T15:56:41.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T15:56:41.209 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2024-02-19T15:56:41.266 INFO:teuthology.orchestra.run.smithi008.stdout:5.14.0-361.el9.x86_64 2024-02-19T15:56:41.267 INFO:teuthology.task.kernel:Running kernel on smithi008: 5.14.0-361.el9.x86_64 2024-02-19T15:56:41.267 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum install -y kernel 2024-02-19T15:56:43.398 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-19T15:56:44.889 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-02-19T15:56:54.462 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - BaseOS 885 kB/s | 8.0 MB 00:09 2024-02-19T15:56:58.163 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-02-19T15:57:09.763 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - CRB 783 kB/s | 6.1 MB 00:07 2024-02-19T15:57:11.831 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 15 kB 00:00 2024-02-19T15:57:13.807 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-19T15:57:18.807 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-02-19T15:57:21.139 INFO:teuthology.orchestra.run.smithi008.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-19T15:57:21.139 INFO:teuthology.orchestra.run.smithi008.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-19T15:57:21.228 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-02-19T15:57:21.236 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-02-19T15:57:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-02-19T15:57:21.236 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-02-19T15:57:21.236 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-02-19T15:57:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-19T15:57:21.236 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout:Install 4 Packages 2024-02-19T15:57:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:57:21.238 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 100 M 2024-02-19T15:57:21.238 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 123 M 2024-02-19T15:57:21.238 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-02-19T15:57:21.742 INFO:teuthology.orchestra.run.smithi008.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 12 MB/s | 6.1 MB 00:00 2024-02-19T15:57:22.334 INFO:teuthology.orchestra.run.smithi008.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-19T15:57:23.151 INFO:teuthology.orchestra.run.smithi008.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-19T15:57:23.551 INFO:teuthology.orchestra.run.smithi008.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 19 MB/s | 33 MB 00:01 2024-02-19T15:57:23.554 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-02-19T15:57:23.555 INFO:teuthology.orchestra.run.smithi008.stdout:Total 43 MB/s | 100 MB 00:02 2024-02-19T15:57:23.555 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-02-19T15:57:23.986 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-02-19T15:57:23.986 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-02-19T15:57:25.068 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-02-19T15:57:25.069 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-02-19T15:57:27.666 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-02-19T15:57:28.367 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-19T15:57:28.503 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T15:57:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T15:57:31.599 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T15:57:36.671 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T15:57:36.780 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-19T15:57:41.873 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T15:58:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T15:58:12.528 INFO:teuthology.orchestra.run.smithi008.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-19T15:58:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:58:12.600 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-19T15:58:12.996 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-19T15:58:12.996 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-19T15:58:12.996 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T15:58:12.996 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T15:58:13.421 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T15:58:13.421 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:58:13.421 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-02-19T15:58:13.421 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-19T15:58:13.422 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-19T15:58:13.422 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-19T15:58:13.422 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-19T15:58:13.422 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:58:13.422 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-02-19T15:58:13.926 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q kernel | sort -rV | head -n 1 2024-02-19T15:58:14.068 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-19T15:58:14.068 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-19T15:58:14.069 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-19T15:58:14.069 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-19T15:58:14.069 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-19T15:58:14.069 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-19T15:58:14.069 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi008.front.sepia.ceph.com, path=None, version=distro) 2024-02-19T15:58:14.069 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q kernel | sort -rV | head -n 1 2024-02-19T15:58:14.181 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-19T15:58:14.182 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-19T15:58:14.182 DEBUG:teuthology.orchestra.run.smithi008:> sudo rpm -qi grub2-tools 2024-02-19T15:58:14.219 INFO:teuthology.orchestra.run.smithi008.stdout:Name : grub2-tools 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Epoch : 1 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Version : 2.06 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Release : 68.el9 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Architecture: x86_64 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Group : Unspecified 2024-02-19T15:58:14.220 INFO:teuthology.orchestra.run.smithi008.stdout:Size : 8264254 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:License : GPLv3+ 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:Packager : builder@centos.org 2024-02-19T15:58:14.221 INFO:teuthology.orchestra.run.smithi008.stdout:Vendor : CentOS 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout:Summary : Support tools for GRUB. 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout:Description : 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-19T15:58:14.222 INFO:teuthology.orchestra.run.smithi008.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-19T15:58:14.223 INFO:teuthology.orchestra.run.smithi008.stdout:hardware devices. 2024-02-19T15:58:14.223 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T15:58:14.223 INFO:teuthology.orchestra.run.smithi008.stdout:This subpackage provides tools for support of all platforms. 2024-02-19T15:58:14.224 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-19T15:58:14.225 INFO:teuthology.task.kernel:Updating grub on smithi008 to boot 5.14.0-421.el9.x86_64 2024-02-19T15:58:14.225 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-19T15:58:14.801 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2024-02-19T15:58:16.124 INFO:teuthology.orchestra.run.smithi008.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-19T15:58:16.181 INFO:teuthology.orchestra.run.smithi008.stderr:done 2024-02-19T15:58:16.183 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2024-02-19T15:58:16.198 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.nPQnEaWjWf 2024-02-19T15:58:16.198 DEBUG:teuthology.orchestra.run.smithi008:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nPQnEaWjWf 2024-02-19T15:58:16.270 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.nPQnEaWjWf 2024-02-19T15:58:16.411 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.nPQnEaWjWf is 6KB 2024-02-19T15:58:16.432 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.nPQnEaWjWf 2024-02-19T15:58:16.447 DEBUG:teuthology.orchestra.run.smithi008:> sudo /bin/ls /boot/loader/entries || true 2024-02-19T15:58:16.515 INFO:teuthology.orchestra.run.smithi008.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-19T15:58:16.515 INFO:teuthology.orchestra.run.smithi008.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-19T15:58:16.515 INFO:teuthology.orchestra.run.smithi008.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-19T15:58:16.515 INFO:teuthology.orchestra.run.smithi008.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-19T15:58:16.516 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-19T15:58:16.659 DEBUG:teuthology.orchestra.run.smithi008:> sudo shutdown -r now 2024-02-19T15:58:46.662 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-19T15:58:46.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T15:58:46.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T15:59:46.667 DEBUG:teuthology.orchestra.remote:timed out 2024-02-19T15:59:55.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T15:59:55.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T15:59:56.194 DEBUG:teuthology.orchestra.run.smithi008:> true 2024-02-19T15:59:56.563 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T15:59:56.563 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-421.el9.x86_64"... 2024-02-19T15:59:56.563 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2024-02-19T15:59:56.577 INFO:teuthology.orchestra.run.smithi008.stdout:5.14.0-421.el9.x86_64 2024-02-19T15:59:56.577 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-19T15:59:56.577 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-19T15:59:56.577 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-19T15:59:57.578 DEBUG:teuthology.parallel:result is None 2024-02-19T15:59:57.579 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-19T15:59:57.586 INFO:teuthology.task.internal:Creating test directory... 2024-02-19T15:59:57.587 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-19T15:59:57.589 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-19T15:59:57.592 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-19T15:59:57.610 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-19T15:59:57.616 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-19T15:59:57.626 INFO:teuthology.task.internal:Creating archive directory... 2024-02-19T15:59:57.626 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-19T15:59:57.642 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-19T15:59:57.649 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-19T15:59:57.686 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-19T15:59:57.692 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-19T15:59:57.692 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-19T15:59:57.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T15:59:57.729 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-19T15:59:57.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T15:59:57.745 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-19T15:59:57.747 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-19T15:59:57.771 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-19T15:59:57.803 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T15:59:57.826 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T15:59:57.827 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T15:59:57.842 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T15:59:57.850 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T15:59:57.873 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T15:59:57.875 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-19T15:59:57.882 INFO:teuthology.task.internal:Configuring sudo... 2024-02-19T15:59:57.882 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-19T15:59:57.884 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-19T15:59:57.887 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-19T15:59:57.921 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-19T15:59:57.930 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-19T15:59:57.930 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-19T15:59:57.950 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-19T15:59:57.960 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-19T15:59:57.977 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T15:59:58.040 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T15:59:58.079 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T15:59:58.158 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T15:59:58.196 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T15:59:58.196 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-19T15:59:58.259 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T15:59:58.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T15:59:58.363 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T15:59:58.407 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T15:59:58.450 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T15:59:58.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-19T15:59:58.519 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T15:59:58.563 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T15:59:58.662 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T15:59:58.705 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T15:59:58.750 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T15:59:58.750 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-19T15:59:58.820 DEBUG:teuthology.orchestra.run.smithi008:> sudo service rsyslog restart 2024-02-19T15:59:58.822 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-02-19T15:59:58.824 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-02-19T15:59:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T15:59:58.878 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T15:59:58.908 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T15:59:59.359 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-19T15:59:59.367 INFO:teuthology.task.internal:Starting timer... 2024-02-19T15:59:59.367 INFO:teuthology.run_tasks:Running task pcp... 2024-02-19T15:59:59.377 INFO:teuthology.run_tasks:Running task selinux... 2024-02-19T15:59:59.385 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-19T15:59:59.386 DEBUG:teuthology.orchestra.run.smithi008:> sudo service auditd rotate 2024-02-19T15:59:59.490 INFO:teuthology.orchestra.run.smithi008.stdout:Rotating logs: 2024-02-19T15:59:59.492 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2024-02-19T15:59:59.543 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2024-02-19T15:59:59.546 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-02-19T15:59:59.606 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-02-19T15:59:59.608 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-19T15:59:59.609 DEBUG:teuthology.orchestra.run.smithi008:> /usr/sbin/getenforce 2024-02-19T15:59:59.625 INFO:teuthology.orchestra.run.smithi008.stdout:Permissive 2024-02-19T15:59:59.625 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2024-02-19T15:59:59.662 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2024-02-19T15:59:59.662 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-02-19T15:59:59.682 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-02-19T15:59:59.682 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi008.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-02-19T15:59:59.683 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T15:59:59.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T15:59:59.710 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T15:59:59.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T15:59:59.739 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T15:59:59.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T15:59:59.767 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-19T15:59:59.767 DEBUG:teuthology.orchestra.run.smithi008:> sudo /usr/sbin/setenforce permissive 2024-02-19T15:59:59.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2024-02-19T15:59:59.839 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-02-19T15:59:59.874 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-19T15:59:59.884 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-19T15:59:59.891 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-19T16:00:00.039 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-19T16:00:00.055 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-19T16:00:00.057 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi008.front.sepia.ceph.com,smithi156.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-19T16:09:03.009 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi008.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-02-19T16:09:03.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T16:09:03.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T16:09:03.099 DEBUG:teuthology.orchestra.run.smithi008:> true 2024-02-19T16:09:03.174 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T16:09:03.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-02-19T16:09:03.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T16:09:03.268 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-02-19T16:09:03.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-02-19T16:09:03.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-19T16:09:03.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T16:09:03.429 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-19T16:09:03.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-19T16:09:03.455 INFO:teuthology.run_tasks:Running task clock... 2024-02-19T16:09:03.474 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-19T16:09:03.474 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-19T16:09:03.474 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T16:09:03.476 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-19T16:09:03.477 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T16:09:03.479 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-19T16:09:03.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T16:09:03.506 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-19T16:09:03.507 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-19T16:09:03.523 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-19T16:09:03.524 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-19T16:09:03.525 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-19T16:09:03.542 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-19T16:09:03.552 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2024-02-19T16:09:03.554 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: ntpd: command not found 2024-02-19T16:09:03.566 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2024-02-19T16:09:03.568 INFO:teuthology.orchestra.run.smithi008.stdout:506 Cannot talk to daemon 2024-02-19T16:09:03.571 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-02-19T16:09:03.582 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-19T16:09:03.585 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-02-19T16:09:03.585 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-19T16:09:03.599 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-19T16:09:03.601 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-19T16:09:03.601 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-19T16:09:03.618 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-19T16:09:03.643 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-02-19T16:09:03.646 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T16:09:03.647 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-02-19T16:09:03.647 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.647 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.647 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.647 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.661 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-02-19T16:09:03.664 INFO:teuthology.orchestra.run.smithi008.stderr:bash: line 1: ntpq: command not found 2024-02-19T16:09:03.664 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T16:09:03.664 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-19T16:09:03.665 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.665 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.665 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.665 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.667 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T16:09:03.667 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2024-02-19T16:09:03.667 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.667 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.667 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.667 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:09:03.668 INFO:teuthology.run_tasks:Running task install... 2024-02-19T16:09:03.678 DEBUG:teuthology.task.install:project ceph 2024-02-19T16:09:03.678 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee'}} 2024-02-19T16:09:03.678 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee'} 2024-02-19T16:09:03.678 INFO:teuthology.task.install:Using flavor: default 2024-02-19T16:09:03.687 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-19T16:09:03.687 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-19T16:09:03.687 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee', 'tag': None, 'wait_for_package': False} 2024-02-19T16:09:03.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:09:03.690 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee', 'tag': None, 'wait_for_package': False} 2024-02-19T16:09:03.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:09:03.692 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee', 'tag': None, 'wait_for_package': False} 2024-02-19T16:09:03.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:09:03.843 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/ 2024-02-19T16:09:03.843 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1451.g9f8908ee 2024-02-19T16:09:03.846 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/ 2024-02-19T16:09:03.846 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1451.g9f8908ee 2024-02-19T16:09:03.935 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/ 2024-02-19T16:09:03.936 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1451.g9f8908ee 2024-02-19T16:09:03.958 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-19T16:09:03.959 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:09:03.959 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-19T16:09:03.987 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-19T16:09:03.987 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-19T16:09:04.017 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-19T16:09:04.017 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:09:04.017 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-19T16:09:04.046 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-19T16:09:04.047 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:09:04.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-19T16:09:04.048 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-19T16:09:04.048 DEBUG:teuthology.orchestra.run.smithi008:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-19T16:09:04.054 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-19T16:09:04.076 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-19T16:09:04.076 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-19T16:09:04.116 DEBUG:teuthology.orchestra.run.smithi008:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-19T16:09:04.134 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-19T16:09:04.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-19T16:09:04.196 DEBUG:teuthology.orchestra.run.smithi008:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-19T16:09:04.200 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-02-19T16:09:04.202 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-02-19T16:09:04.223 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-19T16:09:04.262 INFO:teuthology.orchestra.run.smithi008.stdout:check_obsoletes = 1 2024-02-19T16:09:04.263 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum clean all 2024-02-19T16:09:04.289 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2024-02-19T16:09:04.291 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2024-02-19T16:09:04.437 INFO:teuthology.orchestra.run.smithi191.stdout:74 files removed 2024-02-19T16:09:04.464 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-19T16:09:04.518 INFO:teuthology.orchestra.run.smithi008.stdout:74 files removed 2024-02-19T16:09:04.526 INFO:teuthology.orchestra.run.smithi156.stdout:74 files removed 2024-02-19T16:09:04.546 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-19T16:09:04.554 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-19T16:09:05.161 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 205 kB/s | 83 kB 00:00 2024-02-19T16:09:05.274 INFO:teuthology.orchestra.run.smithi008.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-02-19T16:09:05.298 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 184 kB/s | 83 kB 00:00 2024-02-19T16:09:05.545 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-02-19T16:09:05.666 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-02-19T16:09:05.674 INFO:teuthology.orchestra.run.smithi008.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-02-19T16:09:05.971 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-02-19T16:09:05.992 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-02-19T16:09:06.092 INFO:teuthology.orchestra.run.smithi008.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-02-19T16:09:06.449 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-19T16:09:06.769 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-02-19T16:09:06.877 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 7.8 MB/s | 6.7 MB 00:00 2024-02-19T16:09:07.916 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2024-02-19T16:09:08.261 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-02-19T16:09:08.361 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-02-19T16:09:08.866 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-02-19T16:09:09.176 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-02-19T16:09:10.460 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-02-19T16:09:11.044 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-02-19T16:09:12.345 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-02-19T16:09:15.017 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 19 MB 00:05 2024-02-19T16:09:16.588 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-02-19T16:09:18.947 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 15 kB 00:01 2024-02-19T16:09:19.620 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 6.1 MB 00:05 2024-02-19T16:09:20.276 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2024-02-19T16:09:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-02-19T16:09:22.316 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2024-02-19T16:09:23.100 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.1 MB 00:04 2024-02-19T16:09:24.493 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 15 kB 00:00 2024-02-19T16:09:25.474 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-02-19T16:09:26.445 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-19T16:09:27.221 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-02-19T16:09:28.105 INFO:teuthology.orchestra.run.smithi008.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-19T16:09:28.106 INFO:teuthology.orchestra.run.smithi008.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-19T16:09:28.120 INFO:teuthology.orchestra.run.smithi008.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-19T16:09:28.121 INFO:teuthology.orchestra.run.smithi008.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-19T16:09:28.122 INFO:teuthology.orchestra.run.smithi008.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-19T16:09:28.123 INFO:teuthology.orchestra.run.smithi008.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-19T16:09:28.125 INFO:teuthology.orchestra.run.smithi008.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-19T16:09:28.128 INFO:teuthology.orchestra.run.smithi008.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-19T16:09:28.129 INFO:teuthology.orchestra.run.smithi008.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-19T16:09:28.214 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================ 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================ 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout: ceph x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 6.6 k 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 5.3 M 2024-02-19T16:09:28.227 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 887 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 145 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 1.5 M 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 138 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 1.8 M 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 7.4 M 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 52 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 9.7 M 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 47 M 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-volume noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 264 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 748 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-mirror x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 217 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-shell x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 27 k 2024-02-19T16:09:28.228 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-top noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 18 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 31 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 744 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 127 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-19T16:09:28.229 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 162 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 320 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 298 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 99 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 85 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.1 M 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 171 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrading: 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-19T16:09:28.230 INFO:teuthology.orchestra.run.smithi008.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: librados2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.4 M 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.1 M 2024-02-19T16:09:28.231 INFO:teuthology.orchestra.run.smithi008.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 20 M 2024-02-19T16:09:28.232 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 31 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 2.3 M 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 248 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 4.7 M 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 17 M 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 15 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 25 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-19T16:09:28.233 INFO:teuthology.orchestra.run.smithi008.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 163 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 498 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 5.1 M 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-19T16:09:28.234 INFO:teuthology.orchestra.run.smithi008.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-19T16:09:28.235 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 45 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 131 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-19T16:09:28.236 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-19T16:09:28.237 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-19T16:09:28.238 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-19T16:09:28.239 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout:Installing weak dependencies: 2024-02-19T16:09:28.240 INFO:teuthology.orchestra.run.smithi008.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================ 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout:Install 150 Packages 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrade 26 Packages 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:09:28.241 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 223 M 2024-02-19T16:09:28.242 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-02-19T16:09:28.983 INFO:teuthology.orchestra.run.smithi008.stdout:(1/176): ceph-19.0.0-1451.g9f8908ee.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-02-19T16:09:29.267 INFO:teuthology.orchestra.run.smithi008.stdout:(2/176): ceph-fuse-19.0.0-1451.g9f8908ee.el9.x8 3.1 MB/s | 887 kB 00:00 2024-02-19T16:09:29.325 INFO:teuthology.orchestra.run.smithi008.stdout:(3/176): ceph-immutable-object-cache-19.0.0-145 2.4 MB/s | 145 kB 00:00 2024-02-19T16:09:29.659 INFO:teuthology.orchestra.run.smithi008.stdout:(4/176): ceph-mds-19.0.0-1451.g9f8908ee.el9.x86 6.9 MB/s | 2.3 MB 00:00 2024-02-19T16:09:29.712 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-19T16:09:29.713 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-19T16:09:29.725 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-19T16:09:29.726 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-19T16:09:29.727 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-19T16:09:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-19T16:09:29.730 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-19T16:09:29.732 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-19T16:09:29.733 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-19T16:09:29.792 INFO:teuthology.orchestra.run.smithi008.stdout:(5/176): ceph-base-19.0.0-1451.g9f8908ee.el9.x8 5.5 MB/s | 5.3 MB 00:00 2024-02-19T16:09:29.814 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 6.6 k 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 5.3 M 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 887 k 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 145 k 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 1.5 M 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 138 k 2024-02-19T16:09:29.836 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 1.8 M 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 7.4 M 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 52 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 9.7 M 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 47 M 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 264 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 748 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 217 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 27 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 18 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 31 k 2024-02-19T16:09:29.837 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 744 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 127 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 162 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 320 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 298 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 99 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 85 k 2024-02-19T16:09:29.838 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.1 M 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 171 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-19T16:09:29.839 INFO:teuthology.orchestra.run.smithi156.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.4 M 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.1 M 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-19T16:09:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 20 M 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 31 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 2.3 M 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 248 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 4.7 M 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 17 M 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 15 k 2024-02-19T16:09:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 25 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 163 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-19T16:09:29.842 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 498 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 5.1 M 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-19T16:09:29.843 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 45 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 131 k 2024-02-19T16:09:29.844 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-19T16:09:29.845 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-19T16:09:29.846 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-19T16:09:29.847 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-19T16:09:29.848 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:Installing weak dependencies: 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:Install 150 Packages 2024-02-19T16:09:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 26 Packages 2024-02-19T16:09:29.850 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:09:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 223 M 2024-02-19T16:09:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-02-19T16:09:29.918 INFO:teuthology.orchestra.run.smithi008.stdout:(6/176): ceph-mgr-19.0.0-1451.g9f8908ee.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-02-19T16:09:30.228 INFO:teuthology.orchestra.run.smithi008.stdout:(7/176): ceph-mon-19.0.0-1451.g9f8908ee.el9.x86 11 MB/s | 4.7 MB 00:00 2024-02-19T16:09:30.847 INFO:teuthology.orchestra.run.smithi156.stdout:(1/176): ceph-19.0.0-1451.g9f8908ee.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-02-19T16:09:31.068 INFO:teuthology.orchestra.run.smithi008.stdout:(8/176): ceph-radosgw-19.0.0-1451.g9f8908ee.el9 12 MB/s | 9.7 MB 00:00 2024-02-19T16:09:31.135 INFO:teuthology.orchestra.run.smithi008.stdout:(9/176): ceph-selinux-19.0.0-1451.g9f8908ee.el9 378 kB/s | 25 kB 00:00 2024-02-19T16:09:31.370 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-02-19T16:09:31.469 INFO:teuthology.orchestra.run.smithi008.stdout:(10/176): ceph-osd-19.0.0-1451.g9f8908ee.el9.x8 11 MB/s | 17 MB 00:01 2024-02-19T16:09:31.472 INFO:teuthology.orchestra.run.smithi156.stdout:(2/176): ceph-fuse-19.0.0-1451.g9f8908ee.el9.x8 1.4 MB/s | 887 kB 00:00 2024-02-19T16:09:31.535 INFO:teuthology.orchestra.run.smithi008.stdout:(11/176): cephfs-mirror-19.0.0-1451.g9f8908ee.e 3.2 MB/s | 217 kB 00:00 2024-02-19T16:09:31.592 INFO:teuthology.orchestra.run.smithi156.stdout:(3/176): ceph-immutable-object-cache-19.0.0-145 1.2 MB/s | 145 kB 00:00 2024-02-19T16:09:31.603 INFO:teuthology.orchestra.run.smithi008.stdout:(12/176): cephfs-shell-19.0.0-1451.g9f8908ee.el 405 kB/s | 27 kB 00:00 2024-02-19T16:09:31.662 INFO:teuthology.orchestra.run.smithi008.stdout:(13/176): libcephfs-devel-19.0.0-1451.g9f8908ee 534 kB/s | 31 kB 00:00 2024-02-19T16:09:31.862 INFO:teuthology.orchestra.run.smithi008.stdout:(14/176): libcephfs2-19.0.0-1451.g9f8908ee.el9. 3.6 MB/s | 744 kB 00:00 2024-02-19T16:09:31.945 INFO:teuthology.orchestra.run.smithi008.stdout:(15/176): libcephsqlite-19.0.0-1451.g9f8908ee.e 1.9 MB/s | 163 kB 00:00 2024-02-19T16:09:32.020 INFO:teuthology.orchestra.run.smithi008.stdout:(16/176): librados-devel-19.0.0-1451.g9f8908ee. 1.7 MB/s | 127 kB 00:00 2024-02-19T16:09:32.171 INFO:teuthology.orchestra.run.smithi008.stdout:(17/176): libradosstriper1-19.0.0-1451.g9f8908e 3.2 MB/s | 498 kB 00:00 2024-02-19T16:09:32.456 INFO:teuthology.orchestra.run.smithi156.stdout:(4/176): ceph-mds-19.0.0-1451.g9f8908ee.el9.x86 2.7 MB/s | 2.3 MB 00:00 2024-02-19T16:09:33.006 INFO:teuthology.orchestra.run.smithi156.stdout:(5/176): ceph-mgr-19.0.0-1451.g9f8908ee.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-02-19T16:09:33.238 INFO:teuthology.orchestra.run.smithi008.stdout:(18/176): librgw2-19.0.0-1451.g9f8908ee.el9.x86 4.8 MB/s | 5.1 MB 00:01 2024-02-19T16:09:33.290 INFO:teuthology.orchestra.run.smithi156.stdout:(6/176): ceph-base-19.0.0-1451.g9f8908ee.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-02-19T16:09:33.313 INFO:teuthology.orchestra.run.smithi008.stdout:(19/176): python3-ceph-argparse-19.0.0-1451.g9f 602 kB/s | 45 kB 00:00 2024-02-19T16:09:33.413 INFO:teuthology.orchestra.run.smithi008.stdout:(20/176): python3-ceph-common-19.0.0-1451.g9f89 1.3 MB/s | 131 kB 00:00 2024-02-19T16:09:33.513 INFO:teuthology.orchestra.run.smithi008.stdout:(21/176): python3-cephfs-19.0.0-1451.g9f8908ee. 1.6 MB/s | 162 kB 00:00 2024-02-19T16:09:33.622 INFO:teuthology.orchestra.run.smithi008.stdout:(22/176): python3-rados-19.0.0-1451.g9f8908ee.e 2.9 MB/s | 320 kB 00:00 2024-02-19T16:09:33.722 INFO:teuthology.orchestra.run.smithi008.stdout:(23/176): python3-rbd-19.0.0-1451.g9f8908ee.el9 2.9 MB/s | 298 kB 00:00 2024-02-19T16:09:33.806 INFO:teuthology.orchestra.run.smithi008.stdout:(24/176): python3-rgw-19.0.0-1451.g9f8908ee.el9 1.2 MB/s | 99 kB 00:00 2024-02-19T16:09:33.890 INFO:teuthology.orchestra.run.smithi008.stdout:(25/176): rbd-fuse-19.0.0-1451.g9f8908ee.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-19T16:09:33.895 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-19T16:09:33.896 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-19T16:09:33.908 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-19T16:09:33.909 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-19T16:09:33.910 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-19T16:09:33.911 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-19T16:09:33.912 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-19T16:09:33.915 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-19T16:09:33.916 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-19T16:09:33.996 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-19T16:09:34.013 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-02-19T16:09:34.013 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-19T16:09:34.013 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-02-19T16:09:34.013 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 6.6 k 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 5.3 M 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 887 k 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 145 k 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 1.5 M 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 138 k 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 1.8 M 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 7.4 M 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 52 k 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 9.7 M 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 47 M 2024-02-19T16:09:34.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 264 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 748 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 217 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 27 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 18 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 31 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 744 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 127 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-19T16:09:34.015 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 162 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 320 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 298 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 99 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 85 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.1 M 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 171 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-19T16:09:34.016 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-19T16:09:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.4 M 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 3.1 M 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-19T16:09:34.018 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 20 M 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 31 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 2.3 M 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 248 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 4.7 M 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 17 M 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1451.g9f8908ee.el9 ceph-noarch 15 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 25 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-19T16:09:34.019 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 163 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 498 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 5.1 M 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-19T16:09:34.020 INFO:teuthology.orchestra.run.smithi191.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-19T16:09:34.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 45 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:19.0.0-1451.g9f8908ee.el9 ceph 131 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-19T16:09:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-19T16:09:34.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-19T16:09:34.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-19T16:09:34.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-19T16:09:34.026 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-02-19T16:09:34.027 INFO:teuthology.orchestra.run.smithi191.stdout:Install 150 Packages 2024-02-19T16:09:34.027 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 26 Packages 2024-02-19T16:09:34.027 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:09:34.027 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 223 M 2024-02-19T16:09:34.027 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-19T16:09:34.648 INFO:teuthology.orchestra.run.smithi191.stdout:(1/176): ceph-19.0.0-1451.g9f8908ee.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-02-19T16:09:34.665 INFO:teuthology.orchestra.run.smithi156.stdout:(7/176): ceph-mon-19.0.0-1451.g9f8908ee.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-02-19T16:09:34.692 INFO:teuthology.orchestra.run.smithi008.stdout:(26/176): ceph-common-19.0.0-1451.g9f8908ee.el9 3.5 MB/s | 20 MB 00:05 2024-02-19T16:09:34.740 INFO:teuthology.orchestra.run.smithi008.stdout:(27/176): rbd-mirror-19.0.0-1451.g9f8908ee.el9. 3.6 MB/s | 3.1 MB 00:00 2024-02-19T16:09:34.799 INFO:teuthology.orchestra.run.smithi008.stdout:(28/176): rbd-nbd-19.0.0-1451.g9f8908ee.el9.x86 1.5 MB/s | 171 kB 00:00 2024-02-19T16:09:34.841 INFO:teuthology.orchestra.run.smithi008.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1451.g 313 kB/s | 31 kB 00:00 2024-02-19T16:09:34.916 INFO:teuthology.orchestra.run.smithi008.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1451.g9f8908e 1.2 MB/s | 138 kB 00:00 2024-02-19T16:09:35.514 INFO:teuthology.orchestra.run.smithi191.stdout:(2/176): ceph-fuse-19.0.0-1451.g9f8908ee.el9.x8 1.0 MB/s | 887 kB 00:00 2024-02-19T16:09:35.583 INFO:teuthology.orchestra.run.smithi008.stdout:(31/176): ceph-test-19.0.0-1451.g9f8908ee.el9.x 11 MB/s | 47 MB 00:04 2024-02-19T16:09:35.625 INFO:teuthology.orchestra.run.smithi008.stdout:(32/176): ceph-mgr-dashboard-19.0.0-1451.g9f890 2.3 MB/s | 1.8 MB 00:00 2024-02-19T16:09:35.648 INFO:teuthology.orchestra.run.smithi191.stdout:(3/176): ceph-immutable-object-cache-19.0.0-145 1.1 MB/s | 145 kB 00:00 2024-02-19T16:09:35.658 INFO:teuthology.orchestra.run.smithi008.stdout:(33/176): ceph-mgr-modules-core-19.0.0-1451.g9f 3.2 MB/s | 248 kB 00:00 2024-02-19T16:09:35.700 INFO:teuthology.orchestra.run.smithi008.stdout:(34/176): ceph-mgr-rook-19.0.0-1451.g9f8908ee.e 695 kB/s | 52 kB 00:00 2024-02-19T16:09:35.734 INFO:teuthology.orchestra.run.smithi008.stdout:(35/176): ceph-prometheus-alerts-19.0.0-1451.g9 204 kB/s | 15 kB 00:00 2024-02-19T16:09:35.834 INFO:teuthology.orchestra.run.smithi008.stdout:(36/176): ceph-volume-19.0.0-1451.g9f8908ee.el9 1.9 MB/s | 264 kB 00:00 2024-02-19T16:09:35.880 INFO:teuthology.orchestra.run.smithi008.stdout:(37/176): cephadm-19.0.0-1451.g9f8908ee.el9.noa 5.1 MB/s | 748 kB 00:00 2024-02-19T16:09:35.901 INFO:teuthology.orchestra.run.smithi008.stdout:(38/176): cephfs-top-19.0.0-1451.g9f8908ee.el9. 273 kB/s | 18 kB 00:00 2024-02-19T16:09:36.115 INFO:teuthology.orchestra.run.smithi156.stdout:(8/176): ceph-osd-19.0.0-1451.g9f8908ee.el9.x86 5.9 MB/s | 17 MB 00:02 2024-02-19T16:09:36.131 INFO:teuthology.orchestra.run.smithi191.stdout:(4/176): ceph-base-19.0.0-1451.g9f8908ee.el9.x8 2.8 MB/s | 5.3 MB 00:01 2024-02-19T16:09:36.135 INFO:teuthology.orchestra.run.smithi008.stdout:(39/176): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-02-19T16:09:36.202 INFO:teuthology.orchestra.run.smithi008.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-02-19T16:09:36.215 INFO:teuthology.orchestra.run.smithi156.stdout:(9/176): ceph-selinux-19.0.0-1451.g9f8908ee.el9 252 kB/s | 25 kB 00:00 2024-02-19T16:09:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:(41/176): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2024-02-19T16:09:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 304 kB/s | 20 kB 00:00 2024-02-19T16:09:36.295 INFO:teuthology.orchestra.run.smithi008.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-02-19T16:09:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 630 kB/s | 32 kB 00:00 2024-02-19T16:09:36.398 INFO:teuthology.orchestra.run.smithi191.stdout:(5/176): ceph-mgr-19.0.0-1451.g9f8908ee.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-02-19T16:09:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-02-19T16:09:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-02-19T16:09:36.487 INFO:teuthology.orchestra.run.smithi008.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-02-19T16:09:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-19T16:09:36.546 INFO:teuthology.orchestra.run.smithi008.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-02-19T16:09:36.672 INFO:teuthology.orchestra.run.smithi008.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 661 kB 00:00 2024-02-19T16:09:36.705 INFO:teuthology.orchestra.run.smithi008.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 578 kB 00:00 2024-02-19T16:09:36.722 INFO:teuthology.orchestra.run.smithi008.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-02-19T16:09:36.748 INFO:teuthology.orchestra.run.smithi008.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-02-19T16:09:36.798 INFO:teuthology.orchestra.run.smithi008.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-19T16:09:36.831 INFO:teuthology.orchestra.run.smithi008.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-02-19T16:09:36.873 INFO:teuthology.orchestra.run.smithi008.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 453 kB/s | 34 kB 00:00 2024-02-19T16:09:37.040 INFO:teuthology.orchestra.run.smithi191.stdout:(6/176): ceph-mds-19.0.0-1451.g9f8908ee.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-02-19T16:09:37.041 INFO:teuthology.orchestra.run.smithi008.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-02-19T16:09:37.124 INFO:teuthology.orchestra.run.smithi008.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-02-19T16:09:37.132 INFO:teuthology.orchestra.run.smithi191.stdout:(7/176): ceph-mon-19.0.0-1451.g9f8908ee.el9.x86 6.4 MB/s | 4.7 MB 00:00 2024-02-19T16:09:37.191 INFO:teuthology.orchestra.run.smithi008.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-02-19T16:09:37.241 INFO:teuthology.orchestra.run.smithi008.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 6.8 MB/s | 811 kB 00:00 2024-02-19T16:09:37.266 INFO:teuthology.orchestra.run.smithi008.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 1.7 MB/s | 134 kB 00:00 2024-02-19T16:09:37.417 INFO:teuthology.orchestra.run.smithi008.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-02-19T16:09:37.459 INFO:teuthology.orchestra.run.smithi008.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-02-19T16:09:37.482 INFO:teuthology.orchestra.run.smithi156.stdout:(10/176): ceph-radosgw-19.0.0-1451.g9f8908ee.el 3.5 MB/s | 9.7 MB 00:02 2024-02-19T16:09:37.484 INFO:teuthology.orchestra.run.smithi008.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-02-19T16:09:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:(11/176): cephfs-mirror-19.0.0-1451.g9f8908ee.e 2.3 MB/s | 217 kB 00:00 2024-02-19T16:09:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:(12/176): cephfs-shell-19.0.0-1451.g9f8908ee.el 651 kB/s | 27 kB 00:00 2024-02-19T16:09:37.666 INFO:teuthology.orchestra.run.smithi156.stdout:(13/176): libcephfs-devel-19.0.0-1451.g9f8908ee 623 kB/s | 31 kB 00:00 2024-02-19T16:09:37.725 INFO:teuthology.orchestra.run.smithi191.stdout:(8/176): ceph-common-19.0.0-1451.g9f8908ee.el9. 5.8 MB/s | 20 MB 00:03 2024-02-19T16:09:37.726 INFO:teuthology.orchestra.run.smithi008.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-02-19T16:09:37.751 INFO:teuthology.orchestra.run.smithi008.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 851 kB/s | 227 kB 00:00 2024-02-19T16:09:37.783 INFO:teuthology.orchestra.run.smithi191.stdout:(9/176): ceph-selinux-19.0.0-1451.g9f8908ee.el9 434 kB/s | 25 kB 00:00 2024-02-19T16:09:37.801 INFO:teuthology.orchestra.run.smithi008.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 364 kB/s | 18 kB 00:00 2024-02-19T16:09:37.816 INFO:teuthology.orchestra.run.smithi156.stdout:(14/176): libcephfs2-19.0.0-1451.g9f8908ee.el9. 4.9 MB/s | 744 kB 00:00 2024-02-19T16:09:37.827 INFO:teuthology.orchestra.run.smithi008.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 325 kB/s | 24 kB 00:00 2024-02-19T16:09:37.852 INFO:teuthology.orchestra.run.smithi008.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-19T16:09:37.877 INFO:teuthology.orchestra.run.smithi008.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-02-19T16:09:37.892 INFO:teuthology.orchestra.run.smithi156.stdout:(15/176): libcephsqlite-19.0.0-1451.g9f8908ee.e 2.1 MB/s | 163 kB 00:00 2024-02-19T16:09:37.967 INFO:teuthology.orchestra.run.smithi156.stdout:(16/176): librados-devel-19.0.0-1451.g9f8908ee. 1.7 MB/s | 127 kB 00:00 2024-02-19T16:09:38.069 INFO:teuthology.orchestra.run.smithi008.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-02-19T16:09:38.120 INFO:teuthology.orchestra.run.smithi008.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 899 kB/s | 217 kB 00:00 2024-02-19T16:09:38.145 INFO:teuthology.orchestra.run.smithi008.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-02-19T16:09:38.167 INFO:teuthology.orchestra.run.smithi156.stdout:(17/176): libradosstriper1-19.0.0-1451.g9f8908e 2.4 MB/s | 498 kB 00:00 2024-02-19T16:09:38.170 INFO:teuthology.orchestra.run.smithi008.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-19T16:09:38.195 INFO:teuthology.orchestra.run.smithi008.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-02-19T16:09:38.221 INFO:teuthology.orchestra.run.smithi008.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-19T16:09:38.246 INFO:teuthology.orchestra.run.smithi008.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-19T16:09:38.271 INFO:teuthology.orchestra.run.smithi008.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-19T16:09:38.297 INFO:teuthology.orchestra.run.smithi008.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-02-19T16:09:38.322 INFO:teuthology.orchestra.run.smithi008.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-02-19T16:09:38.350 INFO:teuthology.orchestra.run.smithi191.stdout:(10/176): ceph-radosgw-19.0.0-1451.g9f8908ee.el 8.0 MB/s | 9.7 MB 00:01 2024-02-19T16:09:38.401 INFO:teuthology.orchestra.run.smithi156.stdout:(18/176): ceph-common-19.0.0-1451.g9f8908ee.el9 2.6 MB/s | 20 MB 00:07 2024-02-19T16:09:38.417 INFO:teuthology.orchestra.run.smithi191.stdout:(11/176): cephfs-mirror-19.0.0-1451.g9f8908ee.e 3.2 MB/s | 217 kB 00:00 2024-02-19T16:09:38.484 INFO:teuthology.orchestra.run.smithi191.stdout:(12/176): cephfs-shell-19.0.0-1451.g9f8908ee.el 406 kB/s | 27 kB 00:00 2024-02-19T16:09:38.535 INFO:teuthology.orchestra.run.smithi156.stdout:(19/176): python3-ceph-argparse-19.0.0-1451.g9f 339 kB/s | 45 kB 00:00 2024-02-19T16:09:38.568 INFO:teuthology.orchestra.run.smithi191.stdout:(13/176): libcephfs-devel-19.0.0-1451.g9f8908ee 374 kB/s | 31 kB 00:00 2024-02-19T16:09:38.598 INFO:teuthology.orchestra.run.smithi008.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 141 kB/s | 39 kB 00:00 2024-02-19T16:09:38.726 INFO:teuthology.orchestra.run.smithi156.stdout:(20/176): python3-ceph-common-19.0.0-1451.g9f89 686 kB/s | 131 kB 00:00 2024-02-19T16:09:38.793 INFO:teuthology.orchestra.run.smithi191.stdout:(14/176): libcephfs2-19.0.0-1451.g9f8908ee.el9. 3.2 MB/s | 744 kB 00:00 2024-02-19T16:09:38.848 INFO:teuthology.orchestra.run.smithi008.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-02-19T16:09:38.860 INFO:teuthology.orchestra.run.smithi156.stdout:(21/176): python3-cephfs-19.0.0-1451.g9f8908ee. 1.2 MB/s | 162 kB 00:00 2024-02-19T16:09:38.873 INFO:teuthology.orchestra.run.smithi008.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 150 kB/s | 41 kB 00:00 2024-02-19T16:09:38.877 INFO:teuthology.orchestra.run.smithi191.stdout:(15/176): libcephsqlite-19.0.0-1451.g9f8908ee.e 1.9 MB/s | 163 kB 00:00 2024-02-19T16:09:38.898 INFO:teuthology.orchestra.run.smithi008.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-19T16:09:38.915 INFO:teuthology.orchestra.run.smithi008.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 813 kB/s | 34 kB 00:00 2024-02-19T16:09:38.961 INFO:teuthology.orchestra.run.smithi191.stdout:(16/176): librados-devel-19.0.0-1451.g9f8908ee. 1.5 MB/s | 127 kB 00:00 2024-02-19T16:09:39.024 INFO:teuthology.orchestra.run.smithi008.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 348 kB/s | 43 kB 00:00 2024-02-19T16:09:39.049 INFO:teuthology.orchestra.run.smithi008.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 300 kB 00:00 2024-02-19T16:09:39.066 INFO:teuthology.orchestra.run.smithi008.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-02-19T16:09:39.091 INFO:teuthology.orchestra.run.smithi008.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-02-19T16:09:39.102 INFO:teuthology.orchestra.run.smithi156.stdout:(22/176): python3-rados-19.0.0-1451.g9f8908ee.e 1.3 MB/s | 320 kB 00:00 2024-02-19T16:09:39.108 INFO:teuthology.orchestra.run.smithi008.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-19T16:09:39.125 INFO:teuthology.orchestra.run.smithi008.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-02-19T16:09:39.142 INFO:teuthology.orchestra.run.smithi008.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-02-19T16:09:39.161 INFO:teuthology.orchestra.run.smithi191.stdout:(17/176): libradosstriper1-19.0.0-1451.g9f8908e 2.4 MB/s | 498 kB 00:00 2024-02-19T16:09:39.167 INFO:teuthology.orchestra.run.smithi008.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-19T16:09:39.210 INFO:teuthology.orchestra.run.smithi008.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 12 MB/s | 802 kB 00:00 2024-02-19T16:09:39.235 INFO:teuthology.orchestra.run.smithi008.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 194 kB 00:00 2024-02-19T16:09:39.260 INFO:teuthology.orchestra.run.smithi008.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-19T16:09:39.285 INFO:teuthology.orchestra.run.smithi008.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-19T16:09:39.328 INFO:teuthology.orchestra.run.smithi008.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-02-19T16:09:39.352 INFO:teuthology.orchestra.run.smithi156.stdout:(23/176): python3-rbd-19.0.0-1451.g9f8908ee.el9 1.2 MB/s | 298 kB 00:00 2024-02-19T16:09:39.356 INFO:teuthology.orchestra.run.smithi008.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-02-19T16:09:39.378 INFO:teuthology.orchestra.run.smithi008.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-02-19T16:09:39.403 INFO:teuthology.orchestra.run.smithi008.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-02-19T16:09:39.429 INFO:teuthology.orchestra.run.smithi008.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-19T16:09:39.461 INFO:teuthology.orchestra.run.smithi156.stdout:(24/176): python3-rgw-19.0.0-1451.g9f8908ee.el9 917 kB/s | 99 kB 00:00 2024-02-19T16:09:39.553 INFO:teuthology.orchestra.run.smithi156.stdout:(25/176): rbd-fuse-19.0.0-1451.g9f8908ee.el9.x8 931 kB/s | 85 kB 00:00 2024-02-19T16:09:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-02-19T16:09:39.653 INFO:teuthology.orchestra.run.smithi156.stdout:(26/176): librgw2-19.0.0-1451.g9f8908ee.el9.x86 3.4 MB/s | 5.1 MB 00:01 2024-02-19T16:09:39.745 INFO:teuthology.orchestra.run.smithi156.stdout:(27/176): rbd-nbd-19.0.0-1451.g9f8908ee.el9.x86 1.8 MB/s | 171 kB 00:00 2024-02-19T16:09:39.754 INFO:teuthology.orchestra.run.smithi008.stdout:(104/176): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-02-19T16:09:39.820 INFO:teuthology.orchestra.run.smithi156.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1451.g 420 kB/s | 31 kB 00:00 2024-02-19T16:09:39.888 INFO:teuthology.orchestra.run.smithi008.stdout:(105/176): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-02-19T16:09:39.895 INFO:teuthology.orchestra.run.smithi156.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1451.g9f8908e 1.8 MB/s | 138 kB 00:00 2024-02-19T16:09:39.946 INFO:teuthology.orchestra.run.smithi008.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-02-19T16:09:39.980 INFO:teuthology.orchestra.run.smithi008.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 582 kB/s | 19 kB 00:00 2024-02-19T16:09:40.422 INFO:teuthology.orchestra.run.smithi008.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-02-19T16:09:40.448 INFO:teuthology.orchestra.run.smithi008.stdout:(109/176): python3-devel-3.9.17-2.el9.x86_64.rp 235 kB/s | 245 kB 00:01 2024-02-19T16:09:40.471 INFO:teuthology.orchestra.run.smithi156.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1451.g9f890 3.1 MB/s | 1.8 MB 00:00 2024-02-19T16:09:40.472 INFO:teuthology.orchestra.run.smithi008.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-02-19T16:09:40.478 INFO:teuthology.orchestra.run.smithi191.stdout:(18/176): librgw2-19.0.0-1451.g9f8908ee.el9.x86 3.9 MB/s | 5.1 MB 00:01 2024-02-19T16:09:40.515 INFO:teuthology.orchestra.run.smithi008.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-02-19T16:09:40.540 INFO:teuthology.orchestra.run.smithi008.stdout:(112/176): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-02-19T16:09:40.574 INFO:teuthology.orchestra.run.smithi008.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-02-19T16:09:40.649 INFO:teuthology.orchestra.run.smithi008.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-02-19T16:09:40.683 INFO:teuthology.orchestra.run.smithi008.stdout:(115/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-02-19T16:09:40.708 INFO:teuthology.orchestra.run.smithi008.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-02-19T16:09:40.733 INFO:teuthology.orchestra.run.smithi008.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-02-19T16:09:40.758 INFO:teuthology.orchestra.run.smithi008.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-02-19T16:09:40.792 INFO:teuthology.orchestra.run.smithi008.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-02-19T16:09:40.842 INFO:teuthology.orchestra.run.smithi008.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-02-19T16:09:40.876 INFO:teuthology.orchestra.run.smithi008.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-02-19T16:09:40.893 INFO:teuthology.orchestra.run.smithi008.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-02-19T16:09:40.926 INFO:teuthology.orchestra.run.smithi008.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-02-19T16:09:40.977 INFO:teuthology.orchestra.run.smithi008.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-02-19T16:09:41.002 INFO:teuthology.orchestra.run.smithi008.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 434 kB/s | 11 kB 00:00 2024-02-19T16:09:41.003 INFO:teuthology.orchestra.run.smithi191.stdout:(19/176): ceph-test-19.0.0-1451.g9f8908ee.el9.x 15 MB/s | 47 MB 00:03 2024-02-19T16:09:41.029 INFO:teuthology.orchestra.run.smithi191.stdout:(20/176): python3-ceph-argparse-19.0.0-1451.g9f 82 kB/s | 45 kB 00:00 2024-02-19T16:09:41.036 INFO:teuthology.orchestra.run.smithi008.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 527 kB/s | 18 kB 00:00 2024-02-19T16:09:41.046 INFO:teuthology.orchestra.run.smithi156.stdout:(31/176): rbd-mirror-19.0.0-1451.g9f8908ee.el9. 2.1 MB/s | 3.1 MB 00:01 2024-02-19T16:09:41.054 INFO:teuthology.orchestra.run.smithi191.stdout:(21/176): python3-ceph-common-19.0.0-1451.g9f89 2.5 MB/s | 131 kB 00:00 2024-02-19T16:09:41.069 INFO:teuthology.orchestra.run.smithi008.stdout:(127/176): python3-jaraco-collections-3.0.0-8.e 696 kB/s | 23 kB 00:00 2024-02-19T16:09:41.096 INFO:teuthology.orchestra.run.smithi191.stdout:(22/176): python3-cephfs-19.0.0-1451.g9f8908ee. 2.3 MB/s | 162 kB 00:00 2024-02-19T16:09:41.146 INFO:teuthology.orchestra.run.smithi156.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1451.g9f 2.4 MB/s | 248 kB 00:00 2024-02-19T16:09:41.155 INFO:teuthology.orchestra.run.smithi191.stdout:(23/176): python3-rados-19.0.0-1451.g9f8908ee.e 3.1 MB/s | 320 kB 00:00 2024-02-19T16:09:41.161 INFO:teuthology.orchestra.run.smithi008.stdout:(128/176): ceph-mgr-diskprediction-local-19.0.0 1.2 MB/s | 7.4 MB 00:06 2024-02-19T16:09:41.180 INFO:teuthology.orchestra.run.smithi191.stdout:(24/176): python3-rbd-19.0.0-1451.g9f8908ee.el9 3.5 MB/s | 298 kB 00:00 2024-02-19T16:09:41.186 INFO:teuthology.orchestra.run.smithi008.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-02-19T16:09:41.205 INFO:teuthology.orchestra.run.smithi156.stdout:(33/176): ceph-mgr-rook-19.0.0-1451.g9f8908ee.e 898 kB/s | 52 kB 00:00 2024-02-19T16:09:41.206 INFO:teuthology.orchestra.run.smithi191.stdout:(25/176): python3-rgw-19.0.0-1451.g9f8908ee.el9 1.9 MB/s | 99 kB 00:00 2024-02-19T16:09:41.231 INFO:teuthology.orchestra.run.smithi191.stdout:(26/176): rbd-fuse-19.0.0-1451.g9f8908ee.el9.x8 1.7 MB/s | 85 kB 00:00 2024-02-19T16:09:41.237 INFO:teuthology.orchestra.run.smithi008.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-02-19T16:09:41.279 INFO:teuthology.orchestra.run.smithi008.stdout:(131/176): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-02-19T16:09:41.290 INFO:teuthology.orchestra.run.smithi191.stdout:(27/176): rbd-nbd-19.0.0-1451.g9f8908ee.el9.x86 2.9 MB/s | 171 kB 00:00 2024-02-19T16:09:41.304 INFO:teuthology.orchestra.run.smithi008.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-02-19T16:09:41.313 INFO:teuthology.orchestra.run.smithi156.stdout:(34/176): ceph-mgr-diskprediction-local-19.0.0- 8.8 MB/s | 7.4 MB 00:00 2024-02-19T16:09:41.329 INFO:teuthology.orchestra.run.smithi008.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-02-19T16:09:41.330 INFO:teuthology.orchestra.run.smithi156.stdout:(35/176): ceph-prometheus-alerts-19.0.0-1451.g9 122 kB/s | 15 kB 00:00 2024-02-19T16:09:41.349 INFO:teuthology.orchestra.run.smithi191.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1451.g 539 kB/s | 31 kB 00:00 2024-02-19T16:09:41.355 INFO:teuthology.orchestra.run.smithi008.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-02-19T16:09:41.389 INFO:teuthology.orchestra.run.smithi008.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-02-19T16:09:41.397 INFO:teuthology.orchestra.run.smithi156.stdout:(36/176): ceph-volume-19.0.0-1451.g9f8908ee.el9 3.1 MB/s | 264 kB 00:00 2024-02-19T16:09:41.414 INFO:teuthology.orchestra.run.smithi008.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-02-19T16:09:41.424 INFO:teuthology.orchestra.run.smithi191.stdout:(29/176): rbd-mirror-19.0.0-1451.g9f8908ee.el9. 14 MB/s | 3.1 MB 00:00 2024-02-19T16:09:41.439 INFO:teuthology.orchestra.run.smithi156.stdout:(37/176): cephfs-top-19.0.0-1451.g9f8908ee.el9. 441 kB/s | 18 kB 00:00 2024-02-19T16:09:41.447 INFO:teuthology.orchestra.run.smithi008.stdout:(137/176): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-02-19T16:09:41.450 INFO:teuthology.orchestra.run.smithi191.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1451.g9f8908e 1.3 MB/s | 138 kB 00:00 2024-02-19T16:09:41.472 INFO:teuthology.orchestra.run.smithi008.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 485 kB/s | 28 kB 00:00 2024-02-19T16:09:41.498 INFO:teuthology.orchestra.run.smithi008.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-02-19T16:09:41.523 INFO:teuthology.orchestra.run.smithi008.stdout:(140/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-02-19T16:09:41.548 INFO:teuthology.orchestra.run.smithi008.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-02-19T16:09:41.567 INFO:teuthology.orchestra.run.smithi191.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1451.g9f890 13 MB/s | 1.8 MB 00:00 2024-02-19T16:09:41.573 INFO:teuthology.orchestra.run.smithi156.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 662 kB 00:00 2024-02-19T16:09:41.573 INFO:teuthology.orchestra.run.smithi008.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-02-19T16:09:41.599 INFO:teuthology.orchestra.run.smithi008.stdout:(143/176): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-02-19T16:09:41.624 INFO:teuthology.orchestra.run.smithi008.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-02-19T16:09:41.625 INFO:teuthology.orchestra.run.smithi191.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1451.g9f 4.2 MB/s | 248 kB 00:00 2024-02-19T16:09:41.640 INFO:teuthology.orchestra.run.smithi156.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2024-02-19T16:09:41.641 INFO:teuthology.orchestra.run.smithi008.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-02-19T16:09:41.666 INFO:teuthology.orchestra.run.smithi008.stdout:(146/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-02-19T16:09:41.667 INFO:teuthology.orchestra.run.smithi191.stdout:(33/176): ceph-mgr-rook-19.0.0-1451.g9f8908ee.e 1.2 MB/s | 52 kB 00:00 2024-02-19T16:09:41.673 INFO:teuthology.orchestra.run.smithi156.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-02-19T16:09:41.691 INFO:teuthology.orchestra.run.smithi008.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-02-19T16:09:41.707 INFO:teuthology.orchestra.run.smithi156.stdout:(41/176): cephadm-19.0.0-1451.g9f8908ee.el9.noa 1.9 MB/s | 748 kB 00:00 2024-02-19T16:09:41.709 INFO:teuthology.orchestra.run.smithi191.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1451.g9 367 kB/s | 15 kB 00:00 2024-02-19T16:09:41.717 INFO:teuthology.orchestra.run.smithi008.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-02-19T16:09:41.724 INFO:teuthology.orchestra.run.smithi156.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 406 kB/s | 20 kB 00:00 2024-02-19T16:09:41.742 INFO:teuthology.orchestra.run.smithi008.stdout:(149/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-02-19T16:09:41.749 INFO:teuthology.orchestra.run.smithi156.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-02-19T16:09:41.768 INFO:teuthology.orchestra.run.smithi191.stdout:(35/176): ceph-volume-19.0.0-1451.g9f8908ee.el9 4.4 MB/s | 264 kB 00:00 2024-02-19T16:09:41.777 INFO:teuthology.orchestra.run.smithi156.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 607 kB/s | 32 kB 00:00 2024-02-19T16:09:41.852 INFO:teuthology.orchestra.run.smithi191.stdout:(36/176): cephadm-19.0.0-1451.g9f8908ee.el9.noa 8.8 MB/s | 748 kB 00:00 2024-02-19T16:09:41.869 INFO:teuthology.orchestra.run.smithi156.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-02-19T16:09:41.892 INFO:teuthology.orchestra.run.smithi008.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-02-19T16:09:41.894 INFO:teuthology.orchestra.run.smithi156.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 150 kB/s | 17 kB 00:00 2024-02-19T16:09:41.902 INFO:teuthology.orchestra.run.smithi191.stdout:(37/176): cephfs-top-19.0.0-1451.g9f8908ee.el9. 365 kB/s | 18 kB 00:00 2024-02-19T16:09:41.928 INFO:teuthology.orchestra.run.smithi156.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-02-19T16:09:41.934 INFO:teuthology.orchestra.run.smithi008.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-02-19T16:09:41.954 INFO:teuthology.orchestra.run.smithi156.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 159 kB 00:00 2024-02-19T16:09:41.979 INFO:teuthology.orchestra.run.smithi156.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-02-19T16:09:42.004 INFO:teuthology.orchestra.run.smithi156.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-19T16:09:42.030 INFO:teuthology.orchestra.run.smithi156.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-02-19T16:09:42.046 INFO:teuthology.orchestra.run.smithi156.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 596 kB/s | 24 kB 00:00 2024-02-19T16:09:42.060 INFO:teuthology.orchestra.run.smithi008.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 18 MB/s | 2.3 MB 00:00 2024-02-19T16:09:42.061 INFO:teuthology.orchestra.run.smithi191.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.1 MB/s | 662 kB 00:00 2024-02-19T16:09:42.063 INFO:teuthology.orchestra.run.smithi156.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-02-19T16:09:42.088 INFO:teuthology.orchestra.run.smithi156.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-02-19T16:09:42.114 INFO:teuthology.orchestra.run.smithi156.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-19T16:09:42.128 INFO:teuthology.orchestra.run.smithi191.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2024-02-19T16:09:42.131 INFO:teuthology.orchestra.run.smithi156.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 823 kB/s | 34 kB 00:00 2024-02-19T16:09:42.152 INFO:teuthology.orchestra.run.smithi008.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-02-19T16:09:42.187 INFO:teuthology.orchestra.run.smithi191.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-02-19T16:09:42.227 INFO:teuthology.orchestra.run.smithi008.stdout:(154/176): librbd1-19.0.0-1451.g9f8908ee.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-02-19T16:09:42.240 INFO:teuthology.orchestra.run.smithi156.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-02-19T16:09:42.252 INFO:teuthology.orchestra.run.smithi008.stdout:(155/176): NetworkManager-team-1.45.91-1.el9.x8 412 kB/s | 41 kB 00:00 2024-02-19T16:09:42.277 INFO:teuthology.orchestra.run.smithi008.stdout:(156/176): NetworkManager-tui-1.45.91-1.el9.x86 4.8 MB/s | 246 kB 00:00 2024-02-19T16:09:42.298 INFO:teuthology.orchestra.run.smithi156.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-02-19T16:09:42.304 INFO:teuthology.orchestra.run.smithi191.stdout:(41/176): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-02-19T16:09:42.311 INFO:teuthology.orchestra.run.smithi008.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-02-19T16:09:42.330 INFO:teuthology.orchestra.run.smithi191.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 20 kB 00:00 2024-02-19T16:09:42.332 INFO:teuthology.orchestra.run.smithi156.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-02-19T16:09:42.336 INFO:teuthology.orchestra.run.smithi008.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 605 kB/s | 35 kB 00:00 2024-02-19T16:09:42.355 INFO:teuthology.orchestra.run.smithi191.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-02-19T16:09:42.362 INFO:teuthology.orchestra.run.smithi008.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-02-19T16:09:42.374 INFO:teuthology.orchestra.run.smithi156.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-02-19T16:09:42.380 INFO:teuthology.orchestra.run.smithi191.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-02-19T16:09:42.387 INFO:teuthology.orchestra.run.smithi008.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-02-19T16:09:42.399 INFO:teuthology.orchestra.run.smithi156.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-02-19T16:09:42.421 INFO:teuthology.orchestra.run.smithi008.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 14 MB/s | 802 kB 00:00 2024-02-19T16:09:42.447 INFO:teuthology.orchestra.run.smithi191.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 264 kB/s | 17 kB 00:00 2024-02-19T16:09:42.479 INFO:teuthology.orchestra.run.smithi008.stdout:(162/176): librados2-19.0.0-1451.g9f8908ee.el9. 4.4 MB/s | 3.4 MB 00:00 2024-02-19T16:09:42.505 INFO:teuthology.orchestra.run.smithi008.stdout:(163/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 3.7 MB/s | 433 kB 00:00 2024-02-19T16:09:42.514 INFO:teuthology.orchestra.run.smithi191.stdout:(46/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-02-19T16:09:42.530 INFO:teuthology.orchestra.run.smithi008.stdout:(164/176): libacl-2.3.1-4.el9.x86_64.rpm 202 kB/s | 22 kB 00:00 2024-02-19T16:09:42.540 INFO:teuthology.orchestra.run.smithi191.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 110 kB/s | 10 kB 00:00 2024-02-19T16:09:42.555 INFO:teuthology.orchestra.run.smithi008.stdout:(165/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-02-19T16:09:42.557 INFO:teuthology.orchestra.run.smithi156.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-02-19T16:09:42.565 INFO:teuthology.orchestra.run.smithi191.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-02-19T16:09:42.580 INFO:teuthology.orchestra.run.smithi008.stdout:(166/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-19T16:09:42.591 INFO:teuthology.orchestra.run.smithi191.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-02-19T16:09:42.591 INFO:teuthology.orchestra.run.smithi156.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-02-19T16:09:42.605 INFO:teuthology.orchestra.run.smithi008.stdout:(167/176): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 135 kB 00:00 2024-02-19T16:09:42.616 INFO:teuthology.orchestra.run.smithi191.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-19T16:09:42.616 INFO:teuthology.orchestra.run.smithi156.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-19T16:09:42.631 INFO:teuthology.orchestra.run.smithi008.stdout:(168/176): libsmartcols-2.37.4-18.el9.x86_64.rp 839 kB/s | 63 kB 00:00 2024-02-19T16:09:42.656 INFO:teuthology.orchestra.run.smithi008.stdout:(169/176): libuuid-2.37.4-18.el9.x86_64.rpm 554 kB/s | 28 kB 00:00 2024-02-19T16:09:42.692 INFO:teuthology.orchestra.run.smithi191.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 578 kB 00:00 2024-02-19T16:09:42.709 INFO:teuthology.orchestra.run.smithi191.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 265 kB/s | 24 kB 00:00 2024-02-19T16:09:42.759 INFO:teuthology.orchestra.run.smithi191.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 694 kB/s | 46 kB 00:00 2024-02-19T16:09:42.773 INFO:teuthology.orchestra.run.smithi008.stdout:(170/176): openssl-3.0.7-25.el9.x86_64.rpm 6.9 MB/s | 1.2 MB 00:00 2024-02-19T16:09:42.784 INFO:teuthology.orchestra.run.smithi191.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-19T16:09:42.815 INFO:teuthology.orchestra.run.smithi008.stdout:(171/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 12 MB/s | 2.1 MB 00:00 2024-02-19T16:09:42.820 INFO:teuthology.orchestra.run.smithi191.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-02-19T16:09:42.828 INFO:teuthology.orchestra.run.smithi156.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-02-19T16:09:42.835 INFO:teuthology.orchestra.run.smithi191.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 678 kB/s | 34 kB 00:00 2024-02-19T16:09:42.842 INFO:teuthology.orchestra.run.smithi156.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 227 kB 00:00 2024-02-19T16:09:42.867 INFO:teuthology.orchestra.run.smithi156.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 586 kB/s | 24 kB 00:00 2024-02-19T16:09:42.882 INFO:teuthology.orchestra.run.smithi008.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 9.8 MB/s | 2.2 MB 00:00 2024-02-19T16:09:42.902 INFO:teuthology.orchestra.run.smithi156.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 308 kB/s | 18 kB 00:00 2024-02-19T16:09:42.907 INFO:teuthology.orchestra.run.smithi008.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 3.3 MB/s | 438 kB 00:00 2024-02-19T16:09:42.932 INFO:teuthology.orchestra.run.smithi008.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 259 kB/s | 30 kB 00:00 2024-02-19T16:09:42.934 INFO:teuthology.orchestra.run.smithi156.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-02-19T16:09:42.966 INFO:teuthology.orchestra.run.smithi008.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-02-19T16:09:42.968 INFO:teuthology.orchestra.run.smithi156.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 471 kB/s | 32 kB 00:00 2024-02-19T16:09:42.994 INFO:teuthology.orchestra.run.smithi191.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-02-19T16:09:43.133 INFO:teuthology.orchestra.run.smithi008.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-02-19T16:09:43.136 INFO:teuthology.orchestra.run.smithi191.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 9.9 MB/s | 3.0 MB 00:00 2024-02-19T16:09:43.139 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-02-19T16:09:43.139 INFO:teuthology.orchestra.run.smithi008.stdout:Total 15 MB/s | 223 MB 00:14 2024-02-19T16:09:43.170 INFO:teuthology.orchestra.run.smithi191.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 8.8 MB/s | 1.5 MB 00:00 2024-02-19T16:09:43.177 INFO:teuthology.orchestra.run.smithi156.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-02-19T16:09:43.184 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-02-19T16:09:43.202 INFO:teuthology.orchestra.run.smithi156.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 930 kB/s | 217 kB 00:00 2024-02-19T16:09:43.203 INFO:teuthology.orchestra.run.smithi191.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-02-19T16:09:43.227 INFO:teuthology.orchestra.run.smithi156.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-19T16:09:43.229 INFO:teuthology.orchestra.run.smithi191.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-02-19T16:09:43.253 INFO:teuthology.orchestra.run.smithi156.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-19T16:09:43.278 INFO:teuthology.orchestra.run.smithi156.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-02-19T16:09:43.303 INFO:teuthology.orchestra.run.smithi156.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-19T16:09:43.328 INFO:teuthology.orchestra.run.smithi156.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-19T16:09:43.345 INFO:teuthology.orchestra.run.smithi156.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-02-19T16:09:43.371 INFO:teuthology.orchestra.run.smithi156.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-02-19T16:09:43.388 INFO:teuthology.orchestra.run.smithi191.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-02-19T16:09:43.396 INFO:teuthology.orchestra.run.smithi156.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 854 kB/s | 43 kB 00:00 2024-02-19T16:09:43.430 INFO:teuthology.orchestra.run.smithi191.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-02-19T16:09:43.455 INFO:teuthology.orchestra.run.smithi191.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-02-19T16:09:43.511 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-02-19T16:09:43.511 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-02-19T16:09:43.630 INFO:teuthology.orchestra.run.smithi156.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 167 kB/s | 39 kB 00:00 2024-02-19T16:09:43.681 INFO:teuthology.orchestra.run.smithi191.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-02-19T16:09:43.706 INFO:teuthology.orchestra.run.smithi191.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 908 kB/s | 227 kB 00:00 2024-02-19T16:09:43.748 INFO:teuthology.orchestra.run.smithi191.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 586 kB/s | 24 kB 00:00 2024-02-19T16:09:43.773 INFO:teuthology.orchestra.run.smithi191.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 270 kB/s | 18 kB 00:00 2024-02-19T16:09:43.799 INFO:teuthology.orchestra.run.smithi191.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-19T16:09:43.824 INFO:teuthology.orchestra.run.smithi191.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 629 kB/s | 32 kB 00:00 2024-02-19T16:09:43.863 INFO:teuthology.orchestra.run.smithi156.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-02-19T16:09:43.889 INFO:teuthology.orchestra.run.smithi156.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 160 kB/s | 41 kB 00:00 2024-02-19T16:09:43.914 INFO:teuthology.orchestra.run.smithi156.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-19T16:09:43.939 INFO:teuthology.orchestra.run.smithi156.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 673 kB/s | 34 kB 00:00 2024-02-19T16:09:43.964 INFO:teuthology.orchestra.run.smithi156.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 866 kB/s | 43 kB 00:00 2024-02-19T16:09:43.975 INFO:teuthology.orchestra.run.smithi191.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-02-19T16:09:43.990 INFO:teuthology.orchestra.run.smithi156.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-02-19T16:09:44.000 INFO:teuthology.orchestra.run.smithi191.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-02-19T16:09:44.015 INFO:teuthology.orchestra.run.smithi156.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-02-19T16:09:44.025 INFO:teuthology.orchestra.run.smithi191.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-19T16:09:44.040 INFO:teuthology.orchestra.run.smithi156.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-02-19T16:09:44.051 INFO:teuthology.orchestra.run.smithi191.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-19T16:09:44.065 INFO:teuthology.orchestra.run.smithi156.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-02-19T16:09:44.068 INFO:teuthology.orchestra.run.smithi191.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 979 kB/s | 41 kB 00:00 2024-02-19T16:09:44.090 INFO:teuthology.orchestra.run.smithi156.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-02-19T16:09:44.102 INFO:teuthology.orchestra.run.smithi191.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-19T16:09:44.116 INFO:teuthology.orchestra.run.smithi156.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-02-19T16:09:44.136 INFO:teuthology.orchestra.run.smithi191.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-02-19T16:09:44.141 INFO:teuthology.orchestra.run.smithi156.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-19T16:09:44.161 INFO:teuthology.orchestra.run.smithi191.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-02-19T16:09:44.175 INFO:teuthology.orchestra.run.smithi156.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-19T16:09:44.179 INFO:teuthology.orchestra.run.smithi191.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-02-19T16:09:44.196 INFO:teuthology.orchestra.run.smithi191.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-02-19T16:09:44.209 INFO:teuthology.orchestra.run.smithi156.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 194 kB 00:00 2024-02-19T16:09:44.234 INFO:teuthology.orchestra.run.smithi156.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-02-19T16:09:44.259 INFO:teuthology.orchestra.run.smithi156.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-19T16:09:44.318 INFO:teuthology.orchestra.run.smithi156.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-02-19T16:09:44.343 INFO:teuthology.orchestra.run.smithi156.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-02-19T16:09:44.371 INFO:teuthology.orchestra.run.smithi191.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 222 kB/s | 39 kB 00:00 2024-02-19T16:09:44.560 INFO:teuthology.orchestra.run.smithi156.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 513 kB/s | 124 kB 00:00 2024-02-19T16:09:44.585 INFO:teuthology.orchestra.run.smithi156.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 467 kB/s | 113 kB 00:00 2024-02-19T16:09:44.611 INFO:teuthology.orchestra.run.smithi156.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-19T16:09:44.663 INFO:teuthology.orchestra.run.smithi191.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-02-19T16:09:44.688 INFO:teuthology.orchestra.run.smithi191.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 130 kB/s | 41 kB 00:00 2024-02-19T16:09:44.713 INFO:teuthology.orchestra.run.smithi191.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-19T16:09:44.739 INFO:teuthology.orchestra.run.smithi191.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 677 kB/s | 34 kB 00:00 2024-02-19T16:09:44.745 INFO:teuthology.orchestra.run.smithi156.stdout:(103/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.5 MB/s | 245 kB 00:00 2024-02-19T16:09:44.764 INFO:teuthology.orchestra.run.smithi191.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 864 kB/s | 43 kB 00:00 2024-02-19T16:09:44.770 INFO:teuthology.orchestra.run.smithi156.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-02-19T16:09:44.789 INFO:teuthology.orchestra.run.smithi191.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-02-19T16:09:44.814 INFO:teuthology.orchestra.run.smithi191.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-02-19T16:09:44.840 INFO:teuthology.orchestra.run.smithi191.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-02-19T16:09:44.857 INFO:teuthology.orchestra.run.smithi191.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-19T16:09:44.874 INFO:teuthology.orchestra.run.smithi191.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-02-19T16:09:44.899 INFO:teuthology.orchestra.run.smithi191.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-02-19T16:09:44.924 INFO:teuthology.orchestra.run.smithi191.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-19T16:09:44.966 INFO:teuthology.orchestra.run.smithi191.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 12 MB/s | 802 kB 00:00 2024-02-19T16:09:44.970 INFO:teuthology.orchestra.run.smithi156.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-02-19T16:09:44.991 INFO:teuthology.orchestra.run.smithi191.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 194 kB 00:00 2024-02-19T16:09:45.017 INFO:teuthology.orchestra.run.smithi191.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-19T16:09:45.042 INFO:teuthology.orchestra.run.smithi191.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-19T16:09:45.075 INFO:teuthology.orchestra.run.smithi191.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-02-19T16:09:45.101 INFO:teuthology.orchestra.run.smithi191.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-02-19T16:09:45.134 INFO:teuthology.orchestra.run.smithi191.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.1 MB/s | 124 kB 00:00 2024-02-19T16:09:45.160 INFO:teuthology.orchestra.run.smithi191.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 1.9 MB/s | 113 kB 00:00 2024-02-19T16:09:45.185 INFO:teuthology.orchestra.run.smithi191.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-19T16:09:45.269 INFO:teuthology.orchestra.run.smithi191.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-02-19T16:09:45.329 INFO:teuthology.orchestra.run.smithi156.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 981 kB/s | 548 kB 00:00 2024-02-19T16:09:45.427 INFO:teuthology.orchestra.run.smithi191.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 919 kB/s | 245 kB 00:00 2024-02-19T16:09:45.446 INFO:teuthology.orchestra.run.smithi156.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-02-19T16:09:45.469 INFO:teuthology.orchestra.run.smithi191.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-02-19T16:09:45.537 INFO:teuthology.orchestra.run.smithi156.stdout:(108/176): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-02-19T16:09:45.573 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-02-19T16:09:45.573 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-02-19T16:09:45.638 INFO:teuthology.orchestra.run.smithi156.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-02-19T16:09:45.746 INFO:teuthology.orchestra.run.smithi156.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-02-19T16:09:45.889 INFO:teuthology.orchestra.run.smithi156.stdout:(111/176): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-02-19T16:09:45.905 INFO:teuthology.orchestra.run.smithi156.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-02-19T16:09:46.014 INFO:teuthology.orchestra.run.smithi156.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-02-19T16:09:46.099 INFO:teuthology.orchestra.run.smithi156.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-02-19T16:09:46.190 INFO:teuthology.orchestra.run.smithi156.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-02-19T16:09:46.211 INFO:teuthology.orchestra.run.smithi191.stdout:(106/176): ceph-osd-19.0.0-1451.g9f8908ee.el9.x 1.8 MB/s | 17 MB 00:09 2024-02-19T16:09:46.215 INFO:teuthology.orchestra.run.smithi156.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-02-19T16:09:46.298 INFO:teuthology.orchestra.run.smithi156.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-02-19T16:09:46.365 INFO:teuthology.orchestra.run.smithi156.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 100 kB/s | 15 kB 00:00 2024-02-19T16:09:46.441 INFO:teuthology.orchestra.run.smithi156.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-02-19T16:09:46.491 INFO:teuthology.orchestra.run.smithi156.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-02-19T16:09:46.583 INFO:teuthology.orchestra.run.smithi156.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-02-19T16:09:46.608 INFO:teuthology.orchestra.run.smithi156.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-02-19T16:09:46.716 INFO:teuthology.orchestra.run.smithi156.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-02-19T16:09:46.843 INFO:teuthology.orchestra.run.smithi156.stdout:(124/176): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-02-19T16:09:46.868 INFO:teuthology.orchestra.run.smithi156.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 962 kB/s | 144 kB 00:00 2024-02-19T16:09:46.942 INFO:teuthology.orchestra.run.smithi156.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-02-19T16:09:46.968 INFO:teuthology.orchestra.run.smithi156.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-02-19T16:09:47.043 INFO:teuthology.orchestra.run.smithi156.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-02-19T16:09:47.068 INFO:teuthology.orchestra.run.smithi156.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-02-19T16:09:47.144 INFO:teuthology.orchestra.run.smithi156.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-02-19T16:09:47.235 INFO:teuthology.orchestra.run.smithi156.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-02-19T16:09:47.269 INFO:teuthology.orchestra.run.smithi156.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-02-19T16:09:47.336 INFO:teuthology.orchestra.run.smithi156.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-02-19T16:09:47.379 INFO:teuthology.orchestra.run.smithi156.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 532 kB/s | 58 kB 00:00 2024-02-19T16:09:47.470 INFO:teuthology.orchestra.run.smithi156.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-02-19T16:09:47.487 INFO:teuthology.orchestra.run.smithi156.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-02-19T16:09:47.571 INFO:teuthology.orchestra.run.smithi156.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-02-19T16:09:47.596 INFO:teuthology.orchestra.run.smithi156.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-02-19T16:09:47.713 INFO:teuthology.orchestra.run.smithi156.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-02-19T16:09:47.738 INFO:teuthology.orchestra.run.smithi156.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-02-19T16:09:47.814 INFO:teuthology.orchestra.run.smithi156.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-02-19T16:09:47.830 INFO:teuthology.orchestra.run.smithi156.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-02-19T16:09:47.914 INFO:teuthology.orchestra.run.smithi156.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-02-19T16:09:48.014 INFO:teuthology.orchestra.run.smithi156.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-02-19T16:09:48.040 INFO:teuthology.orchestra.run.smithi156.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-02-19T16:09:48.140 INFO:teuthology.orchestra.run.smithi156.stdout:(146/176): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-02-19T16:09:48.174 INFO:teuthology.orchestra.run.smithi156.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-02-19T16:09:48.249 INFO:teuthology.orchestra.run.smithi156.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-02-19T16:09:48.466 INFO:teuthology.orchestra.run.smithi156.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-02-19T16:09:48.691 INFO:teuthology.orchestra.run.smithi156.stdout:(150/176): librados2-19.0.0-1451.g9f8908ee.el9. 7.6 MB/s | 3.4 MB 00:00 2024-02-19T16:09:48.724 INFO:teuthology.orchestra.run.smithi156.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-19T16:09:48.816 INFO:teuthology.orchestra.run.smithi156.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 25 MB/s | 2.3 MB 00:00 2024-02-19T16:09:48.917 INFO:teuthology.orchestra.run.smithi156.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-02-19T16:09:48.950 INFO:teuthology.orchestra.run.smithi156.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-02-19T16:09:48.992 INFO:teuthology.orchestra.run.smithi156.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 5.8 MB/s | 246 kB 00:00 2024-02-19T16:09:49.026 INFO:teuthology.orchestra.run.smithi156.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-02-19T16:09:49.068 INFO:teuthology.orchestra.run.smithi156.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 848 kB/s | 35 kB 00:00 2024-02-19T16:09:49.119 INFO:teuthology.orchestra.run.smithi156.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-02-19T16:09:49.168 INFO:teuthology.orchestra.run.smithi156.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-02-19T16:09:49.227 INFO:teuthology.orchestra.run.smithi156.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-19T16:09:49.286 INFO:teuthology.orchestra.run.smithi156.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 7.3 MB/s | 433 kB 00:00 2024-02-19T16:09:49.330 INFO:teuthology.orchestra.run.smithi156.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 502 kB/s | 22 kB 00:00 2024-02-19T16:09:49.364 INFO:teuthology.orchestra.run.smithi156.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 107 kB 00:00 2024-02-19T16:09:49.406 INFO:teuthology.orchestra.run.smithi156.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 154 kB 00:00 2024-02-19T16:09:49.439 INFO:teuthology.orchestra.run.smithi156.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 3.9 MB/s | 135 kB 00:00 2024-02-19T16:09:49.481 INFO:teuthology.orchestra.run.smithi156.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 63 kB 00:00 2024-02-19T16:09:49.506 INFO:teuthology.orchestra.run.smithi156.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 28 kB 00:00 2024-02-19T16:09:49.590 INFO:teuthology.orchestra.run.smithi156.stdout:(168/176): openssl-3.0.7-25.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-02-19T16:09:49.641 INFO:teuthology.orchestra.run.smithi156.stdout:(169/176): librbd1-19.0.0-1451.g9f8908ee.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-02-19T16:09:49.716 INFO:teuthology.orchestra.run.smithi156.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2024-02-19T16:09:49.766 INFO:teuthology.orchestra.run.smithi156.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-02-19T16:09:49.792 INFO:teuthology.orchestra.run.smithi156.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-02-19T16:09:49.816 INFO:teuthology.orchestra.run.smithi156.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 609 kB/s | 30 kB 00:00 2024-02-19T16:09:49.842 INFO:teuthology.orchestra.run.smithi156.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-02-19T16:09:49.876 INFO:teuthology.orchestra.run.smithi156.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-02-19T16:09:50.617 INFO:teuthology.orchestra.run.smithi156.stdout:(176/176): ceph-test-19.0.0-1451.g9f8908ee.el9. 3.3 MB/s | 47 MB 00:14 2024-02-19T16:09:50.622 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-02-19T16:09:50.622 INFO:teuthology.orchestra.run.smithi156.stdout:Total 11 MB/s | 223 MB 00:20 2024-02-19T16:09:50.671 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-02-19T16:09:50.948 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-02-19T16:09:50.948 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-02-19T16:09:52.610 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-02-19T16:09:52.610 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-02-19T16:09:52.764 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-02-19T16:09:52.997 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-19T16:09:53.123 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-19T16:09:53.177 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-19T16:09:53.206 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-19T16:09:53.265 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-19T16:09:53.305 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-19T16:09:53.370 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-19T16:09:53.409 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-19T16:09:53.433 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-19T16:09:53.492 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-19T16:09:53.853 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-19T16:09:54.017 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:09:54.133 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:09:54.159 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:09:54.235 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:09:54.263 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 13/202 2024-02-19T16:09:54.432 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-19T16:09:54.478 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-19T16:09:54.506 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:09:54.602 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:09:54.631 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 17/202 2024-02-19T16:09:54.680 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 17/202 2024-02-19T16:09:54.710 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-19T16:09:54.733 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-19T16:09:54.749 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-19T16:09:54.969 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-19T16:09:54.998 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-19T16:09:55.037 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-19T16:09:55.214 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-19T16:09:55.336 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-19T16:09:55.396 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-19T16:09:55.478 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:09:55.522 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:09:55.728 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:09:55.839 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-19T16:09:56.060 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-19T16:09:56.094 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-19T16:09:56.165 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-19T16:09:56.226 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-19T16:09:56.256 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-19T16:09:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-19T16:09:56.332 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-19T16:09:56.397 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-19T16:09:56.431 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-19T16:09:56.475 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-19T16:09:56.507 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-19T16:09:56.535 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-19T16:09:56.606 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-19T16:09:56.640 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9. 39/202 2024-02-19T16:09:56.685 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el 40/202 2024-02-19T16:09:56.712 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_6 41/202 2024-02-19T16:09:56.784 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-19T16:09:56.827 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-19T16:09:56.902 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-19T16:09:57.040 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-19T16:09:57.088 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-19T16:09:57.647 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-19T16:09:57.785 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-19T16:09:57.908 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-19T16:09:57.977 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-19T16:09:57.987 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-19T16:09:58.025 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-19T16:09:58.062 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_6 52/202 2024-02-19T16:09:58.102 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-19T16:09:58.204 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-19T16:09:58.271 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-19T16:09:58.722 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-19T16:09:58.928 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-19T16:09:59.007 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-19T16:09:59.051 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-19T16:09:59.202 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-19T16:09:59.253 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-19T16:09:59.319 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-19T16:09:59.393 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-19T16:09:59.428 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-19T16:09:59.465 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-19T16:09:59.513 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-19T16:09:59.548 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-19T16:09:59.592 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-19T16:09:59.634 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-19T16:09:59.680 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-02-19T16:09:59.784 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-19T16:09:59.796 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-19T16:09:59.822 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-19T16:09:59.830 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-19T16:09:59.853 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-19T16:09:59.860 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-19T16:09:59.873 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-19T16:09:59.903 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-19T16:09:59.932 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-19T16:09:59.971 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-19T16:10:00.002 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-19T16:10:00.024 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-19T16:10:00.043 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-19T16:10:00.066 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-19T16:10:00.084 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-19T16:10:00.091 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-19T16:10:00.109 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-19T16:10:00.134 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-19T16:10:00.134 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-19T16:10:00.219 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-19T16:10:00.350 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-19T16:10:00.457 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-19T16:10:00.458 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-19T16:10:00.509 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-19T16:10:00.517 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:10:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-19T16:10:00.621 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:10:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-19T16:10:00.643 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:10:00.717 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:10:00.747 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 13/202 2024-02-19T16:10:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-19T16:10:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-19T16:10:00.899 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-19T16:10:00.944 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-19T16:10:00.972 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:10:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-19T16:10:01.058 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:10:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-19T16:10:01.082 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 17/202 2024-02-19T16:10:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-19T16:10:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-19T16:10:01.129 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 17/202 2024-02-19T16:10:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-19T16:10:01.169 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-19T16:10:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-19T16:10:01.192 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-19T16:10:01.209 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-19T16:10:01.220 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-19T16:10:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-19T16:10:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-19T16:10:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-19T16:10:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-19T16:10:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-19T16:10:01.397 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-19T16:10:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-19T16:10:01.429 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-19T16:10:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-19T16:10:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-19T16:10:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-19T16:10:01.535 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-19T16:10:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-19T16:10:01.649 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-19T16:10:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-19T16:10:01.700 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-19T16:10:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-19T16:10:01.772 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:10:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-19T16:10:01.811 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:10:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-19T16:10:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-19T16:10:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-19T16:10:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-19T16:10:02.041 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:10:02.150 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-19T16:10:02.248 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-19T16:10:02.282 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-19T16:10:02.352 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-19T16:10:02.405 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-19T16:10:02.435 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-19T16:10:02.471 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-19T16:10:02.511 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-19T16:10:02.576 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-19T16:10:02.610 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-19T16:10:02.655 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-19T16:10:02.686 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-19T16:10:02.714 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-19T16:10:02.784 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-19T16:10:02.819 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9. 39/202 2024-02-19T16:10:02.825 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-19T16:10:02.864 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el 40/202 2024-02-19T16:10:02.871 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-19T16:10:02.891 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_6 41/202 2024-02-19T16:10:02.904 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-19T16:10:02.953 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-19T16:10:02.961 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-19T16:10:03.005 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-19T16:10:03.043 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-19T16:10:03.079 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-19T16:10:03.080 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-19T16:10:03.212 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-19T16:10:03.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-19T16:10:03.263 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-19T16:10:03.295 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-19T16:10:03.374 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-19T16:10:03.774 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-19T16:10:03.854 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-19T16:10:03.938 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-19T16:10:03.990 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-19T16:10:03.999 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-19T16:10:04.037 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-19T16:10:04.056 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-19T16:10:04.075 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_6 52/202 2024-02-19T16:10:04.106 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-19T16:10:04.178 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-19T16:10:04.201 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-19T16:10:04.240 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-19T16:10:04.476 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-19T16:10:04.647 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-19T16:10:04.693 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-19T16:10:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-19T16:10:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-19T16:10:05.037 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-19T16:10:05.108 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-19T16:10:05.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-19T16:10:05.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-19T16:10:05.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-19T16:10:05.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-19T16:10:05.329 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-19T16:10:05.359 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-19T16:10:05.401 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-19T16:10:05.481 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-19T16:10:05.535 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-19T16:10:05.557 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-19T16:10:05.590 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-19T16:10:05.592 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-19T16:10:05.620 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-19T16:10:05.662 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-19T16:10:05.757 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-19T16:10:05.795 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-19T16:10:05.836 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-19T16:10:05.861 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-19T16:10:05.886 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-19T16:10:05.915 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-19T16:10:05.947 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 125/202 2024-02-19T16:10:05.958 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-19T16:10:06.021 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 125/202 2024-02-19T16:10:06.052 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 126/202 2024-02-19T16:10:06.079 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-19T16:10:06.088 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-19T16:10:06.179 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-19T16:10:06.228 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-19T16:10:06.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-19T16:10:06.333 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-19T16:10:06.519 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-19T16:10:06.566 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-19T16:10:06.568 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-19T16:10:06.592 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 129/202 2024-02-19T16:10:06.676 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 129/202 2024-02-19T16:10:06.701 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 130/202 2024-02-19T16:10:06.704 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-19T16:10:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-19T16:10:06.780 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-19T16:10:06.812 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-19T16:10:06.847 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-19T16:10:06.878 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-19T16:10:06.915 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-19T16:10:06.949 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-19T16:10:06.988 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-19T16:10:07.020 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-19T16:10:07.050 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-19T16:10:07.126 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-19T16:10:07.160 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-19T16:10:07.194 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-19T16:10:07.235 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-19T16:10:07.269 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-19T16:10:07.360 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-19T16:10:07.449 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-19T16:10:07.501 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-19T16:10:07.532 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-19T16:10:07.579 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-19T16:10:07.676 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-19T16:10:07.725 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-19T16:10:08.451 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-19T16:10:08.498 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-19T16:10:08.532 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-19T16:10:08.580 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-19T16:10:08.669 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-19T16:10:08.707 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-19T16:10:08.885 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-19T16:10:08.914 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-19T16:10:08.970 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:08.991 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-19T16:10:09.006 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:09.585 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:09.624 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-19T16:10:09.626 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 132/202 2024-02-19T16:10:09.719 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 132/202 2024-02-19T16:10:09.719 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-19T16:10:09.719 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-19T16:10:09.719 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:09.738 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-19T16:10:09.761 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 133/202 2024-02-19T16:10:10.967 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-19T16:10:11.052 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-19T16:10:11.079 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-19T16:10:11.200 INFO:teuthology.orchestra.run.smithi191.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 12 kB/s | 309 kB 00:25 2024-02-19T16:10:11.225 INFO:teuthology.orchestra.run.smithi191.stdout:(108/176): abseil-cpp-20211102.0-3.el9.x86_64.r 21 kB/s | 548 kB 00:25 2024-02-19T16:10:11.386 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-19T16:10:11.392 INFO:teuthology.orchestra.run.smithi191.stdout:(109/176): libarrow-9.0.0-10.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-02-19T16:10:11.409 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 125/202 2024-02-19T16:10:11.417 INFO:teuthology.orchestra.run.smithi191.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 135 kB/s | 26 kB 00:00 2024-02-19T16:10:11.443 INFO:teuthology.orchestra.run.smithi191.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-02-19T16:10:11.468 INFO:teuthology.orchestra.run.smithi191.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-02-19T16:10:11.481 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 125/202 2024-02-19T16:10:11.493 INFO:teuthology.orchestra.run.smithi191.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-02-19T16:10:11.514 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 126/202 2024-02-19T16:10:11.527 INFO:teuthology.orchestra.run.smithi191.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-02-19T16:10:11.550 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-19T16:10:11.596 INFO:teuthology.orchestra.run.smithi191.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 5.2 MB/s | 548 kB 00:00 2024-02-19T16:10:11.619 INFO:teuthology.orchestra.run.smithi191.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 478 kB/s | 43 kB 00:00 2024-02-19T16:10:11.636 INFO:teuthology.orchestra.run.smithi191.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 823 kB/s | 32 kB 00:00 2024-02-19T16:10:11.661 INFO:teuthology.orchestra.run.smithi191.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-02-19T16:10:11.686 INFO:teuthology.orchestra.run.smithi191.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-02-19T16:10:11.720 INFO:teuthology.orchestra.run.smithi191.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-02-19T16:10:11.754 INFO:teuthology.orchestra.run.smithi191.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-02-19T16:10:11.770 INFO:teuthology.orchestra.run.smithi191.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 711 kB/s | 35 kB 00:00 2024-02-19T16:10:11.804 INFO:teuthology.orchestra.run.smithi191.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-02-19T16:10:11.880 INFO:teuthology.orchestra.run.smithi191.stdout:(124/176): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-02-19T16:10:11.905 INFO:teuthology.orchestra.run.smithi191.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-02-19T16:10:11.922 INFO:teuthology.orchestra.run.smithi191.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-02-19T16:10:11.947 INFO:teuthology.orchestra.run.smithi191.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-02-19T16:10:11.964 INFO:teuthology.orchestra.run.smithi191.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 555 kB/s | 23 kB 00:00 2024-02-19T16:10:11.990 INFO:teuthology.orchestra.run.smithi191.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-02-19T16:10:12.004 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-19T16:10:12.015 INFO:teuthology.orchestra.run.smithi191.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-02-19T16:10:12.037 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 129/202 2024-02-19T16:10:12.057 INFO:teuthology.orchestra.run.smithi191.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-02-19T16:10:12.074 INFO:teuthology.orchestra.run.smithi191.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 806 kB/s | 46 kB 00:00 2024-02-19T16:10:12.107 INFO:teuthology.orchestra.run.smithi191.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-02-19T16:10:12.108 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 129/202 2024-02-19T16:10:12.130 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 130/202 2024-02-19T16:10:12.141 INFO:teuthology.orchestra.run.smithi191.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 859 kB/s | 58 kB 00:00 2024-02-19T16:10:12.166 INFO:teuthology.orchestra.run.smithi191.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-02-19T16:10:12.192 INFO:teuthology.orchestra.run.smithi191.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 330 kB/s | 16 kB 00:00 2024-02-19T16:10:12.217 INFO:teuthology.orchestra.run.smithi191.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-02-19T16:10:12.242 INFO:teuthology.orchestra.run.smithi191.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-02-19T16:10:12.268 INFO:teuthology.orchestra.run.smithi191.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-02-19T16:10:12.293 INFO:teuthology.orchestra.run.smithi191.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-02-19T16:10:12.318 INFO:teuthology.orchestra.run.smithi191.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-02-19T16:10:12.344 INFO:teuthology.orchestra.run.smithi191.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-02-19T16:10:12.369 INFO:teuthology.orchestra.run.smithi191.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-02-19T16:10:12.394 INFO:teuthology.orchestra.run.smithi191.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-02-19T16:10:12.419 INFO:teuthology.orchestra.run.smithi191.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-02-19T16:10:12.456 INFO:teuthology.orchestra.run.smithi191.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-02-19T16:10:12.478 INFO:teuthology.orchestra.run.smithi191.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-02-19T16:10:12.503 INFO:teuthology.orchestra.run.smithi191.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-02-19T16:10:12.562 INFO:teuthology.orchestra.run.smithi191.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-02-19T16:10:12.895 INFO:teuthology.orchestra.run.smithi191.stdout:(150/176): librbd1-19.0.0-1451.g9f8908ee.el9.x8 9.4 MB/s | 3.1 MB 00:00 2024-02-19T16:10:12.937 INFO:teuthology.orchestra.run.smithi191.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-02-19T16:10:13.037 INFO:teuthology.orchestra.run.smithi191.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 23 MB/s | 2.3 MB 00:00 2024-02-19T16:10:13.113 INFO:teuthology.orchestra.run.smithi191.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-02-19T16:10:13.154 INFO:teuthology.orchestra.run.smithi191.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 996 kB/s | 41 kB 00:00 2024-02-19T16:10:13.196 INFO:teuthology.orchestra.run.smithi191.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 5.8 MB/s | 246 kB 00:00 2024-02-19T16:10:13.230 INFO:teuthology.orchestra.run.smithi191.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-02-19T16:10:13.264 INFO:teuthology.orchestra.run.smithi191.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-02-19T16:10:13.305 INFO:teuthology.orchestra.run.smithi191.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-02-19T16:10:13.347 INFO:teuthology.orchestra.run.smithi191.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-02-19T16:10:13.406 INFO:teuthology.orchestra.run.smithi191.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-19T16:10:13.448 INFO:teuthology.orchestra.run.smithi191.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 10 MB/s | 433 kB 00:00 2024-02-19T16:10:13.473 INFO:teuthology.orchestra.run.smithi191.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 884 kB/s | 22 kB 00:00 2024-02-19T16:10:13.507 INFO:teuthology.orchestra.run.smithi191.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 107 kB 00:00 2024-02-19T16:10:13.540 INFO:teuthology.orchestra.run.smithi191.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 154 kB 00:00 2024-02-19T16:10:13.575 INFO:teuthology.orchestra.run.smithi191.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 3.9 MB/s | 135 kB 00:00 2024-02-19T16:10:13.608 INFO:teuthology.orchestra.run.smithi191.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.8 MB/s | 63 kB 00:00 2024-02-19T16:10:13.675 INFO:teuthology.orchestra.run.smithi191.stdout:(167/176): librados2-19.0.0-1451.g9f8908ee.el9. 2.9 MB/s | 3.4 MB 00:01 2024-02-19T16:10:13.709 INFO:teuthology.orchestra.run.smithi191.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 275 kB/s | 28 kB 00:00 2024-02-19T16:10:13.751 INFO:teuthology.orchestra.run.smithi191.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-02-19T16:10:13.809 INFO:teuthology.orchestra.run.smithi191.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 22 MB/s | 2.1 MB 00:00 2024-02-19T16:10:13.859 INFO:teuthology.orchestra.run.smithi191.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.2 MB 00:00 2024-02-19T16:10:13.885 INFO:teuthology.orchestra.run.smithi191.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-02-19T16:10:13.910 INFO:teuthology.orchestra.run.smithi191.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 625 kB/s | 30 kB 00:00 2024-02-19T16:10:13.935 INFO:teuthology.orchestra.run.smithi191.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-02-19T16:10:13.969 INFO:teuthology.orchestra.run.smithi191.stdout:(175/176): grpc-data-1.46.7-10.el9.noarch.rpm 716 B/s | 19 kB 00:27 2024-02-19T16:10:14.002 INFO:teuthology.orchestra.run.smithi191.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-19T16:10:14.007 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-19T16:10:14.007 INFO:teuthology.orchestra.run.smithi191.stdout:Total 5.6 MB/s | 223 MB 00:39 2024-02-19T16:10:14.049 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-19T16:10:14.320 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-19T16:10:14.320 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-19T16:10:14.344 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:14.377 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:14.914 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:14.956 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 132/202 2024-02-19T16:10:15.020 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 132/202 2024-02-19T16:10:15.020 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-19T16:10:15.020 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-19T16:10:15.020 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:15.049 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 133/202 2024-02-19T16:10:15.990 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-19T16:10:15.990 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-19T16:10:22.471 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 133/202 2024-02-19T16:10:22.471 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /sys 2024-02-19T16:10:22.471 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /proc 2024-02-19T16:10:22.471 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /mnt 2024-02-19T16:10:22.472 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /var/tmp 2024-02-19T16:10:22.472 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /home 2024-02-19T16:10:22.472 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /root 2024-02-19T16:10:22.472 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /tmp 2024-02-19T16:10:22.472 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:22.649 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 134/202 2024-02-19T16:10:22.665 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 134/202 2024-02-19T16:10:22.665 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:22.665 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-19T16:10:22.665 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:22.898 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-19T16:10:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-19T16:10:23.039 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-19T16:10:23.079 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-19T16:10:23.101 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-19T16:10:23.157 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-19T16:10:23.195 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-19T16:10:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-19T16:10:23.283 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-19T16:10:23.309 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-19T16:10:23.351 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-19T16:10:23.670 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-19T16:10:23.735 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:10:23.855 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:10:23.894 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:10:23.985 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:10:24.014 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 13/202 2024-02-19T16:10:24.162 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-19T16:10:24.196 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 135/202 2024-02-19T16:10:24.211 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-19T16:10:24.221 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 135/202 2024-02-19T16:10:24.221 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:24.222 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-19T16:10:24.222 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T16:10:24.222 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T16:10:24.222 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:24.240 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:10:24.326 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:10:24.349 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 17/202 2024-02-19T16:10:24.396 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 17/202 2024-02-19T16:10:24.436 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-19T16:10:24.455 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 136/202 2024-02-19T16:10:24.458 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-19T16:10:24.474 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-19T16:10:24.477 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 136/202 2024-02-19T16:10:24.477 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:24.477 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-19T16:10:24.477 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T16:10:24.477 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T16:10:24.477 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:24.642 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-19T16:10:24.672 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-19T16:10:24.704 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-19T16:10:24.802 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-19T16:10:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-19T16:10:24.947 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 137/202 2024-02-19T16:10:24.950 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-19T16:10:24.970 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 137/202 2024-02-19T16:10:24.970 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:24.970 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-19T16:10:24.970 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T16:10:24.970 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T16:10:24.970 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:25.021 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:10:25.066 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-19T16:10:25.076 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:10:25.303 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-19T16:10:25.416 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-19T16:10:25.441 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 139/202 2024-02-19T16:10:25.472 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 139/202 2024-02-19T16:10:25.504 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.e 140/202 2024-02-19T16:10:25.508 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-19T16:10:25.540 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-19T16:10:25.566 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee. 141/202 2024-02-19T16:10:25.595 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 142/202 2024-02-19T16:10:25.599 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-19T16:10:25.653 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-19T16:10:25.684 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-19T16:10:25.719 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-19T16:10:25.760 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-19T16:10:25.823 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-19T16:10:25.836 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 142/202 2024-02-19T16:10:25.858 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-19T16:10:25.871 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 143/202 2024-02-19T16:10:25.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-19T16:10:25.934 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-19T16:10:25.963 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-19T16:10:26.032 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-19T16:10:26.068 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9. 39/202 2024-02-19T16:10:26.111 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el 40/202 2024-02-19T16:10:26.139 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_6 41/202 2024-02-19T16:10:26.208 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-19T16:10:26.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-19T16:10:26.327 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-19T16:10:26.470 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-19T16:10:26.540 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-19T16:10:26.890 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 143/202 2024-02-19T16:10:26.913 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 144/202 2024-02-19T16:10:27.022 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 144/202 2024-02-19T16:10:27.059 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 133/202 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2024-02-19T16:10:27.067 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2024-02-19T16:10:27.068 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2024-02-19T16:10:27.068 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:27.134 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-19T16:10:27.192 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el 145/202 2024-02-19T16:10:27.222 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 134/202 2024-02-19T16:10:27.224 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 146/202 2024-02-19T16:10:27.225 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-19T16:10:27.237 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 134/202 2024-02-19T16:10:27.237 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:27.237 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-19T16:10:27.237 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:27.247 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 146/202 2024-02-19T16:10:27.247 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:27.247 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-19T16:10:27.247 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T16:10:27.247 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T16:10:27.247 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:27.263 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-19T16:10:27.270 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-19T16:10:27.291 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 147/202 2024-02-19T16:10:27.302 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 147/202 2024-02-19T16:10:27.310 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-19T16:10:27.348 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_6 52/202 2024-02-19T16:10:27.356 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 148/202 2024-02-19T16:10:27.379 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-19T16:10:27.467 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-19T16:10:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-19T16:10:27.759 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-19T16:10:27.917 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-19T16:10:27.998 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-19T16:10:28.043 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-19T16:10:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-19T16:10:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-19T16:10:28.261 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-19T16:10:28.293 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 150/202 2024-02-19T16:10:28.301 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-19T16:10:28.316 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 150/202 2024-02-19T16:10:28.316 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:28.316 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-19T16:10:28.316 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T16:10:28.317 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T16:10:28.317 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:28.370 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1451.g9f890 151/202 2024-02-19T16:10:28.373 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-19T16:10:28.391 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1451.g9f890 151/202 2024-02-19T16:10:28.391 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:28.392 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-19T16:10:28.392 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:28.404 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-19T16:10:28.432 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-19T16:10:28.445 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 152/202 2024-02-19T16:10:28.467 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 152/202 2024-02-19T16:10:28.467 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:28.467 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-19T16:10:28.467 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-19T16:10:28.467 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-19T16:10:28.467 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:28.487 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-19T16:10:28.523 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-19T16:10:28.567 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-19T16:10:28.609 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-19T16:10:28.708 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 135/202 2024-02-19T16:10:28.739 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 135/202 2024-02-19T16:10:28.740 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:28.740 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-19T16:10:28.740 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T16:10:28.740 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T16:10:28.740 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:28.741 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-19T16:10:28.788 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:28.789 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-19T16:10:28.810 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:28.810 INFO:teuthology.orchestra.run.smithi008.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:28.810 INFO:teuthology.orchestra.run.smithi008.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-19T16:10:28.810 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T16:10:28.810 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T16:10:28.810 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:28.820 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-19T16:10:28.862 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-19T16:10:28.955 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-19T16:10:28.976 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 136/202 2024-02-19T16:10:28.994 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-19T16:10:28.999 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 136/202 2024-02-19T16:10:28.999 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:28.999 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-19T16:10:28.999 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T16:10:28.999 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T16:10:28.999 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:29.034 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-19T16:10:29.060 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-19T16:10:29.085 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-19T16:10:29.157 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-19T16:10:29.277 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-19T16:10:29.375 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-19T16:10:29.426 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-19T16:10:29.453 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 137/202 2024-02-19T16:10:29.475 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-19T16:10:29.476 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 137/202 2024-02-19T16:10:29.476 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:29.476 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-19T16:10:29.476 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T16:10:29.476 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T16:10:29.476 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:29.530 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-19T16:10:29.572 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-19T16:10:29.712 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-19T16:10:29.763 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-19T16:10:29.892 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-19T16:10:29.936 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-19T16:10:29.966 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 139/202 2024-02-19T16:10:29.969 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-19T16:10:29.997 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 139/202 2024-02-19T16:10:30.001 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-19T16:10:30.028 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.e 140/202 2024-02-19T16:10:30.036 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-19T16:10:30.067 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-19T16:10:30.080 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee. 141/202 2024-02-19T16:10:30.095 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-19T16:10:30.103 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 142/202 2024-02-19T16:10:30.151 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-19T16:10:30.194 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-19T16:10:30.225 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-19T16:10:30.255 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-19T16:10:30.330 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 142/202 2024-02-19T16:10:30.331 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-19T16:10:30.362 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 143/202 2024-02-19T16:10:30.365 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-19T16:10:30.399 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-19T16:10:30.440 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-19T16:10:30.474 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-19T16:10:30.563 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-19T16:10:30.645 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-19T16:10:30.689 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-19T16:10:30.720 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-19T16:10:30.767 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-19T16:10:30.864 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-19T16:10:30.921 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-19T16:10:31.325 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 143/202 2024-02-19T16:10:31.355 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 144/202 2024-02-19T16:10:31.455 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 144/202 2024-02-19T16:10:31.609 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el 145/202 2024-02-19T16:10:31.631 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 146/202 2024-02-19T16:10:31.651 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-19T16:10:31.655 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 146/202 2024-02-19T16:10:31.655 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:31.655 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-19T16:10:31.655 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T16:10:31.655 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T16:10:31.655 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:31.693 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-19T16:10:31.699 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 147/202 2024-02-19T16:10:31.710 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 147/202 2024-02-19T16:10:31.727 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-19T16:10:31.747 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 148/202 2024-02-19T16:10:31.775 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-19T16:10:31.863 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-19T16:10:31.895 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-19T16:10:32.072 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-19T16:10:32.109 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-19T16:10:32.187 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-19T16:10:32.610 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-19T16:10:32.651 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 150/202 2024-02-19T16:10:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 150/202 2024-02-19T16:10:32.674 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:32.674 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-19T16:10:32.674 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T16:10:32.674 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T16:10:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:32.718 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1451.g9f890 151/202 2024-02-19T16:10:32.741 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1451.g9f890 151/202 2024-02-19T16:10:32.741 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:32.741 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-19T16:10:32.741 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:32.786 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 152/202 2024-02-19T16:10:32.808 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 152/202 2024-02-19T16:10:32.808 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:32.808 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-19T16:10:32.809 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-19T16:10:32.809 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-19T16:10:32.809 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:32.824 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-19T16:10:32.939 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-19T16:10:33.120 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:33.143 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:33.143 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:33.143 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-19T16:10:33.144 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T16:10:33.144 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T16:10:33.144 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:33.517 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 154/202 2024-02-19T16:10:33.558 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 155/202 2024-02-19T16:10:33.589 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 156/202 2024-02-19T16:10:33.632 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-19T16:10:33.837 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 158/202 2024-02-19T16:10:33.961 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-19T16:10:34.006 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 160/202 2024-02-19T16:10:34.070 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-19T16:10:34.131 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-19T16:10:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-19T16:10:34.211 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-19T16:10:34.238 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-19T16:10:34.249 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-19T16:10:34.264 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-19T16:10:34.293 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-19T16:10:34.324 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_ 166/202 2024-02-19T16:10:34.353 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-19T16:10:34.423 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-19T16:10:34.483 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-19T16:10:34.518 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-19T16:10:34.573 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-19T16:10:34.605 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 125/202 2024-02-19T16:10:34.695 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 125/202 2024-02-19T16:10:34.726 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 126/202 2024-02-19T16:10:34.970 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-19T16:10:35.083 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:10:35.121 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:10:35.145 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:10:35.220 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-19T16:10:35.249 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 129/202 2024-02-19T16:10:35.320 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 129/202 2024-02-19T16:10:35.342 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 130/202 2024-02-19T16:10:36.033 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 172/202 2024-02-19T16:10:36.105 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-19T16:10:36.147 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-19T16:10:36.242 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-19T16:10:36.275 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-19T16:10:36.309 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-19T16:10:36.334 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-19T16:10:36.361 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-19T16:10:36.361 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-19T16:10:36.437 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-19T16:10:36.465 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-19T16:10:36.495 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-19T16:10:36.495 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-19T16:10:36.589 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-19T16:10:36.621 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-19T16:10:36.665 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-19T16:10:36.690 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-19T16:10:36.717 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-19T16:10:36.750 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-19T16:10:36.784 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-19T16:10:36.834 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-19T16:10:36.871 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-19T16:10:36.898 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-19T16:10:36.922 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-19T16:10:36.948 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-19T16:10:36.972 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-19T16:10:36.979 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:10:36.979 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:10:37.167 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:10:37.167 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-19T16:10:37.250 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-19T16:10:37.275 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-19T16:10:37.300 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-19T16:10:37.325 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-19T16:10:37.358 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-19T16:10:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-19T16:10:37.559 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 154/202 2024-02-19T16:10:37.599 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 155/202 2024-02-19T16:10:37.608 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:37.630 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 156/202 2024-02-19T16:10:37.648 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:37.657 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-19T16:10:37.838 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 158/202 2024-02-19T16:10:37.956 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-19T16:10:38.005 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 160/202 2024-02-19T16:10:38.045 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-19T16:10:38.089 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-19T16:10:38.156 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-19T16:10:38.187 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 131/202 2024-02-19T16:10:38.199 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-19T16:10:38.235 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-19T16:10:38.266 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_ 166/202 2024-02-19T16:10:38.294 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-19T16:10:38.364 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-19T16:10:38.409 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-19T16:10:38.452 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-19T16:10:38.455 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 132/202 2024-02-19T16:10:38.516 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 132/202 2024-02-19T16:10:38.516 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-19T16:10:38.516 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-19T16:10:38.516 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:38.545 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 133/202 2024-02-19T16:10:38.974 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:10:39.017 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:10:39.038 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:10:39.852 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 172/202 2024-02-19T16:10:39.929 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-19T16:10:39.970 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-19T16:10:40.052 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-19T16:10:40.125 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-19T16:10:40.150 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-19T16:10:40.175 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-19T16:10:40.201 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-19T16:10:40.201 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-19T16:10:40.295 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-19T16:10:40.322 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-19T16:10:40.353 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-19T16:10:40.353 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-19T16:10:40.437 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-19T16:10:40.464 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-19T16:10:40.497 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-19T16:10:40.531 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-19T16:10:40.558 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-19T16:10:40.591 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-19T16:10:40.624 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-19T16:10:40.672 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-19T16:10:40.712 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-19T16:10:40.738 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-19T16:10:40.763 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-19T16:10:40.788 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-19T16:10:40.813 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-19T16:10:40.820 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:10:40.820 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:10:41.000 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:10:41.000 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-19T16:10:41.082 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-19T16:10:41.107 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-19T16:10:41.133 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-19T16:10:41.158 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-19T16:10:41.190 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-19T16:10:41.256 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-19T16:10:42.277 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-19T16:10:42.277 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 1/202 2024-02-19T16:10:42.277 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 2/202 2024-02-19T16:10:42.277 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 3/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 4/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1451.g9f890 5/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 6/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 7/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 8/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 9/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 10/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 13/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 14/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_ 15/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:10:42.278 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 17/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_6 18/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 19/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 20/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el 21/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9. 22/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_6 23/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 24/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 25/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 26/202 2024-02-19T16:10:42.279 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 27/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 28/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 29/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee. 30/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 31/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 32/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 33/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el 34/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 35/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.e 36/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 37/202 2024-02-19T16:10:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 38/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 39/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-19T16:10:42.281 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-19T16:10:42.282 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-19T16:10:42.283 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-19T16:10:42.284 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-19T16:10:42.285 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-19T16:10:42.286 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-19T16:10:42.287 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-19T16:10:42.288 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-19T16:10:42.289 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 151/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-19T16:10:42.290 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-19T16:10:42.291 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-19T16:10:42.292 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-19T16:10:42.293 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-19T16:10:45.146 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout:Upgraded: 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: acl-2.3.1-4.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-19T16:10:45.147 INFO:teuthology.orchestra.run.smithi008.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-19T16:10:45.148 INFO:teuthology.orchestra.run.smithi008.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: automake-1.16.2-8.el9.noarch 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: bison-3.7.4-5.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.149 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.150 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: flex-2.6.4-9.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-19T16:10:45.151 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: jq-1.6-16.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-19T16:10:45.152 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: lua-5.4.4-4.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-19T16:10:45.153 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-13.11-1.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-19T16:10:45.154 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-19T16:10:45.155 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-19T16:10:45.156 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ply-3.11-14.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-19T16:10:45.157 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-19T16:10:45.158 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: re2-1:20211101-3.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-19T16:10:45.159 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-19T16:10:45.160 INFO:teuthology.orchestra.run.smithi008.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-19T16:10:45.160 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:10:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-02-19T16:10:45.638 DEBUG:teuthology.parallel:result is None 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 1/202 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 2/202 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 3/202 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 4/202 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1451.g9f890 5/202 2024-02-19T16:10:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 6/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 7/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 8/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 9/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 10/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 13/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 14/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_ 15/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 17/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_6 18/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 19/202 2024-02-19T16:10:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 20/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el 21/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9. 22/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_6 23/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 24/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 25/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 26/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 27/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 28/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 29/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee. 30/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 31/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 32/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 33/202 2024-02-19T16:10:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el 34/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 35/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.e 36/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 37/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 38/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 39/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-19T16:10:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-19T16:10:46.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-19T16:10:46.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-19T16:10:46.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-19T16:10:46.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-19T16:10:46.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-19T16:10:46.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-19T16:10:46.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 151/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-19T16:10:46.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-19T16:10:46.210 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-19T16:10:46.211 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-19T16:10:46.212 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-19T16:10:48.944 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-19T16:10:48.944 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:48.944 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2024-02-19T16:10:48.944 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:48.944 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: acl-2.3.1-4.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.945 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-19T16:10:48.946 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.2-8.el9.noarch 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.7.4-5.el9.x86_64 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.951 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.951 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.951 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.951 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:10:48.952 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-19T16:10:48.952 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.4-9.el9.x86_64 2024-02-19T16:10:48.952 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-19T16:10:48.952 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-19T16:10:48.952 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-19T16:10:48.952 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-16.el9.x86_64 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-19T16:10:48.953 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.954 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.954 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.954 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-19T16:10:48.954 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-19T16:10:48.954 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-19T16:10:48.954 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-19T16:10:48.955 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: lua-5.4.4-4.el9.x86_64 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-19T16:10:48.956 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-13.11-1.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-19T16:10:48.957 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-19T16:10:48.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-19T16:10:48.959 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-19T16:10:48.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-19T16:10:48.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-19T16:10:48.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: re2-1:20211101-3.el9.x86_64 2024-02-19T16:10:48.964 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-19T16:10:48.965 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-19T16:10:48.965 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-19T16:10:48.965 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-19T16:10:48.965 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-19T16:10:48.965 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-19T16:10:48.965 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-19T16:10:48.966 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-02-19T16:10:49.410 DEBUG:teuthology.parallel:result is None 2024-02-19T16:10:50.459 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 133/202 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /root 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-02-19T16:10:50.460 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:50.623 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 134/202 2024-02-19T16:10:50.637 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 134/202 2024-02-19T16:10:50.637 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:50.637 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-19T16:10:50.637 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:52.090 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 135/202 2024-02-19T16:10:52.116 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 135/202 2024-02-19T16:10:52.116 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:52.116 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-19T16:10:52.116 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T16:10:52.116 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T16:10:52.116 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:52.350 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 136/202 2024-02-19T16:10:52.372 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 136/202 2024-02-19T16:10:52.372 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:52.372 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-19T16:10:52.372 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T16:10:52.372 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T16:10:52.372 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:52.844 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 137/202 2024-02-19T16:10:52.866 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 137/202 2024-02-19T16:10:52.866 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:52.866 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-19T16:10:52.866 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T16:10:52.866 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T16:10:52.867 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:52.962 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-19T16:10:53.348 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 139/202 2024-02-19T16:10:53.377 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 139/202 2024-02-19T16:10:53.408 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.e 140/202 2024-02-19T16:10:53.468 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee. 141/202 2024-02-19T16:10:53.491 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 142/202 2024-02-19T16:10:53.718 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 142/202 2024-02-19T16:10:53.751 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 143/202 2024-02-19T16:10:54.711 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 143/202 2024-02-19T16:10:54.743 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 144/202 2024-02-19T16:10:54.842 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 144/202 2024-02-19T16:10:55.006 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el 145/202 2024-02-19T16:10:55.037 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 146/202 2024-02-19T16:10:55.059 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 146/202 2024-02-19T16:10:55.059 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:55.059 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-19T16:10:55.060 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T16:10:55.060 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T16:10:55.060 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:55.104 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 147/202 2024-02-19T16:10:55.114 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 147/202 2024-02-19T16:10:55.160 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 148/202 2024-02-19T16:10:56.019 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-19T16:10:56.064 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 150/202 2024-02-19T16:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 150/202 2024-02-19T16:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-19T16:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T16:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T16:10:56.086 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:56.131 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1451.g9f890 151/202 2024-02-19T16:10:56.153 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1451.g9f890 151/202 2024-02-19T16:10:56.153 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:56.153 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-19T16:10:56.153 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:56.207 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 152/202 2024-02-19T16:10:56.229 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 152/202 2024-02-19T16:10:56.229 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:56.229 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-19T16:10:56.229 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-19T16:10:56.229 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-19T16:10:56.229 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:10:56.532 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:56.554 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:10:56.554 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T16:10:56.554 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-19T16:10:56.554 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T16:10:56.555 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T16:10:56.555 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:11:00.953 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 154/202 2024-02-19T16:11:00.995 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 155/202 2024-02-19T16:11:01.025 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 156/202 2024-02-19T16:11:01.052 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-19T16:11:01.235 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 158/202 2024-02-19T16:11:01.360 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-19T16:11:01.400 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 160/202 2024-02-19T16:11:01.439 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-19T16:11:01.492 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-19T16:11:01.575 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-19T16:11:01.618 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-19T16:11:01.662 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-19T16:11:01.684 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_ 166/202 2024-02-19T16:11:01.714 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-19T16:11:01.778 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-19T16:11:01.828 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-19T16:11:01.871 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-19T16:11:02.481 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:11:02.519 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:11:02.538 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-19T16:11:03.355 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 172/202 2024-02-19T16:11:03.431 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-19T16:11:03.472 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-19T16:11:03.552 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-19T16:11:03.601 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-19T16:11:03.626 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-19T16:11:03.651 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-19T16:11:03.678 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-19T16:11:03.678 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-19T16:11:03.754 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-19T16:11:03.798 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-19T16:11:03.829 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-19T16:11:03.829 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-19T16:11:03.897 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-19T16:11:03.923 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-19T16:11:03.965 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-19T16:11:03.990 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-19T16:11:04.017 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-19T16:11:04.050 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-19T16:11:04.082 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-19T16:11:04.122 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-19T16:11:04.154 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-19T16:11:04.180 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-19T16:11:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-19T16:11:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-19T16:11:04.256 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-19T16:11:04.261 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:11:04.262 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:11:04.441 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-19T16:11:04.442 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-19T16:11:04.533 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-19T16:11:04.558 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-19T16:11:04.583 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-19T16:11:04.608 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-19T16:11:04.641 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-19T16:11:04.706 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-19T16:11:09.738 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 1/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 2/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 3/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 4/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1451.g9f890 5/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 6/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 7/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 8/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 9/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 10/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 11/202 2024-02-19T16:11:09.739 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 12/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 13/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 14/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_ 15/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 16/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 17/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_6 18/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86 19/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 20/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el 21/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9. 22/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_6 23/202 2024-02-19T16:11:09.740 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 24/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 25/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 26/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 27/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 28/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 29/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee. 30/202 2024-02-19T16:11:09.760 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noa 31/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.n 32/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8 33/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el 34/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 35/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.e 36/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 37/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 38/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 39/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-19T16:11:09.761 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-19T16:11:09.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-19T16:11:09.763 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-19T16:11:09.764 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-19T16:11:09.765 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-19T16:11:09.766 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-19T16:11:09.767 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-19T16:11:09.768 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-19T16:11:09.769 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 151/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 153/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-19T16:11:09.770 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-19T16:11:09.771 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-19T16:11:09.772 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-19T16:11:09.773 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-19T16:11:09.774 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-19T16:11:09.774 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-19T16:11:09.774 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-19T16:11:09.774 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-19T16:11:09.774 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-19T16:11:12.632 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: acl-2.3.1-4.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-19T16:11:12.633 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.634 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-19T16:11:12.635 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.2-8.el9.noarch 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.7.4-5.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.636 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.637 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:19.0.0-1451.g9f8908ee.el9.noarch 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.4-9.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-19T16:11:12.638 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-16.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-19T16:11:12.639 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-19T16:11:12.640 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: lua-5.4.4-4.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-13.11-1.el9.x86_64 2024-02-19T16:11:12.641 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.642 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-19T16:11:12.643 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-19T16:11:12.644 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply-3.11-14.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-19T16:11:12.645 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-19T16:11:12.646 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.647 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:19.0.0-1451.g9f8908ee.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: re2-1:20211101-3.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-19T16:11:12.648 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:11:12.649 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-19T16:11:13.269 DEBUG:teuthology.parallel:result is None 2024-02-19T16:11:13.269 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:11:13.418 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-19T16:11:13.443 INFO:teuthology.orchestra.run.smithi008.stdout:19.0.0-1451.g9f8908ee.el9 2024-02-19T16:11:13.444 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1451.g9f8908ee.el9 2024-02-19T16:11:13.444 INFO:teuthology.task.install:The correct ceph version 19.0.0-1451.g9f8908ee is installed. 2024-02-19T16:11:13.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:11:13.586 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-19T16:11:13.612 INFO:teuthology.orchestra.run.smithi156.stdout:19.0.0-1451.g9f8908ee.el9 2024-02-19T16:11:13.612 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1451.g9f8908ee.el9 2024-02-19T16:11:13.613 INFO:teuthology.task.install:The correct ceph version 19.0.0-1451.g9f8908ee is installed. 2024-02-19T16:11:13.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:11:13.839 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-19T16:11:13.864 INFO:teuthology.orchestra.run.smithi191.stdout:19.0.0-1451.g9f8908ee.el9 2024-02-19T16:11:13.865 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1451.g9f8908ee.el9 2024-02-19T16:11:13.865 INFO:teuthology.task.install:The correct ceph version 19.0.0-1451.g9f8908ee is installed. 2024-02-19T16:11:13.866 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-19T16:11:13.867 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:11:13.867 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-19T16:11:13.902 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:11:13.902 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-19T16:11:13.937 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:11:13.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-19T16:11:13.971 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-19T16:11:13.972 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:11:13.972 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/daemon-helper 2024-02-19T16:11:14.003 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-19T16:11:14.071 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:11:14.071 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-02-19T16:11:14.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-19T16:11:14.174 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:11:14.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-02-19T16:11:14.206 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-19T16:11:14.276 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-19T16:11:14.276 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:11:14.276 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-19T16:11:14.302 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-19T16:11:14.370 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:11:14.370 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-19T16:11:14.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-19T16:11:14.467 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:11:14.467 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-19T16:11:14.498 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-19T16:11:14.570 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-19T16:11:14.570 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:11:14.571 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/stdin-killer 2024-02-19T16:11:14.598 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-19T16:11:14.665 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:11:14.665 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-02-19T16:11:14.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-19T16:11:14.759 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:11:14.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-02-19T16:11:14.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-19T16:11:14.862 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-19T16:11:15.130 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-19T16:11:15.131 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:11:15.131 INFO:tasks.cephadm:Cluster fsid is 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:11:15.131 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-19T16:11:15.132 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.8', 'mon.b': '172.21.15.156', 'mon.c': '172.21.15.191'} 2024-02-19T16:11:15.132 INFO:tasks.cephadm:First mon is mon.a on smithi008 2024-02-19T16:11:15.132 INFO:tasks.cephadm:First mgr is x 2024-02-19T16:11:15.132 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-19T16:11:15.132 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname $(hostname -s) 2024-02-19T16:11:15.160 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2024-02-19T16:11:15.187 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-02-19T16:11:15.219 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-19T16:11:15.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:11:15.436 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76782/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1451-g9f8908ee', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1451.g9f8908ee'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 14:43:35.953295', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-qdb', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-19T16:11:15.652 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-lusov-qdb, sha1 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:11:15.654 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/x86_64/flavors/default/cephadm 2024-02-19T16:11:15.654 DEBUG:teuthology.orchestra.run.smithi008:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:16.714 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r--. 1 ubuntu ubuntu 766964 Feb 19 16:11 /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:16.715 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:17.706 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r--. 1 ubuntu ubuntu 766964 Feb 19 16:11 /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:17.706 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:18.563 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-r--r--. 1 ubuntu ubuntu 766964 Feb 19 16:11 /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:18.564 DEBUG:teuthology.orchestra.run.smithi008:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:18.582 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:18.600 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-19T16:11:18.664 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee on all hosts... 2024-02-19T16:11:18.664 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee pull 2024-02-19T16:11:18.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee pull 2024-02-19T16:11:18.669 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee pull 2024-02-19T16:11:19.460 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee... 2024-02-19T16:11:19.478 INFO:teuthology.orchestra.run.smithi008.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee... 2024-02-19T16:11:19.552 INFO:teuthology.orchestra.run.smithi191.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee... 2024-02-19T16:11:45.933 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-19T16:11:45.933 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev)", 2024-02-19T16:11:45.934 INFO:teuthology.orchestra.run.smithi191.stdout: "image_id": "cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8", 2024-02-19T16:11:45.934 INFO:teuthology.orchestra.run.smithi191.stdout: "repo_digests": [ 2024-02-19T16:11:45.934 INFO:teuthology.orchestra.run.smithi191.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05" 2024-02-19T16:11:45.934 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-02-19T16:11:45.934 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-19T16:12:03.515 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-19T16:12:03.516 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev)", 2024-02-19T16:12:03.516 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8", 2024-02-19T16:12:03.516 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2024-02-19T16:12:03.516 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05" 2024-02-19T16:12:03.516 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2024-02-19T16:12:03.516 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-19T16:12:06.816 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:12:06.817 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_version": "ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev)", 2024-02-19T16:12:06.817 INFO:teuthology.orchestra.run.smithi008.stdout: "image_id": "cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8", 2024-02-19T16:12:06.817 INFO:teuthology.orchestra.run.smithi008.stdout: "repo_digests": [ 2024-02-19T16:12:06.817 INFO:teuthology.orchestra.run.smithi008.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05" 2024-02-19T16:12:06.817 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:12:06.817 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:12:06.840 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /etc/ceph 2024-02-19T16:12:06.893 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2024-02-19T16:12:06.928 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-02-19T16:12:06.963 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 777 /etc/ceph 2024-02-19T16:12:06.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2024-02-19T16:12:07.020 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-02-19T16:12:07.049 INFO:tasks.cephadm:Writing seed config... 2024-02-19T16:12:07.050 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-19T16:12:07.050 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-19T16:12:07.050 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-19T16:12:07.050 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-19T16:12:07.051 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-19T16:12:07.052 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-19T16:12:07.053 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-19T16:12:07.054 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-02-19T16:12:07.054 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:12:07.054 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-19T16:12:07.069 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 81b9a938-cf41-11ee-95bb-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-19T16:12:07.069 DEBUG:teuthology.orchestra.run.smithi008:mon.a> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a.service 2024-02-19T16:12:07.111 DEBUG:teuthology.orchestra.run.smithi008:mgr.x> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x.service 2024-02-19T16:12:07.154 INFO:tasks.cephadm:Bootstrapping... 2024-02-19T16:12:07.154 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee -v bootstrap --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.8 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:12:07.349 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-02-19T16:12:07.349 INFO:teuthology.orchestra.run.smithi008.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee', '-v', 'bootstrap', '--fsid', '81b9a938-cf41-11ee-95bb-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.8', '--skip-admin-label'] 2024-02-19T16:12:07.375 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-19T16:12:07.375 INFO:teuthology.orchestra.run.smithi008.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-19T16:12:07.375 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying podman|docker is present... 2024-02-19T16:12:07.398 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-19T16:12:07.398 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying lvm2 is present... 2024-02-19T16:12:07.398 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying time synchronization is in place... 2024-02-19T16:12:07.405 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-19T16:12:07.405 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-19T16:12:07.411 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-19T16:12:07.411 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-02-19T16:12:07.417 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout enabled 2024-02-19T16:12:07.423 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout active 2024-02-19T16:12:07.423 INFO:teuthology.orchestra.run.smithi008.stdout:Unit chronyd.service is enabled and running 2024-02-19T16:12:07.423 INFO:teuthology.orchestra.run.smithi008.stdout:Repeating the final host check... 2024-02-19T16:12:07.448 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-19T16:12:07.448 INFO:teuthology.orchestra.run.smithi008.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-19T16:12:07.449 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl is present 2024-02-19T16:12:07.449 INFO:teuthology.orchestra.run.smithi008.stdout:lvcreate is present 2024-02-19T16:12:07.452 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-19T16:12:07.453 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-19T16:12:07.458 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-19T16:12:07.458 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-02-19T16:12:07.464 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout enabled 2024-02-19T16:12:07.470 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout active 2024-02-19T16:12:07.470 INFO:teuthology.orchestra.run.smithi008.stdout:Unit chronyd.service is enabled and running 2024-02-19T16:12:07.470 INFO:teuthology.orchestra.run.smithi008.stdout:Host looks OK 2024-02-19T16:12:07.471 INFO:teuthology.orchestra.run.smithi008.stdout:Cluster fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:07.471 INFO:teuthology.orchestra.run.smithi008.stdout:Acquiring lock 139639986335696 on /run/cephadm/81b9a938-cf41-11ee-95bb-87774f69a715.lock 2024-02-19T16:12:07.471 INFO:teuthology.orchestra.run.smithi008.stdout:Lock 139639986335696 acquired on /run/cephadm/81b9a938-cf41-11ee-95bb-87774f69a715.lock 2024-02-19T16:12:07.471 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying IP 172.21.15.8 port 3300 ... 2024-02-19T16:12:07.471 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying IP 172.21.15.8 port 6789 ... 2024-02-19T16:12:07.471 INFO:teuthology.orchestra.run.smithi008.stdout:Base mon IP(s) is [172.21.15.8:3300, 172.21.15.8:6789], mon addrv is [v2:172.21.15.8:3300,v1:172.21.15.8:6789] 2024-02-19T16:12:07.474 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.8 metric 100 2024-02-19T16:12:07.475 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.8 metric 100 2024-02-19T16:12:07.477 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-19T16:12:07.477 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-19T16:12:07.477 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-19T16:12:07.480 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-19T16:12:07.480 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-19T16:12:07.480 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-19T16:12:07.480 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-19T16:12:07.480 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link noprefixroute 2024-02-19T16:12:07.480 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-19T16:12:07.481 INFO:teuthology.orchestra.run.smithi008.stdout:Mon IP `172.21.15.8` is in CIDR network `172.21.0.0/20` 2024-02-19T16:12:07.481 INFO:teuthology.orchestra.run.smithi008.stdout:Mon IP `172.21.15.8` is in CIDR network `172.21.0.0/20` 2024-02-19T16:12:07.481 INFO:teuthology.orchestra.run.smithi008.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-19T16:12:07.481 INFO:teuthology.orchestra.run.smithi008.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-19T16:12:07.482 INFO:teuthology.orchestra.run.smithi008.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee... 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee... 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Getting image source signatures 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying blob sha256:e117f746de3825d5ccd9ec40f852629f938b24a2ebbef8c1c251eb6fc2a156a1 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying config sha256:cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 2024-02-19T16:12:07.996 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-19T16:12:08.958 INFO:teuthology.orchestra.run.smithi008.stdout:ceph: stdout ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev) 2024-02-19T16:12:08.958 INFO:teuthology.orchestra.run.smithi008.stdout:Ceph version: ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev) 2024-02-19T16:12:08.958 INFO:teuthology.orchestra.run.smithi008.stdout:Extracting ceph user uid/gid from container image... 2024-02-19T16:12:09.879 INFO:teuthology.orchestra.run.smithi008.stdout:stat: stdout 167 167 2024-02-19T16:12:09.880 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial keys... 2024-02-19T16:12:10.826 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQDafdNlCghNDRAASj8teiyj1jujaSV9a6lEWw== 2024-02-19T16:12:11.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQDbfdNlZIyYChAAWuclnATUvQ4AQQ2AInKvGQ== 2024-02-19T16:12:12.704 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQDcfdNlMr99BxAATFxh4e4w9x0n+IF8lMkevA== 2024-02-19T16:12:12.705 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial monmap... 2024-02-19T16:12:13.931 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-19T16:12:13.931 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-19T16:12:13.931 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool for a [v2:172.21.15.8:3300,v1:172.21.15.8:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout:setting min_mon_release = quincy 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: set fsid to 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:12:13.932 INFO:teuthology.orchestra.run.smithi008.stdout:Creating mon... 2024-02-19T16:12:23.674 INFO:teuthology.orchestra.run.smithi008.stdout:create mon.a on 2024-02-19T16:12:23.892 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-19T16:12:24.035 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-19T16:12:24.225 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-81b9a938-cf41-11ee-95bb-87774f69a715.target → /etc/systemd/system/ceph-81b9a938-cf41-11ee-95bb-87774f69a715.target. 2024-02-19T16:12:24.226 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-81b9a938-cf41-11ee-95bb-87774f69a715.target → /etc/systemd/system/ceph-81b9a938-cf41-11ee-95bb-87774f69a715.target. 2024-02-19T16:12:24.550 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a 2024-02-19T16:12:24.551 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to reset failed state of unit ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a.service: Unit ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a.service not loaded. 2024-02-19T16:12:24.712 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-81b9a938-cf41-11ee-95bb-87774f69a715.target.wants/ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a.service → /etc/systemd/system/ceph-81b9a938-cf41-11ee-95bb-87774f69a715@.service. 2024-02-19T16:12:25.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:24 smithi008 podman[32184]: 2024-02-19 16:12:24.947836567 +0000 UTC m=+0.119664861 container create 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-19T16:12:25.221 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T16:12:25.221 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-02-19T16:12:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T16:12:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-02-19T16:12:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-02-19T16:12:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to enable service . firewalld.service is not available 2024-02-19T16:12:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mon to start... 2024-02-19T16:12:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mon... 2024-02-19T16:12:25.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:25 smithi008 podman[32184]: 2024-02-19 16:12:25.151815088 +0000 UTC m=+0.323643383 container init 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-19T16:12:25.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:25 smithi008 podman[32184]: 2024-02-19 16:12:25.154739269 +0000 UTC m=+0.326567564 container start 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-19T16:12:25.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:25 smithi008 bash[32184]: 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc 2024-02-19T16:12:25.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:25 smithi008 systemd[1]: Started Ceph mon.a for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:12:25.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:25 smithi008 ceph-mon[32203]: mkfs 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:25.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:25 smithi008 ceph-mon[32203]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-19T16:12:26.315 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:26 smithi008 ceph-mon[32203]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-19T16:12:26.316 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:26 smithi008 ceph-mon[32203]: monmap e1: 1 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:12:26.316 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:26 smithi008 ceph-mon[32203]: fsmap 2024-02-19T16:12:26.316 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:26 smithi008 ceph-mon[32203]: osdmap e1: 0 total, 0 up, 0 in 2024-02-19T16:12:26.316 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:26 smithi008 ceph-mon[32203]: mgrmap e1: no daemons active 2024-02-19T16:12:26.316 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:26 smithi008 ceph-mon[32203]: from='client.? 172.21.15.8:0/3358120445' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout cluster: 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout id: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout services: 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.518549s) 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout data: 2024-02-19T16:12:26.326 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout pgs: 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.660+0000 7fd4e5599700 1 Processor -- start 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.660+0000 7fd4e5599700 1 -- start start 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.660+0000 7fd4e5599700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e0106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.661+0000 7fd4e5599700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0107250 con 0x7fd4e0106860 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.661+0000 7fd4deffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e0106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.661+0000 7fd4deffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e0106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60418/0 (socket says 172.21.15.8:60418) 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.661+0000 7fd4deffd700 1 -- 172.21.15.8:0/2223395894 learned_addr learned my addr 172.21.15.8:0/2223395894 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:26.327 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.661+0000 7fd4deffd700 1 -- 172.21.15.8:0/2223395894 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0107a60 con 0x7fd4e0106860 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.661+0000 7fd4deffd700 1 --2- 172.21.15.8:0/2223395894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e0106c80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00098a0 tx=0x7fd4d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eae27c07cd512a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4ddffb700 1 -- 172.21.15.8:0/2223395894 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d0004160 con 0x7fd4e0106860 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4ddffb700 1 -- 172.21.15.8:0/2223395894 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00042c0 con 0x7fd4e0106860 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4ddffb700 1 -- 172.21.15.8:0/2223395894 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d00045a0 con 0x7fd4e0106860 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 -- 172.21.15.8:0/2223395894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 msgr2=0x7fd4e0106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 --2- 172.21.15.8:0/2223395894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e0106c80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00098a0 tx=0x7fd4d0009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 -- 172.21.15.8:0/2223395894 shutdown_connections 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 --2- 172.21.15.8:0/2223395894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e0106c80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 -- 172.21.15.8:0/2223395894 >> 172.21.15.8:0/2223395894 conn(0x7fd4e0101e20 msgr2=0x7fd4e0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 -- 172.21.15.8:0/2223395894 shutdown_connections 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.662+0000 7fd4e5599700 1 -- 172.21.15.8:0/2223395894 wait complete. 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4e5599700 1 Processor -- start 2024-02-19T16:12:26.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4e5599700 1 -- start start 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4e5599700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4e5599700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e019b3b0 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4deffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4deffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60424/0 (socket says 172.21.15.8:60424) 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.663+0000 7fd4deffd700 1 -- 172.21.15.8:0/3358120445 learned_addr learned my addr 172.21.15.8:0/3358120445 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4deffd700 1 -- 172.21.15.8:0/3358120445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0009550 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4deffd700 1 --2- 172.21.15.8:0/3358120445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e019ae70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0004000 tx=0x7fd4d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d0011630 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0011790 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e019b5b0 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d0011a30 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e019ba50 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd4d0022070 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.664+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd4d001c450 con 0x7fd4e0106860 2024-02-19T16:12:26.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.665+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e0194b40 con 0x7fd4e0106860 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.667+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd4d0011b90 con 0x7fd4e0106860 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.739+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd4e0107dd0 con 0x7fd4e0106860 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4c7fff700 1 -- 172.21.15.8:0/3358120445 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fd4d0011d30 con 0x7fd4e0106860 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 msgr2=0x7fd4e019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 --2- 172.21.15.8:0/3358120445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e019ae70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0004000 tx=0x7fd4d0004750 comp rx=0 tx=0).stop 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 shutdown_connections 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 --2- 172.21.15.8:0/3358120445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4e0106860 0x7fd4e019ae70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 >> 172.21.15.8:0/3358120445 conn(0x7fd4e0101e20 msgr2=0x7fd4e0191c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 shutdown_connections 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:25.740+0000 7fd4e5599700 1 -- 172.21.15.8:0/3358120445 wait complete. 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:mon is available 2024-02-19T16:12:26.330 INFO:teuthology.orchestra.run.smithi008.stdout:Assimilating anything we can from ceph.conf... 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [global] 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout fsid = 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.8:3300,v1:172.21.15.8:6789] 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-19T16:12:27.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [osd] 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.752+0000 7fdc18d6e700 1 Processor -- start 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.752+0000 7fdc18d6e700 1 -- start start 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc18d6e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc141082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc18d6e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc14108870 con 0x7fdc14107e80 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc1259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc141082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc1259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc141082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60438/0 (socket says 172.21.15.8:60438) 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc1259c700 1 -- 172.21.15.8:0/3939670556 learned_addr learned my addr 172.21.15.8:0/3939670556 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:27.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc1259c700 1 -- 172.21.15.8:0/3939670556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc141089b0 con 0x7fdc14107e80 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.753+0000 7fdc1259c700 1 --2- 172.21.15.8:0/3939670556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc141082a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098a0 tx=0x7fdc04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12713d92c73e5a10 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc1159a700 1 -- 172.21.15.8:0/3939670556 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc04004160 con 0x7fdc14107e80 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc1159a700 1 -- 172.21.15.8:0/3939670556 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc040042c0 con 0x7fdc14107e80 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc1159a700 1 -- 172.21.15.8:0/3939670556 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc040045a0 con 0x7fdc14107e80 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3939670556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 msgr2=0x7fdc141082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 --2- 172.21.15.8:0/3939670556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc141082a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098a0 tx=0x7fdc04009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3939670556 shutdown_connections 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 --2- 172.21.15.8:0/3939670556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc141082a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3939670556 >> 172.21.15.8:0/3939670556 conn(0x7fdc14101590 msgr2=0x7fdc141039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3939670556 shutdown_connections 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.754+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3939670556 wait complete. 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc18d6e700 1 Processor -- start 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc18d6e700 1 -- start start 2024-02-19T16:12:27.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc18d6e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc1419d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc18d6e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419d600 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc1259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc1419d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc1259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc1419d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60454/0 (socket says 172.21.15.8:60454) 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.755+0000 7fdc1259c700 1 -- 172.21.15.8:0/3787781308 learned_addr learned my addr 172.21.15.8:0/3787781308 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc1259c700 1 -- 172.21.15.8:0/3787781308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04009550 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc1259c700 1 --2- 172.21.15.8:0/3787781308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc1419d0c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdc04004000 tx=0x7fdc04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc04011630 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04011790 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1419d800 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc04011a30 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1419dca0 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdc04021070 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.756+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdc0401b8b0 con 0x7fdc14107e80 2024-02-19T16:12:27.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.757+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc14196db0 con 0x7fdc14107e80 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.759+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdc0401bb10 con 0x7fdc14107e80 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.830+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdc14105b30 con 0x7fdc14107e80 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.833+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fdc0401bd60 con 0x7fdc14107e80 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.834+0000 7fdc0b7fe700 1 -- 172.21.15.8:0/3787781308 <== mon.0 v2:172.21.15.8:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc04035a50 con 0x7fdc14107e80 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 msgr2=0x7fdc1419d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 --2- 172.21.15.8:0/3787781308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc1419d0c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdc04004000 tx=0x7fdc04004750 comp rx=0 tx=0).stop 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 shutdown_connections 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 --2- 172.21.15.8:0/3787781308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc14107e80 0x7fdc1419d0c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 >> 172.21.15.8:0/3787781308 conn(0x7fdc14101590 msgr2=0x7fdc14193ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 shutdown_connections 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:26.835+0000 7fdc18d6e700 1 -- 172.21.15.8:0/3787781308 wait complete. 2024-02-19T16:12:27.385 INFO:teuthology.orchestra.run.smithi008.stdout:Generating new minimal ceph.conf... 2024-02-19T16:12:27.472 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:27 smithi008 ceph-mon[32203]: from='client.? 172.21.15.8:0/3787781308' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-19T16:12:27.472 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:27 smithi008 ceph-mon[32203]: from='client.? 172.21.15.8:0/3787781308' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-19T16:12:28.443 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f950274d700 1 Processor -- start 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f950274d700 1 -- start start 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f950274d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f950274d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1083a0 con 0x7f94fc1079b0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f94fbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f94fbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc107dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60464/0 (socket says 172.21.15.8:60464) 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.769+0000 7f94fbfff700 1 -- 172.21.15.8:0/1806309785 learned_addr learned my addr 172.21.15.8:0/1806309785 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.770+0000 7f94fbfff700 1 -- 172.21.15.8:0/1806309785 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc108bb0 con 0x7f94fc1079b0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.770+0000 7f94fbfff700 1 --2- 172.21.15.8:0/1806309785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc107dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0098a0 tx=0x7f94ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f82ab83598b36ab server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.770+0000 7f94faffd700 1 -- 172.21.15.8:0/1806309785 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94ec004160 con 0x7f94fc1079b0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.770+0000 7f94faffd700 1 -- 172.21.15.8:0/1806309785 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94ec0042c0 con 0x7f94fc1079b0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.770+0000 7f94faffd700 1 -- 172.21.15.8:0/1806309785 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94ec011ac0 con 0x7f94fc1079b0 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 -- 172.21.15.8:0/1806309785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 msgr2=0x7f94fc107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:28.444 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 --2- 172.21.15.8:0/1806309785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc107dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0098a0 tx=0x7f94ec009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 -- 172.21.15.8:0/1806309785 shutdown_connections 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 --2- 172.21.15.8:0/1806309785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc107dd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 -- 172.21.15.8:0/1806309785 >> 172.21.15.8:0/1806309785 conn(0x7f94fc102f50 msgr2=0x7f94fc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 -- 172.21.15.8:0/1806309785 shutdown_connections 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.771+0000 7f950274d700 1 -- 172.21.15.8:0/1806309785 wait complete. 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f950274d700 1 Processor -- start 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f950274d700 1 -- start start 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f950274d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f950274d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19bc60 con 0x7f94fc1079b0 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f94fbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f94fbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60476/0 (socket says 172.21.15.8:60476) 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f94fbfff700 1 -- 172.21.15.8:0/1122947072 learned_addr learned my addr 172.21.15.8:0/1122947072 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:28.445 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f94fbfff700 1 -- 172.21.15.8:0/1122947072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec009550 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.772+0000 7f94fbfff700 1 --2- 172.21.15.8:0/1122947072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc19b720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000c00 tx=0x7f94ec011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94ec003aa0 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94ec003c00 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94ec0248a0 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc19be60 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc19c300 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f94ec02d3f0 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.773+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f94ec019770 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.774+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc063610 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.776+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f94ec01ac60 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.846+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f94fc19ee70 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.846+0000 7f94f97fa700 1 -- 172.21.15.8:0/1122947072 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f94ec035090 con 0x7f94fc1079b0 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 msgr2=0x7f94fc19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:28.446 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 --2- 172.21.15.8:0/1122947072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc19b720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000c00 tx=0x7f94ec011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:28.447 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 shutdown_connections 2024-02-19T16:12:28.447 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 --2- 172.21.15.8:0/1122947072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94fc1079b0 0x7f94fc19b720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:28.447 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 >> 172.21.15.8:0/1122947072 conn(0x7f94fc102f50 msgr2=0x7f94fc103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:28.447 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 shutdown_connections 2024-02-19T16:12:28.447 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:27.847+0000 7f950274d700 1 -- 172.21.15.8:0/1122947072 wait complete. 2024-02-19T16:12:28.447 INFO:teuthology.orchestra.run.smithi008.stdout:Restarting the monitor... 2024-02-19T16:12:28.463 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:28 smithi008 ceph-mon[32203]: from='client.? 172.21.15.8:0/1122947072' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:12:28.722 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:28 smithi008 systemd[1]: Stopping Ceph mon.a for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:12:29.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:28 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a[32199]: 2024-02-19T16:12:28.720+0000 7f68dbe4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:12:29.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:28 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a[32199]: 2024-02-19T16:12:28.720+0000 7f68dbe4c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-19T16:12:29.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:28 smithi008 podman[32447]: 2024-02-19 16:12:28.803136708 +0000 UTC m=+0.196526047 container died 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:12:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 podman[32447]: 2024-02-19 16:12:30.372930745 +0000 UTC m=+1.766320084 container cleanup 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-19T16:12:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 bash[32447]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a 2024-02-19T16:12:30.991 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 podman[32461]: 2024-02-19 16:12:30.648985995 +0000 UTC m=+1.843837225 container remove 8ee4aeacbd1a451079a89225c25568ad1087d0a9db2b51123cbe48440ce00fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-19T16:12:30.992 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-19T16:12:30.992 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 systemd[1]: Stopped Ceph mon.a for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:12:30.992 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 systemd[1]: Starting Ceph mon.a for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:12:31.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:30 smithi008 podman[32554]: 2024-02-19 16:12:30.991294867 +0000 UTC m=+0.095882176 container create 92f87135603b00c0d0773d90476f5eee1b9b9f9b299f458507af3a5fc2b10655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-19T16:12:31.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 podman[32554]: 2024-02-19 16:12:30.913463001 +0000 UTC m=+0.018050320 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:12:31.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 podman[32554]: 2024-02-19 16:12:31.193521322 +0000 UTC m=+0.298108634 container init 92f87135603b00c0d0773d90476f5eee1b9b9f9b299f458507af3a5fc2b10655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, ceph=True) 2024-02-19T16:12:31.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 podman[32554]: 2024-02-19 16:12:31.196769797 +0000 UTC m=+0.301357107 container start 92f87135603b00c0d0773d90476f5eee1b9b9f9b299f458507af3a5fc2b10655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-19T16:12:31.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: set uid:gid to 167:167 (ceph:ceph) 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev), process ceph-mon, pid 2 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: pidfile_write: ignore empty --pid-file 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: load: jerasure load: lrc 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: RocksDB version: 7.9.2 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Git sha 0 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Compile date 2024-02-15 13:42:22 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: DB SUMMARY 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: DB Session ID: OIYCTENPMVNSPK6IJZL3 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: CURRENT file: CURRENT 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: IDENTITY file: IDENTITY 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-19T16:12:31.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-19T16:12:31.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100278 ; 2024-02-19T16:12:31.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.error_if_exists: 0 2024-02-19T16:12:31.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.create_if_missing: 0 2024-02-19T16:12:31.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.paranoid_checks: 1 2024-02-19T16:12:31.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-19T16:12:31.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-19T16:12:31.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.env: 0x55d28fd77ae0 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.fs: PosixFileSystem 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.info_log: 0x55d2925ab900 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_file_opening_threads: 16 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.statistics: (nil) 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.use_fsync: 0 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_log_file_size: 0 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.keep_log_file_num: 1000 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.recycle_log_file_num: 0 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.allow_fallocate: 1 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.allow_mmap_reads: 0 2024-02-19T16:12:31.273 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.allow_mmap_writes: 0 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.use_direct_reads: 0 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.create_missing_column_families: 0 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.db_log_dir: 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.wal_dir: 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.advise_random_on_open: 1 2024-02-19T16:12:31.274 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.db_write_buffer_size: 0 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.write_buffer_manager: 0x55d29257f400 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.rate_limiter: (nil) 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.wal_recovery_mode: 2 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.enable_thread_tracking: 0 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.enable_pipelined_write: 0 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.unordered_write: 0 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-19T16:12:31.275 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.row_cache: None 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.wal_filter: None 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.allow_ingest_behind: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.two_write_queues: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.manual_wal_flush: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.wal_compression: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.atomic_flush: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.log_readahead_size: 0 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-19T16:12:31.276 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.best_efforts_recovery: 0 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.allow_data_in_errors: 0 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.db_host_id: __hostname__ 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_background_jobs: 2 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_background_compactions: -1 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_subcompactions: 1 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_total_wal_size: 0 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-19T16:12:31.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_open_files: -1 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bytes_per_sync: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_readahead_size: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_background_flushes: -1 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Compression algorithms supported: 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kZSTD supported: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kXpressCompression supported: 0 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kLZ4HCCompression supported: 1 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kZlibCompression supported: 1 2024-02-19T16:12:31.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kSnappyCompression supported: 1 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kLZ4Compression supported: 1 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: kBZip2Compression supported: 0 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.merge_operator: 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_filter: None 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_filter_factory: None 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.sst_partitioner_factory: None 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-19T16:12:31.279 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d2925aba00) 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: cache_index_and_filter_blocks: 1 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: pin_top_level_index_and_filter: 1 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: index_type: 0 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: data_block_index_type: 0 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: index_shortening: 1 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: checksum: 4 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: no_block_cache: 0 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache: 0x55d2925af090 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache_name: BinnedLRUCache 2024-02-19T16:12:31.280 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache_options: 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: capacity : 536870912 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: num_shard_bits : 4 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: strict_capacity_limit : 0 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: high_pri_pool_ratio: 0.000 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache_compressed: (nil) 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: persistent_cache: (nil) 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_size: 4096 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_size_deviation: 10 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_restart_interval: 16 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: index_block_restart_interval: 1 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: metadata_block_size: 4096 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: partition_filters: 0 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: use_delta_encoding: 1 2024-02-19T16:12:31.281 INFO:journalctl@ceph.mon.a.smithi008.stdout: filter_policy: bloomfilter 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: whole_key_filtering: 1 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: verify_compression: 0 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: read_amp_bytes_per_bit: 0 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: format_version: 5 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: enable_index_compression: 1 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_align: 0 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: max_auto_readahead_size: 262144 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: prepopulate_block_cache: 0 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: initial_auto_readahead_size: 8192 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout: num_file_reads_for_auto_readahead: 2 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.write_buffer_size: 33554432 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_write_buffer_number: 2 2024-02-19T16:12:31.282 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression: NoCompression 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression: Disabled 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.prefix_extractor: nullptr 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.num_levels: 7 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-19T16:12:31.283 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.level: 32767 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.strategy: 0 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.enabled: false 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-19T16:12:31.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.target_file_size_base: 67108864 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-19T16:12:31.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.arena_block_size: 1048576 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.disable_auto_compactions: 0 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-19T16:12:31.286 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.table_properties_collectors: 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.inplace_update_support: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.bloom_locality: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.max_successive_merges: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.paranoid_file_checks: 0 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.force_consistency_checks: 1 2024-02-19T16:12:31.287 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.report_bg_io_stats: 0 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.ttl: 2592000 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.enable_blob_files: false 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.min_blob_size: 0 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.blob_file_size: 268435456 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-19T16:12:31.288 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.blob_file_starting_level: 0 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: df022917-f9a3-43d8-a858-b3f76210d520 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708359151224972, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708359151226017, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95407, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93393, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13912, "raw_average_key_size": 50, "raw_value_size": 85837, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708359151, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "df022917-f9a3-43d8-a858-b3f76210d520", "db_session_id": "OIYCTENPMVNSPK6IJZL3", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708359151226077, "job": 1, "event": "recovery_finished"} 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d2926a0000 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: DB pointer 0x55d29268a000 2024-02-19T16:12:31.289 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** DB Stats ** 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** Compaction Stats [default] ** 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: L0 2/0 95.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:12:31.290 INFO:journalctl@ceph.mon.a.smithi008.stdout: Sum 2/0 95.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** Compaction Stats [default] ** 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-19T16:12:31.291 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative compaction: 0.00 GB write, 30.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval compaction: 0.00 GB write, 30.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: Block cache BinnedLRUCache@0x55d2925af090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: starting mon.a rank 0 at public addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] at bind addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???) e1 preinit fsid 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).mds e1 new map 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).mds e1 print_map 2024-02-19T16:12:31.292 INFO:journalctl@ceph.mon.a.smithi008.stdout: e1 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout: legacy client fscid: -1 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout: No filesystems configured 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T16:12:31.293 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-19T16:12:31.294 INFO:teuthology.orchestra.run.smithi008.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-19T16:12:31.546 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 bash[32554]: 92f87135603b00c0d0773d90476f5eee1b9b9f9b299f458507af3a5fc2b10655 2024-02-19T16:12:31.547 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 systemd[1]: Started Ceph mon.a for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:12:31.547 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-19T16:12:31.547 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: monmap e1: 1 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:12:31.547 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: fsmap 2024-02-19T16:12:31.547 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: osdmap e1: 0 total, 0 up, 0 in 2024-02-19T16:12:31.547 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:31 smithi008 ceph-mon[32573]: mgrmap e1: no daemons active 2024-02-19T16:12:32.207 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd07e62b700 1 Processor -- start 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd07e62b700 1 -- start start 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd07e62b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078107180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd07e62b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078107750 con 0x7fd078106d60 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd077fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd077fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60480/0 (socket says 172.21.15.8:60480) 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd077fff700 1 -- 172.21.15.8:0/1086886023 learned_addr learned my addr 172.21.15.8:0/1086886023 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.650+0000 7fd077fff700 1 -- 172.21.15.8:0/1086886023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078107a80 con 0x7fd078106d60 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd077fff700 1 --2- 172.21.15.8:0/1086886023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078107180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098a0 tx=0x7fd068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5916c1e8e5cfaa55 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd076ffd700 1 -- 172.21.15.8:0/1086886023 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd068004160 con 0x7fd078106d60 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd076ffd700 1 -- 172.21.15.8:0/1086886023 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0680042c0 con 0x7fd078106d60 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd076ffd700 1 -- 172.21.15.8:0/1086886023 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd068011ac0 con 0x7fd078106d60 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd07e62b700 1 -- 172.21.15.8:0/1086886023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 msgr2=0x7fd078107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd07e62b700 1 --2- 172.21.15.8:0/1086886023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078107180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098a0 tx=0x7fd068009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd07e62b700 1 -- 172.21.15.8:0/1086886023 shutdown_connections 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd07e62b700 1 --2- 172.21.15.8:0/1086886023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078107180 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.651+0000 7fd07e62b700 1 -- 172.21.15.8:0/1086886023 >> 172.21.15.8:0/1086886023 conn(0x7fd0781015a0 msgr2=0x7fd0781039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.652+0000 7fd07e62b700 1 -- 172.21.15.8:0/1086886023 shutdown_connections 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.652+0000 7fd07e62b700 1 -- 172.21.15.8:0/1086886023 wait complete. 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.652+0000 7fd07e62b700 1 Processor -- start 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.652+0000 7fd07e62b700 1 -- start start 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.652+0000 7fd07e62b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078106340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.652+0000 7fd07e62b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078106880 con 0x7fd078106d60 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd077fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078106340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd077fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078106340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60496/0 (socket says 172.21.15.8:60496) 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd077fff700 1 -- 172.21.15.8:0/2810961700 learned_addr learned my addr 172.21.15.8:0/2810961700 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd077fff700 1 -- 172.21.15.8:0/2810961700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068009550 con 0x7fd078106d60 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd077fff700 1 --2- 172.21.15.8:0/2810961700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078106340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd068003f70 tx=0x7fd068011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:32.209 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0680039f0 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd068003b50 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07819e420 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd06801a760 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07819e8c0 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.653+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd06801ad30 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.654+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd0680199b0 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.654+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07810acb0 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.656+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd0680249f0 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.726+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd07819eb00 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd0757fa700 1 -- 172.21.15.8:0/2810961700 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd068019b70 con 0x7fd078106d60 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 msgr2=0x7fd078106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 --2- 172.21.15.8:0/2810961700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078106340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd068003f70 tx=0x7fd068011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 shutdown_connections 2024-02-19T16:12:32.210 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 --2- 172.21.15.8:0/2810961700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd078106d60 0x7fd078106340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:32.211 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 >> 172.21.15.8:0/2810961700 conn(0x7fd0781015a0 msgr2=0x7fd0781020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:32.211 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 shutdown_connections 2024-02-19T16:12:32.211 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:31.728+0000 7fd07e62b700 1 -- 172.21.15.8:0/2810961700 wait complete. 2024-02-19T16:12:32.211 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-19T16:12:32.233 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:12:32.233 INFO:teuthology.orchestra.run.smithi008.stdout:Creating mgr... 2024-02-19T16:12:32.233 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-19T16:12:32.234 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-19T16:12:32.477 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x 2024-02-19T16:12:32.477 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to reset failed state of unit ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x.service: Unit ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x.service not loaded. 2024-02-19T16:12:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-81b9a938-cf41-11ee-95bb-87774f69a715.target.wants/ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-81b9a938-cf41-11ee-95bb-87774f69a715@.service. 2024-02-19T16:12:32.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:32 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2810961700' entity='client.admin' 2024-02-19T16:12:33.155 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:33 smithi008 podman[32788]: 2024-02-19 16:12:33.08214025 +0000 UTC m=+0.337591561 container start 8c5a5d54627cf92bf0b4dc70df1b9991c38bd6292ddfffbe3516ea09876bb0b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:12:33.168 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T16:12:33.168 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-02-19T16:12:33.174 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T16:12:33.174 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-02-19T16:12:33.174 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-02-19T16:12:33.174 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to enable service . firewalld.service is not available 2024-02-19T16:12:33.182 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T16:12:33.182 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-02-19T16:12:33.189 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T16:12:33.190 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-02-19T16:12:33.190 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-02-19T16:12:33.190 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-19T16:12:33.190 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr to start... 2024-02-19T16:12:33.190 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr... 2024-02-19T16:12:33.414 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:33 smithi008 bash[32788]: 8c5a5d54627cf92bf0b4dc70df1b9991c38bd6292ddfffbe3516ea09876bb0b9 2024-02-19T16:12:33.414 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:33 smithi008 systemd[1]: Started Ceph mgr.x for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:12:33.673 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:33 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:33.658+0000 7faf431db200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:12:34.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:33 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2829083532' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T16:12:34.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:33 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:33.764+0000 7faf431db200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:12:34.307 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:34.307 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:12:34.307 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:12:34.307 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T16:12:34.307 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:34.308 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T16:12:34.309 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T16:12:34.310 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T16:12:25.223714+0000", 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:34.311 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.591+0000 7ff1e308a700 1 Processor -- start 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.591+0000 7ff1e308a700 1 -- start start 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.591+0000 7ff1e308a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc1093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.591+0000 7ff1e308a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc109970 con 0x7ff1dc108f80 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1e0e26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc1093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1e0e26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc1093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60518/0 (socket says 172.21.15.8:60518) 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1e0e26700 1 -- 172.21.15.8:0/192899103 learned_addr learned my addr 172.21.15.8:0/192899103 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1e0e26700 1 -- 172.21.15.8:0/192899103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc109ab0 con 0x7ff1dc108f80 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1e0e26700 1 --2- 172.21.15.8:0/192899103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc1093a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00098a0 tx=0x7ff1d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aaf6eedfca2390 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1db7fe700 1 -- 172.21.15.8:0/192899103 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d0004160 con 0x7ff1dc108f80 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1db7fe700 1 -- 172.21.15.8:0/192899103 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1d00042c0 con 0x7ff1dc108f80 2024-02-19T16:12:34.312 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.592+0000 7ff1db7fe700 1 -- 172.21.15.8:0/192899103 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d0011ac0 con 0x7ff1dc108f80 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 -- 172.21.15.8:0/192899103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 msgr2=0x7ff1dc1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 --2- 172.21.15.8:0/192899103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc1093a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00098a0 tx=0x7ff1d0009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 -- 172.21.15.8:0/192899103 shutdown_connections 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 --2- 172.21.15.8:0/192899103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc1093a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 -- 172.21.15.8:0/192899103 >> 172.21.15.8:0/192899103 conn(0x7ff1dc1015c0 msgr2=0x7ff1dc1039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 -- 172.21.15.8:0/192899103 shutdown_connections 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 -- 172.21.15.8:0/192899103 wait complete. 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.593+0000 7ff1e308a700 1 Processor -- start 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e308a700 1 -- start start 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e308a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e308a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc19bcf0 con 0x7ff1dc108f80 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e0e26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e0e26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60530/0 (socket says 172.21.15.8:60530) 2024-02-19T16:12:34.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e0e26700 1 -- 172.21.15.8:0/2829083532 learned_addr learned my addr 172.21.15.8:0/2829083532 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e0e26700 1 -- 172.21.15.8:0/2829083532 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009550 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e0e26700 1 --2- 172.21.15.8:0/2829083532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc19b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0011780 tx=0x7ff1d0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d0003af0 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1d0003c50 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d00197e0 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.594+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc19bef0 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.595+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc19c390 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.595+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff1d0025460 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.595+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff1d000bb20 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.595+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc063610 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.598+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff1d002a060 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.672+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff1dc106680 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.672+0000 7ff1d9ffb700 1 -- 172.21.15.8:0/2829083532 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff1d0019d60 con 0x7ff1dc108f80 2024-02-19T16:12:34.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 msgr2=0x7ff1dc19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 --2- 172.21.15.8:0/2829083532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc19b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0011780 tx=0x7ff1d0011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 shutdown_connections 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 --2- 172.21.15.8:0/2829083532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1dc108f80 0x7ff1dc19b7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 >> 172.21.15.8:0/2829083532 conn(0x7ff1dc1015c0 msgr2=0x7ff1dc102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 shutdown_connections 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:33.674+0000 7ff1e308a700 1 -- 172.21.15.8:0/2829083532 wait complete. 2024-02-19T16:12:34.315 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (1/15)... 2024-02-19T16:12:34.667 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:34.428+0000 7faf431db200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:12:34.667 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:34.523+0000 7faf431db200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:12:35.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:34.666+0000 7faf431db200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:12:35.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:34.764+0000 7faf431db200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:12:35.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:34.848+0000 7faf431db200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:12:35.292 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:35.118+0000 7faf431db200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:12:35.292 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:35.232+0000 7faf431db200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:12:35.606 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:35.323+0000 7faf431db200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:12:35.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:35.423+0000 7faf431db200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:12:35.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:35.605+0000 7faf431db200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:12:36.883 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:36.619+0000 7faf431db200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:12:36.883 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:36.760+0000 7faf431db200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:12:37.200 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:36 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/593046286' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T16:12:37.200 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:37.106+0000 7faf431db200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:12:37.200 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:37.199+0000 7faf431db200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T16:12:37.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-19T16:12:37.382 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T16:12:37.383 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-19T16:12:37.384 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T16:12:25.223714+0000", 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:12:37.385 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.748+0000 7f423a5c1700 1 Processor -- start 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.748+0000 7f423a5c1700 1 -- start start 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.748+0000 7f423a5c1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423410a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.748+0000 7f423a5c1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423407bec0 con 0x7f42341083b0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4233fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423410a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4233fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423410a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37454/0 (socket says 172.21.15.8:37454) 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4233fff700 1 -- 172.21.15.8:0/2149682558 learned_addr learned my addr 172.21.15.8:0/2149682558 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4233fff700 1 -- 172.21.15.8:0/2149682558 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423410ace0 con 0x7f42341083b0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4233fff700 1 --2- 172.21.15.8:0/2149682558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423410a7a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42240098a0 tx=0x7f4224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91887e62ea239271 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4232ffd700 1 -- 172.21.15.8:0/2149682558 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4224004160 con 0x7f42341083b0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4232ffd700 1 -- 172.21.15.8:0/2149682558 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42240042c0 con 0x7f42341083b0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.749+0000 7f4232ffd700 1 -- 172.21.15.8:0/2149682558 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4224011ac0 con 0x7f42341083b0 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 -- 172.21.15.8:0/2149682558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 msgr2=0x7f423410a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 --2- 172.21.15.8:0/2149682558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423410a7a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42240098a0 tx=0x7f4224009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 -- 172.21.15.8:0/2149682558 shutdown_connections 2024-02-19T16:12:37.386 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 --2- 172.21.15.8:0/2149682558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423410a7a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 -- 172.21.15.8:0/2149682558 >> 172.21.15.8:0/2149682558 conn(0x7f4234101e40 msgr2=0x7f4234104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 -- 172.21.15.8:0/2149682558 shutdown_connections 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.750+0000 7f423a5c1700 1 -- 172.21.15.8:0/2149682558 wait complete. 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f423a5c1700 1 Processor -- start 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f423a5c1700 1 -- start start 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f423a5c1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f423a5c1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423419bc40 con 0x7f42341083b0 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f4233fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f4233fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37460/0 (socket says 172.21.15.8:37460) 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f4233fff700 1 -- 172.21.15.8:0/593046286 learned_addr learned my addr 172.21.15.8:0/593046286 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f4233fff700 1 -- 172.21.15.8:0/593046286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224009550 con 0x7f42341083b0 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.751+0000 7f4233fff700 1 --2- 172.21.15.8:0/593046286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423419b700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4224009fb0 tx=0x7f4224011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42240038b0 con 0x7f42341083b0 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4224003a10 con 0x7f42341083b0 2024-02-19T16:12:37.387 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423419be40 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42240195e0 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423419c2e0 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4224019c50 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.752+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f42240249e0 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.753+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423410bb40 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.757+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4224003b80 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.831+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f423419ee50 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.832+0000 7f42317fa700 1 -- 172.21.15.8:0/593046286 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f422401ab00 con 0x7f42341083b0 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.832+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 msgr2=0x7f423419b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.832+0000 7f423a5c1700 1 --2- 172.21.15.8:0/593046286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423419b700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4224009fb0 tx=0x7f4224011b20 comp rx=0 tx=0).stop 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.833+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 shutdown_connections 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.833+0000 7f423a5c1700 1 --2- 172.21.15.8:0/593046286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42341083b0 0x7f423419b700 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:37.388 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.833+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 >> 172.21.15.8:0/593046286 conn(0x7f4234101e40 msgr2=0x7f4234102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:37.389 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.833+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 shutdown_connections 2024-02-19T16:12:37.389 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:36.833+0000 7f423a5c1700 1 -- 172.21.15.8:0/593046286 wait complete. 2024-02-19T16:12:37.389 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (2/15)... 2024-02-19T16:12:37.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:37.403+0000 7faf431db200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:12:37.880 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:37.592+0000 7faf431db200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:12:38.141 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:37.880+0000 7faf431db200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:12:38.141 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:38.140+0000 7faf431db200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:12:38.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:38.380+0000 7faf431db200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:12:39.503 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:39.203+0000 7faf431db200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:12:40.077 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:39.790+0000 7faf431db200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:12:40.077 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:39.889+0000 7faf431db200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:12:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3312890406' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T16:12:40.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:40.126+0000 7faf431db200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:12:40.663 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:40.663 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:12:40.663 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T16:12:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T16:12:40.665 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T16:12:40.666 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T16:12:40.667 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T16:12:25.223714+0000", 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:12:40.668 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f8fe6700 1 Processor -- start 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f8fe6700 1 -- start start 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f8fe6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f4106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f8fe6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4107250 con 0x7f77f4106860 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f4106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f4106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37468/0 (socket says 172.21.15.8:37468) 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.994+0000 7f77f259c700 1 -- 172.21.15.8:0/2753390397 learned_addr learned my addr 172.21.15.8:0/2753390397 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.995+0000 7f77f259c700 1 -- 172.21.15.8:0/2753390397 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4107a60 con 0x7f77f4106860 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.995+0000 7f77f259c700 1 --2- 172.21.15.8:0/2753390397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f4106c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77e40098a0 tx=0x7f77e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d685d77f7ff9f5e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.995+0000 7f77f159a700 1 -- 172.21.15.8:0/2753390397 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77e4004160 con 0x7f77f4106860 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.995+0000 7f77f159a700 1 -- 172.21.15.8:0/2753390397 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e40042c0 con 0x7f77f4106860 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.995+0000 7f77f159a700 1 -- 172.21.15.8:0/2753390397 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77e4011ac0 con 0x7f77f4106860 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 -- 172.21.15.8:0/2753390397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 msgr2=0x7f77f4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:40.669 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 --2- 172.21.15.8:0/2753390397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f4106c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77e40098a0 tx=0x7f77e4009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 -- 172.21.15.8:0/2753390397 shutdown_connections 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 --2- 172.21.15.8:0/2753390397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f4106c80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 -- 172.21.15.8:0/2753390397 >> 172.21.15.8:0/2753390397 conn(0x7f77f4101e40 msgr2=0x7f77f4104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 -- 172.21.15.8:0/2753390397 shutdown_connections 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.996+0000 7f77f8fe6700 1 -- 172.21.15.8:0/2753390397 wait complete. 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f8fe6700 1 Processor -- start 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f8fe6700 1 -- start start 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f8fe6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f8fe6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f419bcc0 con 0x7f77f4106860 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37484/0 (socket says 172.21.15.8:37484) 2024-02-19T16:12:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f259c700 1 -- 172.21.15.8:0/3312890406 learned_addr learned my addr 172.21.15.8:0/3312890406 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f259c700 1 -- 172.21.15.8:0/3312890406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4009550 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.997+0000 7f77f259c700 1 --2- 172.21.15.8:0/3312890406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f419b780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77e4003f70 tx=0x7f77e4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77e40039f0 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e4003b50 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f419bec0 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77e401a690 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f419c360 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f77e401ad20 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.998+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f77e400bb80 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:39.999+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f4195900 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.002+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f77e40249f0 con 0x7f77f4106860 2024-02-19T16:12:40.671 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.076+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f77f410aec0 con 0x7f77f4106860 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.076+0000 7f77eb7fe700 1 -- 172.21.15.8:0/3312890406 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f77e400bd40 con 0x7f77f4106860 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 msgr2=0x7f77f419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 --2- 172.21.15.8:0/3312890406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f419b780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77e4003f70 tx=0x7f77e4011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 shutdown_connections 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 --2- 172.21.15.8:0/3312890406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77f4106860 0x7f77f419b780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 >> 172.21.15.8:0/3312890406 conn(0x7f77f4101e40 msgr2=0x7f77f4102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 shutdown_connections 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:40.077+0000 7f77f8fe6700 1 -- 172.21.15.8:0/3312890406 wait complete. 2024-02-19T16:12:40.672 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (3/15)... 2024-02-19T16:12:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: Activating manager daemon x 2024-02-19T16:12:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: mgrmap e2: x(active, starting, since 0.00303275s) 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: Manager daemon x is now available 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' 2024-02-19T16:12:41.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:41 smithi008 ceph-mon[32573]: from='mgr.14100 172.21.15.8:0/2344217305' entity='mgr.x' 2024-02-19T16:12:42.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:42 smithi008 ceph-mon[32573]: mgrmap e3: x(active, since 1.00689s) 2024-02-19T16:12:43.961 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.962 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T16:12:43.963 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.964 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T16:12:43.964 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:43.964 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T16:12:43.964 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T16:12:43.964 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T16:12:43.978 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T16:12:43.979 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T16:12:43.979 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T16:12:43.979 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.979 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T16:12:43.979 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T16:12:43.979 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T16:12:43.980 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T16:12:43.980 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T16:12:43.980 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T16:12:43.981 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T16:12:43.982 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.982 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T16:12:43.982 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-19T16:12:43.982 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T16:12:43.982 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T16:12:43.983 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T16:12:43.983 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T16:12:43.983 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T16:12:43.983 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-02-19T16:12:43.984 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:43.984 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.984 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T16:12:43.984 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T16:12:43.985 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T16:12:25.223714+0000", 2024-02-19T16:12:43.985 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T16:12:43.985 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-02-19T16:12:43.985 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T16:12:43.985 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:12:43.986 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f12597ea700 1 Processor -- start 2024-02-19T16:12:43.986 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f12597ea700 1 -- start start 2024-02-19T16:12:43.986 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f12597ea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:43.986 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f12597ea700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125407c8c0 con 0x7f125407de90 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f1252ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f1252ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37542/0 (socket says 172.21.15.8:37542) 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.106+0000 7f1252ffd700 1 -- 172.21.15.8:0/3343771959 learned_addr learned my addr 172.21.15.8:0/3343771959 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f1252ffd700 1 -- 172.21.15.8:0/3343771959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125407ca00 con 0x7f125407de90 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f1252ffd700 1 --2- 172.21.15.8:0/3343771959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125407c2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f12440098a0 tx=0x7f1244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab059b3c4e114ff5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f1251ffb700 1 -- 172.21.15.8:0/3343771959 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1244004160 con 0x7f125407de90 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f1251ffb700 1 -- 172.21.15.8:0/3343771959 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12440042c0 con 0x7f125407de90 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f1251ffb700 1 -- 172.21.15.8:0/3343771959 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1244011ac0 con 0x7f125407de90 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f12597ea700 1 -- 172.21.15.8:0/3343771959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 msgr2=0x7f125407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.107+0000 7f12597ea700 1 --2- 172.21.15.8:0/3343771959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125407c2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f12440098a0 tx=0x7f1244009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 -- 172.21.15.8:0/3343771959 shutdown_connections 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 --2- 172.21.15.8:0/3343771959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125407c2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:43.987 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 -- 172.21.15.8:0/3343771959 >> 172.21.15.8:0/3343771959 conn(0x7f1254102fa0 msgr2=0x7f12541053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 -- 172.21.15.8:0/3343771959 shutdown_connections 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 -- 172.21.15.8:0/3343771959 wait complete. 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 Processor -- start 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.108+0000 7f12597ea700 1 -- start start 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f12597ea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f12597ea700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125419de70 con 0x7f125407de90 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f1252ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f1252ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37552/0 (socket says 172.21.15.8:37552) 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f1252ffd700 1 -- 172.21.15.8:0/4113938270 learned_addr learned my addr 172.21.15.8:0/4113938270 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f1252ffd700 1 -- 172.21.15.8:0/4113938270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009550 con 0x7f125407de90 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f1252ffd700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125419d930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fb0 tx=0x7f1244011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12440038b0 con 0x7f125407de90 2024-02-19T16:12:43.988 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1244003a10 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12440195e0 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125419e070 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.109+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125419e510 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.110+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 3) v1 ==== 45815+0+0 (secure 0 0 0) 0x7f1244019850 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.110+0000 7f123bfff700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f123c038d80 0x7f123c03b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.110+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125410ddb0 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.110+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f124404cfe0 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.110+0000 7f12527fc700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f123c038d80 0x7f123c03b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.111+0000 7f12527fc700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f123c038d80 0x7f123c03b240 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1248006fd0 tx=0x7f1248006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.116+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1244018c20 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.143+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f1244019ce0 con 0x7f125407de90 2024-02-19T16:12:43.989 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.402+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f12541a1180 con 0x7f125407de90 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.402+0000 7f123bfff700 1 -- 172.21.15.8:0/4113938270 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f1244018a40 con 0x7f125407de90 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f123c038d80 msgr2=0x7f123c03b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f123c038d80 0x7f123c03b240 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1248006fd0 tx=0x7f1248006e40 comp rx=0 tx=0).stop 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 msgr2=0x7f125419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125419d930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fb0 tx=0x7f1244011b20 comp rx=0 tx=0).stop 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 shutdown_connections 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f123c038d80 0x7f123c03b240 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 --2- 172.21.15.8:0/4113938270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f125407de90 0x7f125419d930 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 >> 172.21.15.8:0/4113938270 conn(0x7f1254102fa0 msgr2=0x7f1254103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 shutdown_connections 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:43.405+0000 7f12597ea700 1 -- 172.21.15.8:0/4113938270 wait complete. 2024-02-19T16:12:43.990 INFO:teuthology.orchestra.run.smithi008.stdout:mgr is available 2024-02-19T16:12:44.375 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:44 smithi008 ceph-mon[32573]: mgrmap e4: x(active, since 3s) 2024-02-19T16:12:44.376 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4113938270' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T16:12:45.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:45.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [global] 2024-02-19T16:12:45.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout fsid = 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:45.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-19T16:12:45.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-19T16:12:45.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-19T16:12:45.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-19T16:12:45.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:45.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-19T16:12:45.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-19T16:12:45.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-02-19T16:12:45.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [osd] 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f79816a0700 1 Processor -- start 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f79816a0700 1 -- start start 2024-02-19T16:12:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f79816a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:45.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f79816a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c107250 con 0x7f797c106860 2024-02-19T16:12:45.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f797affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:45.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f797affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37558/0 (socket says 172.21.15.8:37558) 2024-02-19T16:12:45.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f797affd700 1 -- 172.21.15.8:0/849650774 learned_addr learned my addr 172.21.15.8:0/849650774 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:45.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f797affd700 1 -- 172.21.15.8:0/849650774 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c107a60 con 0x7f797c106860 2024-02-19T16:12:45.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.397+0000 7f797affd700 1 --2- 172.21.15.8:0/849650774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c106c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f796c00bb40 tx=0x7f796c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f1af7ea2afd18 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:45.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f7979ffb700 1 -- 172.21.15.8:0/849650774 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f796c00dbe0 con 0x7f797c106860 2024-02-19T16:12:45.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f7979ffb700 1 -- 172.21.15.8:0/849650774 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f796c0044f0 con 0x7f797c106860 2024-02-19T16:12:45.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f7979ffb700 1 -- 172.21.15.8:0/849650774 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f796c010470 con 0x7f797c106860 2024-02-19T16:12:45.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 -- 172.21.15.8:0/849650774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 msgr2=0x7f797c106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:45.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 --2- 172.21.15.8:0/849650774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c106c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f796c00bb40 tx=0x7f796c00be50 comp rx=0 tx=0).stop 2024-02-19T16:12:45.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 -- 172.21.15.8:0/849650774 shutdown_connections 2024-02-19T16:12:45.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 --2- 172.21.15.8:0/849650774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c106c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:45.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 -- 172.21.15.8:0/849650774 >> 172.21.15.8:0/849650774 conn(0x7f797c101e20 msgr2=0x7f797c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:45.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 -- 172.21.15.8:0/849650774 shutdown_connections 2024-02-19T16:12:45.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.398+0000 7f79816a0700 1 -- 172.21.15.8:0/849650774 wait complete. 2024-02-19T16:12:45.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f79816a0700 1 Processor -- start 2024-02-19T16:12:45.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f79816a0700 1 -- start start 2024-02-19T16:12:45.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f79816a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:45.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f79816a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c199aa0 con 0x7f797c106860 2024-02-19T16:12:45.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f797affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:45.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f797affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37572/0 (socket says 172.21.15.8:37572) 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.399+0000 7f797affd700 1 -- 172.21.15.8:0/2808688341 learned_addr learned my addr 172.21.15.8:0/2808688341 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f797affd700 1 -- 172.21.15.8:0/2808688341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c00b7f0 con 0x7f797c106860 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f797affd700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c199560 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f796c00d650 tx=0x7f796c0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f796c01e070 con 0x7f797c106860 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f796c010bb0 con 0x7f797c106860 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c199ca0 con 0x7f797c106860 2024-02-19T16:12:45.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f796c022b30 con 0x7f797c106860 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.400+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c19a140 con 0x7f797c106860 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.401+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f796c02c440 con 0x7f797c106860 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.401+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c1930f0 con 0x7f797c106860 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.401+0000 7f7973fff700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f7968038e90 0x7f796803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.401+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f796c04f5c0 con 0x7f797c106860 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.401+0000 7f797a7fc700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f7968038e90 0x7f796803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.402+0000 7f797a7fc700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f7968038e90 0x7f796803b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7964006fd0 tx=0x7f7964006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:45.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.406+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f796c0193f0 con 0x7f797c106860 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.607+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f797c07afe0 con 0x7f797c106860 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.608+0000 7f7973fff700 1 -- 172.21.15.8:0/2808688341 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f796c027020 con 0x7f797c106860 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f7968038e90 msgr2=0x7f796803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f7968038e90 0x7f796803b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7964006fd0 tx=0x7f7964006e40 comp rx=0 tx=0).stop 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 msgr2=0x7f797c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c199560 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f796c00d650 tx=0x7f796c0040b0 comp rx=0 tx=0).stop 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 shutdown_connections 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f7968038e90 0x7f796803b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 --2- 172.21.15.8:0/2808688341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f797c106860 0x7f797c199560 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 >> 172.21.15.8:0/2808688341 conn(0x7f797c101e20 msgr2=0x7f797c190230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 shutdown_connections 2024-02-19T16:12:45.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:44.610+0000 7f79816a0700 1 -- 172.21.15.8:0/2808688341 wait complete. 2024-02-19T16:12:45.170 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling cephadm module... 2024-02-19T16:12:45.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:45 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2808688341' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-19T16:12:46.510 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:46 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/506003289' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-19T16:12:46.511 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: ignoring --setuser ceph since I am not root 2024-02-19T16:12:46.511 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: ignoring --setgroup ceph since I am not root 2024-02-19T16:12:46.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:46.757+0000 7fbe9cff9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.559+0000 7f55b3e1f700 1 Processor -- start 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.559+0000 7f55b3e1f700 1 -- start start 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.559+0000 7f55b3e1f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac108280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.559+0000 7f55b3e1f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac108850 con 0x7f55ac107e60 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.559+0000 7f55b1bbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b1bbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac108280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46418/0 (socket says 172.21.15.8:46418) 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b1bbb700 1 -- 172.21.15.8:0/2066064101 learned_addr learned my addr 172.21.15.8:0/2066064101 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b1bbb700 1 -- 172.21.15.8:0/2066064101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac106a70 con 0x7f55ac107e60 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b1bbb700 1 --2- 172.21.15.8:0/2066064101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac108280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55a40098a0 tx=0x7f55a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c9f3b77c17ad8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b0bb9700 1 -- 172.21.15.8:0/2066064101 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a4004160 con 0x7f55ac107e60 2024-02-19T16:12:46.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b0bb9700 1 -- 172.21.15.8:0/2066064101 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55a40042c0 con 0x7f55ac107e60 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.560+0000 7f55b0bb9700 1 -- 172.21.15.8:0/2066064101 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a4011ac0 con 0x7f55ac107e60 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 -- 172.21.15.8:0/2066064101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 msgr2=0x7f55ac108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 --2- 172.21.15.8:0/2066064101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac108280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55a40098a0 tx=0x7f55a4009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 -- 172.21.15.8:0/2066064101 shutdown_connections 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 --2- 172.21.15.8:0/2066064101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac108280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 -- 172.21.15.8:0/2066064101 >> 172.21.15.8:0/2066064101 conn(0x7f55ac1015e0 msgr2=0x7f55ac103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 -- 172.21.15.8:0/2066064101 shutdown_connections 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 -- 172.21.15.8:0/2066064101 wait complete. 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 Processor -- start 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.561+0000 7f55b3e1f700 1 -- start start 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b3e1f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b3e1f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac19bd40 con 0x7f55ac107e60 2024-02-19T16:12:46.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b1bbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b1bbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46430/0 (socket says 172.21.15.8:46430) 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b1bbb700 1 -- 172.21.15.8:0/506003289 learned_addr learned my addr 172.21.15.8:0/506003289 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b1bbb700 1 -- 172.21.15.8:0/506003289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009550 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b1bbb700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac19b800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55a4011780 tx=0x7f55a4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a4003af0 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55a4003c50 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac19bf40 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a40197e0 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.562+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac19c3e0 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.563+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f55a4019a10 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.563+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ac10acb0 con 0x7f55ac107e60 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.563+0000 7f55aaffd700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f5594038ee0 0x7f559403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:46.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.563+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f55a404d3b0 con 0x7f55ac107e60 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.563+0000 7f55b13ba700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f5594038ee0 0x7f559403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.564+0000 7f55b13ba700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f5594038ee0 0x7f559403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55a0006fd0 tx=0x7f55a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.568+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f55a4019cc0 con 0x7f55ac107e60 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:45.847+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f55ac10aeb0 con 0x7f55ac107e60 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.146+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f55a4018350 con 0x7f55ac107e60 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.148+0000 7f55aaffd700 1 -- 172.21.15.8:0/506003289 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f55a400b900 con 0x7f55ac107e60 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f5594038ee0 msgr2=0x7f559403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f5594038ee0 0x7f559403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55a0006fd0 tx=0x7f55a0006e40 comp rx=0 tx=0).stop 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 msgr2=0x7f55ac19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac19b800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55a4011780 tx=0x7f55a4011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 shutdown_connections 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f5594038ee0 0x7f559403b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 --2- 172.21.15.8:0/506003289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55ac107e60 0x7f55ac19b800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 >> 172.21.15.8:0/506003289 conn(0x7f55ac1015e0 msgr2=0x7f55ac1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:46.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 shutdown_connections 2024-02-19T16:12:46.814 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:46.150+0000 7f55b3e1f700 1 -- 172.21.15.8:0/506003289 wait complete. 2024-02-19T16:12:47.148 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:46.858+0000 7fbe9cff9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:12:47.453 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/506003289' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-19T16:12:47.454 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-mon[32573]: mgrmap e5: x(active, since 6s) 2024-02-19T16:12:47.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:47.527+0000 7fbe9cff9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:12:47.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:47.626+0000 7fbe9cff9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:12:48.085 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:47.771+0000 7fbe9cff9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:12:48.085 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:47.870+0000 7fbe9cff9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:12:48.085 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:47.955+0000 7fbe9cff9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:12:48.093 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.246+0000 7f58477a0700 1 Processor -- start 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f58477a0700 1 -- start start 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f58477a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584010a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f58477a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584007be60 con 0x7f5840108410 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f584553c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584010a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f584553c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584010a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46446/0 (socket says 172.21.15.8:46446) 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f584553c700 1 -- 172.21.15.8:0/3961069765 learned_addr learned my addr 172.21.15.8:0/3961069765 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:48.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f584553c700 1 -- 172.21.15.8:0/3961069765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584010ad40 con 0x7f5840108410 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f584553c700 1 --2- 172.21.15.8:0/3961069765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584010a800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58340098a0 tx=0x7f5834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045fb4f03d6715e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.247+0000 7f5833fff700 1 -- 172.21.15.8:0/3961069765 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5834004160 con 0x7f5840108410 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f5833fff700 1 -- 172.21.15.8:0/3961069765 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58340042c0 con 0x7f5840108410 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f5833fff700 1 -- 172.21.15.8:0/3961069765 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5834011ac0 con 0x7f5840108410 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 -- 172.21.15.8:0/3961069765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 msgr2=0x7f584010a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 --2- 172.21.15.8:0/3961069765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584010a800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58340098a0 tx=0x7f5834009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 -- 172.21.15.8:0/3961069765 shutdown_connections 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 --2- 172.21.15.8:0/3961069765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584010a800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 -- 172.21.15.8:0/3961069765 >> 172.21.15.8:0/3961069765 conn(0x7f5840101e40 msgr2=0x7f5840104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 -- 172.21.15.8:0/3961069765 shutdown_connections 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.248+0000 7f58477a0700 1 -- 172.21.15.8:0/3961069765 wait complete. 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f58477a0700 1 Processor -- start 2024-02-19T16:12:48.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f58477a0700 1 -- start start 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f58477a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f58477a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019bc40 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f584553c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f584553c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584019b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46452/0 (socket says 172.21.15.8:46452) 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f584553c700 1 -- 172.21.15.8:0/4289401584 learned_addr learned my addr 172.21.15.8:0/4289401584 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.249+0000 7f584553c700 1 -- 172.21.15.8:0/4289401584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834009550 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.250+0000 7f584553c700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584019b700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5834009fb0 tx=0x7f5834011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.250+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58340038b0 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.250+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5834003a10 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.250+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584019be40 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.250+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58340195e0 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.250+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584019c2e0 con 0x7f5840108410 2024-02-19T16:12:48.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.251+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584010bb40 con 0x7f5840108410 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.251+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f5834019870 con 0x7f5840108410 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.251+0000 7f58327fc700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 0x7f582c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.251+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f583404cff0 con 0x7f5840108410 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.251+0000 7f5844d3b700 1 -- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 msgr2=0x7f582c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.251+0000 7f5844d3b700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 0x7f582c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.256+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5834081230 con 0x7f5840108410 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.452+0000 7f5844d3b700 1 -- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 msgr2=0x7f582c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.452+0000 7f5844d3b700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 0x7f582c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.518+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f584019ebc0 con 0x7f5840108410 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.518+0000 7f58327fc700 1 -- 172.21.15.8:0/4289401584 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5834023d80 con 0x7f5840108410 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 msgr2=0x7f582c03b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 0x7f582c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 msgr2=0x7f584019b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:48.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584019b700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5834009fb0 tx=0x7f5834011b20 comp rx=0 tx=0).stop 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 shutdown_connections 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7f582c038ee0 0x7f582c03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 --2- 172.21.15.8:0/4289401584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5840108410 0x7f584019b700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 >> 172.21.15.8:0/4289401584 conn(0x7f5840101e40 msgr2=0x7f5840102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 shutdown_connections 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:47.520+0000 7f58477a0700 1 -- 172.21.15.8:0/4289401584 wait complete. 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for the mgr to restart... 2024-02-19T16:12:48.098 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr epoch 5... 2024-02-19T16:12:48.345 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:48 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4289401584' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-19T16:12:48.345 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:48.228+0000 7fbe9cff9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:12:48.345 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:48.343+0000 7fbe9cff9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:12:48.713 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:48.431+0000 7fbe9cff9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:12:48.714 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:48.532+0000 7fbe9cff9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:12:48.714 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:48.712+0000 7fbe9cff9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:12:50.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:49.706+0000 7fbe9cff9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:12:50.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:49.840+0000 7fbe9cff9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:12:50.474 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:50.179+0000 7fbe9cff9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:12:50.474 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:50.271+0000 7fbe9cff9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:12:50.474 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:50.472+0000 7fbe9cff9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:12:50.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:50.662+0000 7fbe9cff9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:12:51.216 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:50.955+0000 7fbe9cff9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:12:51.216 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:51.215+0000 7fbe9cff9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:12:51.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:51.458+0000 7fbe9cff9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:12:52.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:52.283+0000 7fbe9cff9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:12:53.184 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:52.863+0000 7fbe9cff9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:12:53.185 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:52.963+0000 7fbe9cff9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:12:53.185 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:12:53.183+0000 7fbe9cff9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:12:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: Active manager daemon x restarted 2024-02-19T16:12:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: Activating manager daemon x 2024-02-19T16:12:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: osdmap e2: 0 total, 0 up, 0 in 2024-02-19T16:12:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: mgrmap e6: x(active, starting, since 0.00330702s) 2024-02-19T16:12:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:12:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: Manager daemon x is now available 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:12:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:53 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:12:54.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: Found migration_current of "None". Setting to last migration. 2024-02-19T16:12:54.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:12:54.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-19T16:12:54.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-19T16:12:54.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:12:54.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:12:54.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: mgrmap e7: x(active, since 1.0051s) 2024-02-19T16:12:54.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:54 smithi008 ceph-mon[32573]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-19T16:12:54.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:12:54.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff643100700 1 Processor -- start 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff643100700 1 -- start start 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff643100700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff643100700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c1072b0 con 0x7ff63c1068c0 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff640e9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff640e9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46460/0 (socket says 172.21.15.8:46460) 2024-02-19T16:12:54.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.529+0000 7ff640e9c700 1 -- 172.21.15.8:0/2349673029 learned_addr learned my addr 172.21.15.8:0/2349673029 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.530+0000 7ff640e9c700 1 -- 172.21.15.8:0/2349673029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c107ac0 con 0x7ff63c1068c0 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.530+0000 7ff640e9c700 1 --2- 172.21.15.8:0/2349673029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c106ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6300098a0 tx=0x7ff630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5892d6cebe95408 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.530+0000 7ff63b7fe700 1 -- 172.21.15.8:0/2349673029 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff630004160 con 0x7ff63c1068c0 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.530+0000 7ff63b7fe700 1 -- 172.21.15.8:0/2349673029 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6300042c0 con 0x7ff63c1068c0 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.530+0000 7ff63b7fe700 1 -- 172.21.15.8:0/2349673029 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff630011ac0 con 0x7ff63c1068c0 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- 172.21.15.8:0/2349673029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 msgr2=0x7ff63c106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 --2- 172.21.15.8:0/2349673029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c106ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6300098a0 tx=0x7ff630009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- 172.21.15.8:0/2349673029 shutdown_connections 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 --2- 172.21.15.8:0/2349673029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c106ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- 172.21.15.8:0/2349673029 >> 172.21.15.8:0/2349673029 conn(0x7ff63c101e80 msgr2=0x7ff63c1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- 172.21.15.8:0/2349673029 shutdown_connections 2024-02-19T16:12:54.854 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- 172.21.15.8:0/2349673029 wait complete. 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 Processor -- start 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- start start 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c191910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.531+0000 7ff643100700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c191e50 con 0x7ff63c1068c0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff640e9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c191910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff640e9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c191910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46468/0 (socket says 172.21.15.8:46468) 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff640e9c700 1 -- 172.21.15.8:0/1053781433 learned_addr learned my addr 172.21.15.8:0/1053781433 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff640e9c700 1 -- 172.21.15.8:0/1053781433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630009550 con 0x7ff63c1068c0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff640e9c700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c191910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff630003f70 tx=0x7ff630011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6300039f0 con 0x7ff63c1068c0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff630003b50 con 0x7ff63c1068c0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c190090 con 0x7ff63c1068c0 2024-02-19T16:12:54.855 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff63001a690 con 0x7ff63c1068c0 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.532+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c190530 con 0x7ff63c1068c0 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.533+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ff630024430 con 0x7ff63c1068c0 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.533+0000 7ff639ffb700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.533+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 --> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff62c03bab0 con 0x7ff62c038ee0 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.533+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff63004d0c0 con 0x7ff63c1068c0 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.533+0000 7ff63bfff700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 msgr2=0x7ff62c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.534+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.735+0000 7ff63bfff700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 msgr2=0x7ff62c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:48.735+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:49.136+0000 7ff63bfff700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 msgr2=0x7ff62c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:54.856 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:49.136+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:49.937+0000 7ff63bfff700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 msgr2=0x7ff62c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:49.937+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:51.539+0000 7ff63bfff700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 msgr2=0x7ff62c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3125167978 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:51.539+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:53.199+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7ff63002d910 con 0x7ff63c1068c0 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:53.199+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 msgr2=0x7ff62c03b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:53.199+0000 7ff639ffb700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.200+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7ff63004e680 con 0x7ff63c1068c0 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.200+0000 7ff639ffb700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ff62c03c7d0 0x7ff62c03ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.200+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff62c03bab0 con 0x7ff62c03c7d0 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.201+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ff62c03c7d0 0x7ff62c03ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:54.857 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.202+0000 7ff63bfff700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ff62c03c7d0 0x7ff62c03ebc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff628003a10 tx=0x7ff6280092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.202+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff62c03bab0 con 0x7ff62c03c7d0 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff63c108920 con 0x7ff62c03c7d0 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff639ffb700 1 -- 172.21.15.8:0/1053781433 <== mgr.14120 v2:172.21.15.8:6800/886492473 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff62800a050 con 0x7ff62c03c7d0 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ff62c03c7d0 msgr2=0x7ff62c03ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ff62c03c7d0 0x7ff62c03ebc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff628003a10 tx=0x7ff6280092b0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 msgr2=0x7ff63c191910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c191910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff630003f70 tx=0x7ff630011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 shutdown_connections 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ff62c03c7d0 0x7ff62c03ebc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff63c1068c0 0x7ff63c191910 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.858 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 --2- 172.21.15.8:0/1053781433 >> [v2:172.21.15.8:6800/3125167978,v1:172.21.15.8:6801/3125167978] conn(0x7ff62c038ee0 0x7ff62c03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:54.859 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 >> 172.21.15.8:0/1053781433 conn(0x7ff63c101e80 msgr2=0x7ff63c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:54.859 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 shutdown_connections 2024-02-19T16:12:54.859 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:54.209+0000 7ff643100700 1 -- 172.21.15.8:0/1053781433 wait complete. 2024-02-19T16:12:54.859 INFO:teuthology.orchestra.run.smithi008.stdout:mgr epoch 5 is available 2024-02-19T16:12:54.859 INFO:teuthology.orchestra.run.smithi008.stdout:Setting orchestrator backend to cephadm... 2024-02-19T16:12:55.497 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:55 smithi008 ceph-mon[32573]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-19T16:12:55.498 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:55 smithi008 ceph-mon[32573]: [19/Feb/2024:16:12:54] ENGINE Bus STARTING 2024-02-19T16:12:55.498 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:55 smithi008 ceph-mon[32573]: [19/Feb/2024:16:12:54] ENGINE Serving on http://172.21.15.8:8765 2024-02-19T16:12:55.498 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:55 smithi008 ceph-mon[32573]: [19/Feb/2024:16:12:54] ENGINE Serving on https://172.21.15.8:7150 2024-02-19T16:12:55.498 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:55 smithi008 ceph-mon[32573]: [19/Feb/2024:16:12:54] ENGINE Bus STARTED 2024-02-19T16:12:55.498 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:55 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:12:56.082 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16cd4b0700 1 Processor -- start 2024-02-19T16:12:56.082 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16cd4b0700 1 -- start start 2024-02-19T16:12:56.082 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16cd4b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c8106090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:56.082 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16cd4b0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c8106660 con 0x7f16c8105c70 2024-02-19T16:12:56.082 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16c6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c8106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:56.082 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16c6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c8106090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46566/0 (socket says 172.21.15.8:46566) 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16c6ffd700 1 -- 172.21.15.8:0/3911678692 learned_addr learned my addr 172.21.15.8:0/3911678692 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16c6ffd700 1 -- 172.21.15.8:0/3911678692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c8108c00 con 0x7f16c8105c70 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.268+0000 7f16c6ffd700 1 --2- 172.21.15.8:0/3911678692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c8106090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098a0 tx=0x7f16b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a755ea8955ee92e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16c5ffb700 1 -- 172.21.15.8:0/3911678692 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b8004160 con 0x7f16c8105c70 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16c5ffb700 1 -- 172.21.15.8:0/3911678692 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b80042c0 con 0x7f16c8105c70 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16c5ffb700 1 -- 172.21.15.8:0/3911678692 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b8011ac0 con 0x7f16c8105c70 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 -- 172.21.15.8:0/3911678692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 msgr2=0x7f16c8106090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:56.083 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 --2- 172.21.15.8:0/3911678692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c8106090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098a0 tx=0x7f16b8009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 -- 172.21.15.8:0/3911678692 shutdown_connections 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 --2- 172.21.15.8:0/3911678692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c8106090 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 -- 172.21.15.8:0/3911678692 >> 172.21.15.8:0/3911678692 conn(0x7f16c81015e0 msgr2=0x7f16c8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 -- 172.21.15.8:0/3911678692 shutdown_connections 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.269+0000 7f16cd4b0700 1 -- 172.21.15.8:0/3911678692 wait complete. 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16cd4b0700 1 Processor -- start 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16cd4b0700 1 -- start start 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16cd4b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16cd4b0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819bd00 con 0x7f16c8105c70 2024-02-19T16:12:56.084 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16c6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16c6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c819b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46580/0 (socket says 172.21.15.8:46580) 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16c6ffd700 1 -- 172.21.15.8:0/2416542268 learned_addr learned my addr 172.21.15.8:0/2416542268 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.270+0000 7f16c6ffd700 1 -- 172.21.15.8:0/2416542268 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009550 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.271+0000 7f16c6ffd700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c819b7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16b8011780 tx=0x7f16b8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.271+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b8003af0 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.271+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b8003c50 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.271+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b80197e0 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.271+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c819bf00 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.271+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c819c3a0 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.272+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f16b801a430 con 0x7f16c8105c70 2024-02-19T16:12:56.085 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.272+0000 7f16affff700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f16b0038f30 0x7f16b003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.272+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c810be30 con 0x7f16c8105c70 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.272+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16b804d240 con 0x7f16c8105c70 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.272+0000 7f16c67fc700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f16b0038f30 0x7f16b003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.272+0000 7f16c67fc700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f16b0038f30 0x7f16b003b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16bc006fd0 tx=0x7f16bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.277+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f16b802da40 con 0x7f16c8105c70 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.496+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f16c8102870 con 0x7f16b0038f30 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.501+0000 7f16affff700 1 -- 172.21.15.8:0/2416542268 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f16c8102870 con 0x7f16b0038f30 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f16b0038f30 msgr2=0x7f16b003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f16b0038f30 0x7f16b003b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16bc006fd0 tx=0x7f16bc006e40 comp rx=0 tx=0).stop 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 msgr2=0x7f16c819b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c819b7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16b8011780 tx=0x7f16b8011b40 comp rx=0 tx=0).stop 2024-02-19T16:12:56.086 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 shutdown_connections 2024-02-19T16:12:56.087 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f16b0038f30 0x7f16b003b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:56.087 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 --2- 172.21.15.8:0/2416542268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16c8105c70 0x7f16c819b7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:56.087 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.503+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 >> 172.21.15.8:0/2416542268 conn(0x7f16c81015e0 msgr2=0x7f16c8102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:56.087 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.504+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 shutdown_connections 2024-02-19T16:12:56.087 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:55.504+0000 7f16cd4b0700 1 -- 172.21.15.8:0/2416542268 wait complete. 2024-02-19T16:12:56.328 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:56 smithi008 ceph-mon[32573]: mgrmap e8: x(active, since 2s) 2024-02-19T16:12:56.328 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:56 smithi008 ceph-mon[32573]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:12:56.328 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:56 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:12:56.328 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:56 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:12:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:57 smithi008 ceph-mon[32573]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f9040685700 1 Processor -- start 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f9040685700 1 -- start start 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f9040685700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f9038106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f9040685700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90381072a0 con 0x7f90381068b0 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f903e421700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f9038106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f903e421700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f9038106cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51346/0 (socket says 172.21.15.8:51346) 2024-02-19T16:12:57.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.535+0000 7f903e421700 1 -- 172.21.15.8:0/299014946 learned_addr learned my addr 172.21.15.8:0/299014946 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.536+0000 7f903e421700 1 -- 172.21.15.8:0/299014946 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038107ab0 con 0x7f90381068b0 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.536+0000 7f903e421700 1 --2- 172.21.15.8:0/299014946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f9038106cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90340098a0 tx=0x7f9034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef12ec04ca428fd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.536+0000 7f903d41f700 1 -- 172.21.15.8:0/299014946 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9034004160 con 0x7f90381068b0 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.536+0000 7f903d41f700 1 -- 172.21.15.8:0/299014946 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90340042c0 con 0x7f90381068b0 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.536+0000 7f903d41f700 1 -- 172.21.15.8:0/299014946 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9034011ac0 con 0x7f90381068b0 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 -- 172.21.15.8:0/299014946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 msgr2=0x7f9038106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 --2- 172.21.15.8:0/299014946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f9038106cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90340098a0 tx=0x7f9034009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 -- 172.21.15.8:0/299014946 shutdown_connections 2024-02-19T16:12:57.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 --2- 172.21.15.8:0/299014946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f9038106cd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 -- 172.21.15.8:0/299014946 >> 172.21.15.8:0/299014946 conn(0x7f9038101e10 msgr2=0x7f9038104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 -- 172.21.15.8:0/299014946 shutdown_connections 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 -- 172.21.15.8:0/299014946 wait complete. 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 Processor -- start 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.537+0000 7f9040685700 1 -- start start 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f9040685700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f903819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f9040685700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903819bcd0 con 0x7f90381068b0 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f903e421700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f903819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f903e421700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f903819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51360/0 (socket says 172.21.15.8:51360) 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f903e421700 1 -- 172.21.15.8:0/2318914113 learned_addr learned my addr 172.21.15.8:0/2318914113 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f903e421700 1 -- 172.21.15.8:0/2318914113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034009550 con 0x7f90381068b0 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f903e421700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f903819b790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9034009fb0 tx=0x7f9034011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90340038b0 con 0x7f90381068b0 2024-02-19T16:12:57.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9034003a10 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903819bed0 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.538+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90340195e0 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.539+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903819c370 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.539+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f9034019850 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.539+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038195880 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.540+0000 7f902f7fe700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f9024038ee0 0x7f902403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.540+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f903404d270 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.540+0000 7f903dc20700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f9024038ee0 0x7f902403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.540+0000 7f903dc20700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f9024038ee0 0x7f902403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9028006fd0 tx=0x7f9028006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.545+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9034018750 con 0x7f90381068b0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.751+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f903802dc00 con 0x7f9024038ee0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.752+0000 7f902f7fe700 1 -- 172.21.15.8:0/2318914113 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f903802dc00 con 0x7f9024038ee0 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f9024038ee0 msgr2=0x7f902403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:57.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f9024038ee0 0x7f902403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9028006fd0 tx=0x7f9028006e40 comp rx=0 tx=0).stop 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 msgr2=0x7f903819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f903819b790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9034009fb0 tx=0x7f9034011b20 comp rx=0 tx=0).stop 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 shutdown_connections 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f9024038ee0 0x7f902403b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 --2- 172.21.15.8:0/2318914113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90381068b0 0x7f903819b790 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 >> 172.21.15.8:0/2318914113 conn(0x7f9038101e10 msgr2=0x7f9038102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 shutdown_connections 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:56.754+0000 7f9040685700 1 -- 172.21.15.8:0/2318914113 wait complete. 2024-02-19T16:12:57.319 INFO:teuthology.orchestra.run.smithi008.stdout:Generating ssh key... 2024-02-19T16:12:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-mon[32573]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:12:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-mon[32573]: Generating ssh key... 2024-02-19T16:12:58.520 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: Generating public/private rsa key pair. 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: Your identification has been saved in /tmp/tmpiaacqv0b/key. 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: Your public key has been saved in /tmp/tmpiaacqv0b/key.pub. 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: The key fingerprint is: 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: SHA256:lif06TtBLfaH20bYLBnWfVditDb8J19zf81P2q++VcY ceph-81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: The key's randomart image is: 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: +---[RSA 3072]----+ 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | .. | 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | .o..| 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | . . o=o.| 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | . o+.+..++| 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | So++ B..E| 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | . +. * ==O| 2024-02-19T16:12:58.521 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | .. * O| 2024-02-19T16:12:58.522 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | ... o=o| 2024-02-19T16:12:58.522 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: | .. o=o=| 2024-02-19T16:12:58.522 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:12:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: +----[SHA256]-----+ 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.737+0000 7f7460c8a700 1 Processor -- start 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.737+0000 7f7460c8a700 1 -- start start 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.737+0000 7f7460c8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c107d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.737+0000 7f7460c8a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c108340 con 0x7f745c107950 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.737+0000 7f745a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c107d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c107d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51370/0 (socket says 172.21.15.8:51370) 2024-02-19T16:12:58.845 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745a59c700 1 -- 172.21.15.8:0/3040738214 learned_addr learned my addr 172.21.15.8:0/3040738214 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745a59c700 1 -- 172.21.15.8:0/3040738214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c108b50 con 0x7f745c107950 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745a59c700 1 --2- 172.21.15.8:0/3040738214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c107d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74500098a0 tx=0x7f7450009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772b602ca4c2967e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745959a700 1 -- 172.21.15.8:0/3040738214 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7450004160 con 0x7f745c107950 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745959a700 1 -- 172.21.15.8:0/3040738214 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74500042c0 con 0x7f745c107950 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.738+0000 7f745959a700 1 -- 172.21.15.8:0/3040738214 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7450011ac0 con 0x7f745c107950 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 -- 172.21.15.8:0/3040738214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 msgr2=0x7f745c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 --2- 172.21.15.8:0/3040738214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c107d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74500098a0 tx=0x7f7450009bb0 comp rx=0 tx=0).stop 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 -- 172.21.15.8:0/3040738214 shutdown_connections 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 --2- 172.21.15.8:0/3040738214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c107d70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 -- 172.21.15.8:0/3040738214 >> 172.21.15.8:0/3040738214 conn(0x7f745c102f50 msgr2=0x7f745c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 -- 172.21.15.8:0/3040738214 shutdown_connections 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.739+0000 7f7460c8a700 1 -- 172.21.15.8:0/3040738214 wait complete. 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f7460c8a700 1 Processor -- start 2024-02-19T16:12:58.846 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f7460c8a700 1 -- start start 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f7460c8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f7460c8a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c19bcb0 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f745a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f745a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51384/0 (socket says 172.21.15.8:51384) 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f745a59c700 1 -- 172.21.15.8:0/3888093753 learned_addr learned my addr 172.21.15.8:0/3888093753 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f745a59c700 1 -- 172.21.15.8:0/3888093753 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450009550 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.740+0000 7f745a59c700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c19b770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7450009fb0 tx=0x7f7450011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.741+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74500038b0 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.741+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c19beb0 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.741+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c19c350 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.741+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7450003a10 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.741+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74500195e0 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.741+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7450019920 con 0x7f745c107950 2024-02-19T16:12:58.847 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.742+0000 7f744b7fe700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7444038e90 0x7f744403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.742+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c10bd80 con 0x7f745c107950 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.742+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f745004d2d0 con 0x7f745c107950 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.742+0000 7f7459d9b700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7444038e90 0x7f744403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.742+0000 7f7459d9b700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7444038e90 0x7f744403b350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f744c006fd0 tx=0x7f744c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.747+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f745002b340 con 0x7f745c107950 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:57.953+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f745c1041f0 con 0x7f7444038e90 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.335+0000 7f744b7fe700 1 -- 172.21.15.8:0/3888093753 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f745c1041f0 con 0x7f7444038e90 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7444038e90 msgr2=0x7f744403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7444038e90 0x7f744403b350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f744c006fd0 tx=0x7f744c006e40 comp rx=0 tx=0).stop 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 msgr2=0x7f745c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c19b770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7450009fb0 tx=0x7f7450011b20 comp rx=0 tx=0).stop 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 shutdown_connections 2024-02-19T16:12:58.848 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7444038e90 0x7f744403b350 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:58.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 --2- 172.21.15.8:0/3888093753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f745c107950 0x7f745c19b770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:12:58.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.337+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 >> 172.21.15.8:0/3888093753 conn(0x7f745c102f50 msgr2=0x7f745c103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:12:58.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.338+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 shutdown_connections 2024-02-19T16:12:58.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:58.338+0000 7f7460c8a700 1 -- 172.21.15.8:0/3888093753 wait complete. 2024-02-19T16:12:59.501 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:59 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:12:59.502 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:12:59 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:00.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:13:00.094 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.272+0000 7fcd8c51f700 1 Processor -- start 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.272+0000 7fcd8c51f700 1 -- start start 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8c51f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd84107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8c51f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84108390 con 0x7fcd841079a0 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8a2bb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd84107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8a2bb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd84107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51398/0 (socket says 172.21.15.8:51398) 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8a2bb700 1 -- 172.21.15.8:0/967659948 learned_addr learned my addr 172.21.15.8:0/967659948 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8a2bb700 1 -- 172.21.15.8:0/967659948 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84108b50 con 0x7fcd841079a0 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd8a2bb700 1 --2- 172.21.15.8:0/967659948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd84107dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd740098a0 tx=0x7fcd74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7f2a9fc3944e50 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:00.095 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd892b9700 1 -- 172.21.15.8:0/967659948 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd74004160 con 0x7fcd841079a0 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd892b9700 1 -- 172.21.15.8:0/967659948 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd740042c0 con 0x7fcd841079a0 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.273+0000 7fcd892b9700 1 -- 172.21.15.8:0/967659948 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd74011ac0 con 0x7fcd841079a0 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 -- 172.21.15.8:0/967659948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 msgr2=0x7fcd84107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 --2- 172.21.15.8:0/967659948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd84107dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd740098a0 tx=0x7fcd74009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 -- 172.21.15.8:0/967659948 shutdown_connections 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 --2- 172.21.15.8:0/967659948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd84107dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 -- 172.21.15.8:0/967659948 >> 172.21.15.8:0/967659948 conn(0x7fcd84102f60 msgr2=0x7fcd84105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 -- 172.21.15.8:0/967659948 shutdown_connections 2024-02-19T16:13:00.096 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.274+0000 7fcd8c51f700 1 -- 172.21.15.8:0/967659948 wait complete. 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8c51f700 1 Processor -- start 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8c51f700 1 -- start start 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8c51f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd8407db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8c51f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8407e0c0 con 0x7fcd841079a0 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8a2bb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd8407db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8a2bb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd8407db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51410/0 (socket says 172.21.15.8:51410) 2024-02-19T16:13:00.097 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8a2bb700 1 -- 172.21.15.8:0/489016127 learned_addr learned my addr 172.21.15.8:0/489016127 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8a2bb700 1 -- 172.21.15.8:0/489016127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74009550 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.275+0000 7fcd8a2bb700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd8407db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009fb0 tx=0x7fcd74011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.276+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd740038b0 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.276+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd74003a10 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.276+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8407c2b0 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.276+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd740195e0 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.276+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8407c750 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.277+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fcd74019850 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.277+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8410bd80 con 0x7fcd841079a0 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.277+0000 7fcd7b7fe700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fcd70038ee0 0x7fcd7003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:00.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.277+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcd7404d0c0 con 0x7fcd841079a0 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.277+0000 7fcd89aba700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fcd70038ee0 0x7fcd7003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.278+0000 7fcd89aba700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fcd70038ee0 0x7fcd7003b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006fd0 tx=0x7fcd80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.282+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fcd74003b80 con 0x7fcd841079a0 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.487+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd8407ce40 con 0x7fcd70038ee0 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.488+0000 7fcd7b7fe700 1 -- 172.21.15.8:0/489016127 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fcd8407ce40 con 0x7fcd70038ee0 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fcd70038ee0 msgr2=0x7fcd7003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fcd70038ee0 0x7fcd7003b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006fd0 tx=0x7fcd80006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 msgr2=0x7fcd8407db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd8407db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009fb0 tx=0x7fcd74011b20 comp rx=0 tx=0).stop 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 shutdown_connections 2024-02-19T16:13:00.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fcd70038ee0 0x7fcd7003b3a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 --2- 172.21.15.8:0/489016127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841079a0 0x7fcd8407db80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 >> 172.21.15.8:0/489016127 conn(0x7fcd84102f60 msgr2=0x7fcd84103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 shutdown_connections 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:12:59.490+0000 7fcd8c51f700 1 -- 172.21.15.8:0/489016127 wait complete. 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:Adding key to root@localhost authorized_keys... 2024-02-19T16:13:00.100 INFO:teuthology.orchestra.run.smithi008.stdout:Adding host smithi008... 2024-02-19T16:13:00.513 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:00 smithi008 ceph-mon[32573]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:01.619 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:01 smithi008 ceph-mon[32573]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi008", "addr": "172.21.15.8", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:02.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:02 smithi008 ceph-mon[32573]: Deploying cephadm binary to smithi008 2024-02-19T16:13:05.776 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Added host 'smithi008' with addr '172.21.15.8' 2024-02-19T16:13:05.776 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e92e1700 1 Processor -- start 2024-02-19T16:13:05.776 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e92e1700 1 -- start start 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e92e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e4106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e92e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4107250 con 0x7f57e4106860 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e4106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e4106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51414/0 (socket says 172.21.15.8:51414) 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.557+0000 7f57e2ffd700 1 -- 172.21.15.8:0/2099445176 learned_addr learned my addr 172.21.15.8:0/2099445176 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e2ffd700 1 -- 172.21.15.8:0/2099445176 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4107a60 con 0x7f57e4106860 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e2ffd700 1 --2- 172.21.15.8:0/2099445176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e4106c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57d400bb40 tx=0x7f57d400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba18dc5a83f4cdbd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e1ffb700 1 -- 172.21.15.8:0/2099445176 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57d400dbe0 con 0x7f57e4106860 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e1ffb700 1 -- 172.21.15.8:0/2099445176 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57d40044f0 con 0x7f57e4106860 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e1ffb700 1 -- 172.21.15.8:0/2099445176 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57d4010470 con 0x7f57e4106860 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e92e1700 1 -- 172.21.15.8:0/2099445176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 msgr2=0x7f57e4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.558+0000 7f57e92e1700 1 --2- 172.21.15.8:0/2099445176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e4106c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57d400bb40 tx=0x7f57d400be50 comp rx=0 tx=0).stop 2024-02-19T16:13:05.777 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 -- 172.21.15.8:0/2099445176 shutdown_connections 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 --2- 172.21.15.8:0/2099445176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e4106c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 -- 172.21.15.8:0/2099445176 >> 172.21.15.8:0/2099445176 conn(0x7f57e4101e20 msgr2=0x7f57e4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 -- 172.21.15.8:0/2099445176 shutdown_connections 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 -- 172.21.15.8:0/2099445176 wait complete. 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 Processor -- start 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.559+0000 7f57e92e1700 1 -- start start 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e92e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e92e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e419cf60 con 0x7f57e4106860 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51422/0 (socket says 172.21.15.8:51422) 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e2ffd700 1 -- 172.21.15.8:0/281486998 learned_addr learned my addr 172.21.15.8:0/281486998 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e2ffd700 1 -- 172.21.15.8:0/281486998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d400b7f0 con 0x7f57e4106860 2024-02-19T16:13:05.778 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e2ffd700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e419ca20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d650 tx=0x7f57d40040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57d40106f0 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e419d160 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57d4010d10 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57d4019d10 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.560+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e4199560 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.561+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f57d4019510 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.561+0000 7f57cbfff700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f57cc038e90 0x7f57cc03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.561+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e4063610 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.561+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f57d404d3d0 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.562+0000 7f57e27fc700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f57cc038e90 0x7f57cc03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.562+0000 7f57e27fc700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f57cc038e90 0x7f57cc03b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006fd0 tx=0x7f57d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.566+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f57d402a970 con 0x7f57e4106860 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:00.773+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi008", "addr": "172.21.15.8", "target": ["mon-mgr", ""]}) v1 -- 0x7f57e4192be0 con 0x7f57cc038e90 2024-02-19T16:13:05.779 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.145+0000 7f57cbfff700 1 -- 172.21.15.8:0/281486998 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f57e4192be0 con 0x7f57cc038e90 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f57cc038e90 msgr2=0x7f57cc03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f57cc038e90 0x7f57cc03b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57d8006fd0 tx=0x7f57d8006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 msgr2=0x7f57e419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e419ca20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d650 tx=0x7f57d40040b0 comp rx=0 tx=0).stop 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 shutdown_connections 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f57cc038e90 0x7f57cc03b350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 --2- 172.21.15.8:0/281486998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57e4106860 0x7f57e419ca20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 >> 172.21.15.8:0/281486998 conn(0x7f57e4101e20 msgr2=0x7f57e4190230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 shutdown_connections 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:05.148+0000 7f57e92e1700 1 -- 172.21.15.8:0/281486998 wait complete. 2024-02-19T16:13:05.780 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying unmanaged mon service... 2024-02-19T16:13:06.359 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:06 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:06.359 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:06 smithi008 ceph-mon[32573]: Added host smithi008 2024-02-19T16:13:06.359 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:06 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:07.154 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-19T16:13:07.154 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f19163f6700 1 Processor -- start 2024-02-19T16:13:07.154 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f19163f6700 1 -- start start 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f19163f6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f1910107d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f19163f6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1910108330 con 0x7f1910107940 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f190ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f1910107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f190ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f1910107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40580/0 (socket says 172.21.15.8:40580) 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f190ffff700 1 -- 172.21.15.8:0/337291303 learned_addr learned my addr 172.21.15.8:0/337291303 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.192+0000 7f190ffff700 1 -- 172.21.15.8:0/337291303 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910108b40 con 0x7f1910107940 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f190ffff700 1 --2- 172.21.15.8:0/337291303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f1910107d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19000098a0 tx=0x7f1900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1170794d98960c71 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f190effd700 1 -- 172.21.15.8:0/337291303 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1900004160 con 0x7f1910107940 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f190effd700 1 -- 172.21.15.8:0/337291303 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19000042c0 con 0x7f1910107940 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f190effd700 1 -- 172.21.15.8:0/337291303 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1900011ac0 con 0x7f1910107940 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 -- 172.21.15.8:0/337291303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 msgr2=0x7f1910107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 --2- 172.21.15.8:0/337291303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f1910107d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19000098a0 tx=0x7f1900009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:07.155 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 -- 172.21.15.8:0/337291303 shutdown_connections 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 --2- 172.21.15.8:0/337291303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f1910107d60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 -- 172.21.15.8:0/337291303 >> 172.21.15.8:0/337291303 conn(0x7f1910102f00 msgr2=0x7f1910105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 -- 172.21.15.8:0/337291303 shutdown_connections 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.193+0000 7f19163f6700 1 -- 172.21.15.8:0/337291303 wait complete. 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f19163f6700 1 Processor -- start 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f19163f6700 1 -- start start 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f19163f6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f191019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f19163f6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191019bc60 con 0x7f1910107940 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f190ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f191019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f190ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f191019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40596/0 (socket says 172.21.15.8:40596) 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.194+0000 7f190ffff700 1 -- 172.21.15.8:0/3233804325 learned_addr learned my addr 172.21.15.8:0/3233804325 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f190ffff700 1 -- 172.21.15.8:0/3233804325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900009550 con 0x7f1910107940 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f190ffff700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f191019b720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1900003f70 tx=0x7f1900011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:07.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19000039f0 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1900003b50 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191019be60 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f190001a690 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.195+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191019c300 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.196+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f190001a900 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.196+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191010bd70 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.196+0000 7f190d7fa700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f18f8038ee0 0x7f18f803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.196+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f190004d390 con 0x7f1910107940 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.196+0000 7f190f7fe700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f18f8038ee0 0x7f18f803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.197+0000 7f190f7fe700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f18f8038ee0 0x7f18f803b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1904006fd0 tx=0x7f1904006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:07.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.201+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f190001abb0 con 0x7f1910107940 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.423+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1910104100 con 0x7f18f8038ee0 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.428+0000 7f190d7fa700 1 -- 172.21.15.8:0/3233804325 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1910104100 con 0x7f18f8038ee0 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.430+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f18f8038ee0 msgr2=0x7f18f803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.430+0000 7f19163f6700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f18f8038ee0 0x7f18f803b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1904006fd0 tx=0x7f1904006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.430+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 msgr2=0x7f191019b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.430+0000 7f19163f6700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f191019b720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1900003f70 tx=0x7f1900011b40 comp rx=0 tx=0).stop 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.431+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 shutdown_connections 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.431+0000 7f19163f6700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f18f8038ee0 0x7f18f803b3a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.431+0000 7f19163f6700 1 --2- 172.21.15.8:0/3233804325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1910107940 0x7f191019b720 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.431+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 >> 172.21.15.8:0/3233804325 conn(0x7f1910102f00 msgr2=0x7f19101039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.431+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 shutdown_connections 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:06.431+0000 7f19163f6700 1 -- 172.21.15.8:0/3233804325 wait complete. 2024-02-19T16:13:07.158 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying unmanaged mgr service... 2024-02-19T16:13:07.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:07 smithi008 ceph-mon[32573]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:07.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:07 smithi008 ceph-mon[32573]: Saving service mon spec with placement count:5 2024-02-19T16:13:07.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:07 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:08 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:08 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:08 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:09.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-19T16:13:09.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.419+0000 7f73ded2e700 1 Processor -- start 2024-02-19T16:13:09.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.419+0000 7f73ded2e700 1 -- start start 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.419+0000 7f73ded2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.419+0000 7f73ded2e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d807c8c0 con 0x7f73d807de90 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73dcaca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73dcaca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40608/0 (socket says 172.21.15.8:40608) 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73dcaca700 1 -- 172.21.15.8:0/2384812467 learned_addr learned my addr 172.21.15.8:0/2384812467 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73dcaca700 1 -- 172.21.15.8:0/2384812467 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d807ca00 con 0x7f73d807de90 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73dcaca700 1 --2- 172.21.15.8:0/2384812467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d807c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009e80 tx=0x7f73d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a667c84736fafd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73d77fe700 1 -- 172.21.15.8:0/2384812467 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d000e070 con 0x7f73d807de90 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73d77fe700 1 -- 172.21.15.8:0/2384812467 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73d000b8c0 con 0x7f73d807de90 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.420+0000 7f73d77fe700 1 -- 172.21.15.8:0/2384812467 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d00044a0 con 0x7f73d807de90 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 -- 172.21.15.8:0/2384812467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 msgr2=0x7f73d807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 --2- 172.21.15.8:0/2384812467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d807c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009e80 tx=0x7f73d000b330 comp rx=0 tx=0).stop 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 -- 172.21.15.8:0/2384812467 shutdown_connections 2024-02-19T16:13:09.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 --2- 172.21.15.8:0/2384812467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d807c2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 -- 172.21.15.8:0/2384812467 >> 172.21.15.8:0/2384812467 conn(0x7f73d8102f50 msgr2=0x7f73d8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 -- 172.21.15.8:0/2384812467 shutdown_connections 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.421+0000 7f73ded2e700 1 -- 172.21.15.8:0/2384812467 wait complete. 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73ded2e700 1 Processor -- start 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73ded2e700 1 -- start start 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73ded2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73ded2e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d819ddd0 con 0x7f73d807de90 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73dcaca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73dcaca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d819d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40620/0 (socket says 172.21.15.8:40620) 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.422+0000 7f73dcaca700 1 -- 172.21.15.8:0/409328650 learned_addr learned my addr 172.21.15.8:0/409328650 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73dcaca700 1 -- 172.21.15.8:0/409328650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0009b30 con 0x7f73d807de90 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73dcaca700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d819d890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009530 tx=0x7f73d000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d000e040 con 0x7f73d807de90 2024-02-19T16:13:09.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d819dfd0 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73d0003d60 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d0012760 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.423+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d819e4d0 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.424+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f73d0019070 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.424+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d810de50 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.425+0000 7f73d5ffb700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f73c4038ee0 0x7f73c403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.425+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f73d004d350 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.425+0000 7f73d7fff700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f73c4038ee0 0x7f73c403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.425+0000 7f73d7fff700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f73c4038ee0 0x7f73c403b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006fd0 tx=0x7f73c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.430+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f73d001aa20 con 0x7f73d807de90 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.668+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f73d8104100 con 0x7f73c4038ee0 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.673+0000 7f73d5ffb700 1 -- 172.21.15.8:0/409328650 <== mgr.14120 v2:172.21.15.8:6800/886492473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f73d8104100 con 0x7f73c4038ee0 2024-02-19T16:13:09.344 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f73c4038ee0 msgr2=0x7f73c403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f73c4038ee0 0x7f73c403b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006fd0 tx=0x7f73c8006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 msgr2=0x7f73d819d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d819d890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009530 tx=0x7f73d000bd10 comp rx=0 tx=0).stop 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 shutdown_connections 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f73c4038ee0 0x7f73c403b3a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 --2- 172.21.15.8:0/409328650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73d807de90 0x7f73d819d890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.675+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 >> 172.21.15.8:0/409328650 conn(0x7f73d8102f50 msgr2=0x7f73d81039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.676+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 shutdown_connections 2024-02-19T16:13:09.345 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:08.676+0000 7f73ded2e700 1 -- 172.21.15.8:0/409328650 wait complete. 2024-02-19T16:13:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:09 smithi008 ceph-mon[32573]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:09 smithi008 ceph-mon[32573]: Saving service mgr spec with placement count:2 2024-02-19T16:13:10.750 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa9a39d700 1 Processor -- start 2024-02-19T16:13:10.750 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa9a39d700 1 -- start start 2024-02-19T16:13:10.750 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa9a39d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa94106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:10.750 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa9a39d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94107230 con 0x7ffa94106840 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa94106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa94106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40628/0 (socket says 172.21.15.8:40628) 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.775+0000 7ffa93fff700 1 -- 172.21.15.8:0/1100879278 learned_addr learned my addr 172.21.15.8:0/1100879278 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.776+0000 7ffa93fff700 1 -- 172.21.15.8:0/1100879278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94107a40 con 0x7ffa94106840 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.776+0000 7ffa93fff700 1 --2- 172.21.15.8:0/1100879278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa94106c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa840098a0 tx=0x7ffa84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a12a86a35bd67d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.776+0000 7ffa92ffd700 1 -- 172.21.15.8:0/1100879278 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa84004160 con 0x7ffa94106840 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.776+0000 7ffa92ffd700 1 -- 172.21.15.8:0/1100879278 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa840042c0 con 0x7ffa94106840 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.776+0000 7ffa92ffd700 1 -- 172.21.15.8:0/1100879278 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa84011a80 con 0x7ffa94106840 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 -- 172.21.15.8:0/1100879278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 msgr2=0x7ffa94106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 --2- 172.21.15.8:0/1100879278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa94106c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa840098a0 tx=0x7ffa84009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 -- 172.21.15.8:0/1100879278 shutdown_connections 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 --2- 172.21.15.8:0/1100879278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa94106c60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 -- 172.21.15.8:0/1100879278 >> 172.21.15.8:0/1100879278 conn(0x7ffa94101e40 msgr2=0x7ffa94104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:10.751 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 -- 172.21.15.8:0/1100879278 shutdown_connections 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 -- 172.21.15.8:0/1100879278 wait complete. 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 Processor -- start 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.777+0000 7ffa9a39d700 1 -- start start 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa9a39d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa9419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa9a39d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9419de20 con 0x7ffa94106840 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa9419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa9419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40640/0 (socket says 172.21.15.8:40640) 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa93fff700 1 -- 172.21.15.8:0/37352526 learned_addr learned my addr 172.21.15.8:0/37352526 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa93fff700 1 -- 172.21.15.8:0/37352526 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa84009550 con 0x7ffa94106840 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa93fff700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa9419d8e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa84003f20 tx=0x7ffa84011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa840039f0 con 0x7ffa94106840 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa84003b50 con 0x7ffa94106840 2024-02-19T16:13:10.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9419e020 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa8401a690 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.778+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9419e4c0 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.779+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ffa8401a980 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.779+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9407afe0 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.779+0000 7ffa917fa700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ffa7c038ee0 0x7ffa7c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.779+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffa8404d200 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.779+0000 7ffa937fe700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ffa7c038ee0 0x7ffa7c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.780+0000 7ffa937fe700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ffa7c038ee0 0x7ffa7c03b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa88006fd0 tx=0x7ffa88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.784+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ffa8401ac30 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:09.998+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ffa94192c40 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.000+0000 7ffa917fa700 1 -- 172.21.15.8:0/37352526 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ffa84018be0 con 0x7ffa94106840 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ffa7c038ee0 msgr2=0x7ffa7c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ffa7c038ee0 0x7ffa7c03b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa88006fd0 tx=0x7ffa88006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 msgr2=0x7ffa9419d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:10.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa9419d8e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa84003f20 tx=0x7ffa84011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:10.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 shutdown_connections 2024-02-19T16:13:10.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7ffa7c038ee0 0x7ffa7c03b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:10.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 --2- 172.21.15.8:0/37352526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa94106840 0x7ffa9419d8e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:10.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 >> 172.21.15.8:0/37352526 conn(0x7ffa94101e40 msgr2=0x7ffa9418fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:10.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 shutdown_connections 2024-02-19T16:13:10.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:10.002+0000 7ffa9a39d700 1 -- 172.21.15.8:0/37352526 wait complete. 2024-02-19T16:13:11.153 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:11 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/37352526' entity='client.admin' 2024-02-19T16:13:12.041 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.191+0000 7f7a7f971700 1 Processor -- start 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7f971700 1 -- start start 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7f971700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7810a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7f971700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7807be60 con 0x7f7a78108430 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7d70d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7810a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7d70d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7810a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40642/0 (socket says 172.21.15.8:40642) 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7d70d700 1 -- 172.21.15.8:0/133390882 learned_addr learned my addr 172.21.15.8:0/133390882 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.192+0000 7f7a7d70d700 1 -- 172.21.15.8:0/133390882 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7810ad60 con 0x7f7a78108430 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a7d70d700 1 --2- 172.21.15.8:0/133390882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7810a820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a740098a0 tx=0x7f7a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2c0a2b2df19ed0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a6ffff700 1 -- 172.21.15.8:0/133390882 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a74004160 con 0x7f7a78108430 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a6ffff700 1 -- 172.21.15.8:0/133390882 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a740042c0 con 0x7f7a78108430 2024-02-19T16:13:12.042 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a6ffff700 1 -- 172.21.15.8:0/133390882 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a74011a80 con 0x7f7a78108430 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a7f971700 1 -- 172.21.15.8:0/133390882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 msgr2=0x7f7a7810a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a7f971700 1 --2- 172.21.15.8:0/133390882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7810a820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a740098a0 tx=0x7f7a74009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a7f971700 1 -- 172.21.15.8:0/133390882 shutdown_connections 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a7f971700 1 --2- 172.21.15.8:0/133390882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7810a820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.193+0000 7f7a7f971700 1 -- 172.21.15.8:0/133390882 >> 172.21.15.8:0/133390882 conn(0x7f7a78101e20 msgr2=0x7f7a78104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.194+0000 7f7a7f971700 1 -- 172.21.15.8:0/133390882 shutdown_connections 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.194+0000 7f7a7f971700 1 -- 172.21.15.8:0/133390882 wait complete. 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.194+0000 7f7a7f971700 1 Processor -- start 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.194+0000 7f7a7f971700 1 -- start start 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.194+0000 7f7a7f971700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.194+0000 7f7a7f971700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7819bbe0 con 0x7f7a78108430 2024-02-19T16:13:12.043 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7d70d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7d70d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7819b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40644/0 (socket says 172.21.15.8:40644) 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7d70d700 1 -- 172.21.15.8:0/2663677142 learned_addr learned my addr 172.21.15.8:0/2663677142 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7d70d700 1 -- 172.21.15.8:0/2663677142 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a74009550 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7d70d700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7819b6a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a74011740 tx=0x7f7a74011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a74003b20 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a7401a910 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a740248c0 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7819bde0 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.195+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7819c280 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.196+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7a7401aa80 con 0x7f7a78108430 2024-02-19T16:13:12.044 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.196+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7810bba0 con 0x7f7a78108430 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.196+0000 7f7a6e7fc700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7a64038ee0 0x7f7a6403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.197+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a7404d3a0 con 0x7f7a78108430 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.197+0000 7f7a7cf0c700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7a64038ee0 0x7f7a6403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.197+0000 7f7a7cf0c700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7a64038ee0 0x7f7a6403b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a68006fd0 tx=0x7f7a68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.201+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7a7401adf0 con 0x7f7a78108430 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.405+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7a7819eb60 con 0x7f7a78108430 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.408+0000 7f7a6e7fc700 1 -- 172.21.15.8:0/2663677142 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7a7404cd10 con 0x7f7a78108430 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7a64038ee0 msgr2=0x7f7a6403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7a64038ee0 0x7f7a6403b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a68006fd0 tx=0x7f7a68006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 msgr2=0x7f7a7819b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7819b6a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a74011740 tx=0x7f7a74011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:12.045 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 shutdown_connections 2024-02-19T16:13:12.046 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f7a64038ee0 0x7f7a6403b3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:12.046 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 --2- 172.21.15.8:0/2663677142 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a78108430 0x7f7a7819b6a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:12.046 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 >> 172.21.15.8:0/2663677142 conn(0x7f7a78101e20 msgr2=0x7f7a78102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:12.046 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 shutdown_connections 2024-02-19T16:13:12.046 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:11.410+0000 7f7a7f971700 1 -- 172.21.15.8:0/2663677142 wait complete. 2024-02-19T16:13:12.046 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling the dashboard module... 2024-02-19T16:13:12.444 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:12 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2663677142' entity='client.admin' 2024-02-19T16:13:13.906 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:13 smithi008 ceph-mon[32573]: from='mgr.14120 172.21.15.8:0/4129556272' entity='mgr.x' 2024-02-19T16:13:13.907 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:13 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/203777548' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-19T16:13:13.907 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:13 smithi008 ceph-mon[32573]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:13.907 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:13 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: ignoring --setuser ceph since I am not root 2024-02-19T16:13:13.907 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:13 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: ignoring --setgroup ceph since I am not root 2024-02-19T16:13:14.207 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:14 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:14.155+0000 7f1bb8b9a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.462+0000 7f2c5adc9700 1 Processor -- start 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.462+0000 7f2c5adc9700 1 -- start start 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.462+0000 7f2c5adc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5407c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.462+0000 7f2c5adc9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5407c870 con 0x7f2c5407de30 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c58b65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c58b65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5407c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40658/0 (socket says 172.21.15.8:40658) 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c58b65700 1 -- 172.21.15.8:0/3575629234 learned_addr learned my addr 172.21.15.8:0/3575629234 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c58b65700 1 -- 172.21.15.8:0/3575629234 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5407c9b0 con 0x7f2c5407de30 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c58b65700 1 --2- 172.21.15.8:0/3575629234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5407c330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c480098a0 tx=0x7f2c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57669daf465bc37b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c537fe700 1 -- 172.21.15.8:0/3575629234 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c48004160 con 0x7f2c5407de30 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c537fe700 1 -- 172.21.15.8:0/3575629234 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c480042c0 con 0x7f2c5407de30 2024-02-19T16:13:14.215 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.463+0000 7f2c537fe700 1 -- 172.21.15.8:0/3575629234 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c48011a80 con 0x7f2c5407de30 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 -- 172.21.15.8:0/3575629234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 msgr2=0x7f2c5407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 --2- 172.21.15.8:0/3575629234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5407c330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c480098a0 tx=0x7f2c48009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 -- 172.21.15.8:0/3575629234 shutdown_connections 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 --2- 172.21.15.8:0/3575629234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5407c330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 -- 172.21.15.8:0/3575629234 >> 172.21.15.8:0/3575629234 conn(0x7f2c54102f60 msgr2=0x7f2c541053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 -- 172.21.15.8:0/3575629234 shutdown_connections 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.464+0000 7f2c5adc9700 1 -- 172.21.15.8:0/3575629234 wait complete. 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c5adc9700 1 Processor -- start 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c5adc9700 1 -- start start 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c5adc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c5adc9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5419de10 con 0x7f2c5407de30 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c58b65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:14.216 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c58b65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40666/0 (socket says 172.21.15.8:40666) 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c58b65700 1 -- 172.21.15.8:0/203777548 learned_addr learned my addr 172.21.15.8:0/203777548 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c58b65700 1 -- 172.21.15.8:0/203777548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48009550 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c58b65700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5419d8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c48003f20 tx=0x7f2c48011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c480039f0 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c48003b50 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5419e010 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.465+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c4801a6d0 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.466+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5419e4b0 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.466+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f2c4801a990 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.467+0000 7f2c51ffb700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2c44038e90 0x7f2c4403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.467+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2c4804d350 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.467+0000 7f2c53fff700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2c44038e90 0x7f2c4403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.467+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5410dd60 con 0x7f2c5407de30 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.467+0000 7f2c53fff700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2c44038e90 0x7f2c4403b350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006fd0 tx=0x7f2c40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:14.217 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.472+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2c48022330 con 0x7f2c5407de30 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:12.740+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2c541a0fa0 con 0x7f2c5407de30 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.571+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2c4804a030 con 0x7f2c5407de30 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.574+0000 7f2c51ffb700 1 -- 172.21.15.8:0/203777548 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f2c4802da00 con 0x7f2c5407de30 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2c44038e90 msgr2=0x7f2c4403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2c44038e90 0x7f2c4403b350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006fd0 tx=0x7f2c40006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 msgr2=0x7f2c5419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5419d8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c48003f20 tx=0x7f2c48011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 shutdown_connections 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2c44038e90 0x7f2c4403b350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 --2- 172.21.15.8:0/203777548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c5407de30 0x7f2c5419d8d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 >> 172.21.15.8:0/203777548 conn(0x7f2c54102f60 msgr2=0x7f2c54103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 shutdown_connections 2024-02-19T16:13:14.218 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:13.575+0000 7f2c5adc9700 1 -- 172.21.15.8:0/203777548 wait complete. 2024-02-19T16:13:14.459 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:14 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:14.254+0000 7f1bb8b9a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:13:14.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:14 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/203777548' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-19T16:13:14.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:14 smithi008 ceph-mon[32573]: mgrmap e9: x(active, since 20s) 2024-02-19T16:13:15.185 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:14 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:14.935+0000 7f1bb8b9a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:13:15.185 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.040+0000 7f1bb8b9a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:13:15.185 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.184+0000 7f1bb8b9a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:13:15.480 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.283+0000 7f1bb8b9a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:13:15.480 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.369+0000 7f1bb8b9a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:13:15.492 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:13:15.492 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-19T16:13:15.492 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-19T16:13:15.492 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f3015ebd700 1 Processor -- start 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f3015ebd700 1 -- start start 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f3015ebd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f3010107d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f3015ebd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3010108340 con 0x7f3010107950 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f300f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f3010107d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f300f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f3010107d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40682/0 (socket says 172.21.15.8:40682) 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.654+0000 7f300f7fe700 1 -- 172.21.15.8:0/3656834810 learned_addr learned my addr 172.21.15.8:0/3656834810 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.655+0000 7f300f7fe700 1 -- 172.21.15.8:0/3656834810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010108b50 con 0x7f3010107950 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.655+0000 7f300f7fe700 1 --2- 172.21.15.8:0/3656834810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f3010107d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30040098a0 tx=0x7f3004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eb0a0e5216aacd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:15.493 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.655+0000 7f300e7fc700 1 -- 172.21.15.8:0/3656834810 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3004004160 con 0x7f3010107950 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.655+0000 7f300e7fc700 1 -- 172.21.15.8:0/3656834810 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30040042c0 con 0x7f3010107950 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.655+0000 7f300e7fc700 1 -- 172.21.15.8:0/3656834810 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3004011a80 con 0x7f3010107950 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 -- 172.21.15.8:0/3656834810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 msgr2=0x7f3010107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 --2- 172.21.15.8:0/3656834810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f3010107d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30040098a0 tx=0x7f3004009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 -- 172.21.15.8:0/3656834810 shutdown_connections 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 --2- 172.21.15.8:0/3656834810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f3010107d70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 -- 172.21.15.8:0/3656834810 >> 172.21.15.8:0/3656834810 conn(0x7f3010102f50 msgr2=0x7f3010105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 -- 172.21.15.8:0/3656834810 shutdown_connections 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 -- 172.21.15.8:0/3656834810 wait complete. 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.656+0000 7f3015ebd700 1 Processor -- start 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f3015ebd700 1 -- start start 2024-02-19T16:13:15.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f3015ebd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f301007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f3015ebd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301007e0c0 con 0x7f3010107950 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f301007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f301007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40696/0 (socket says 172.21.15.8:40696) 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300f7fe700 1 -- 172.21.15.8:0/1052231094 learned_addr learned my addr 172.21.15.8:0/1052231094 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300f7fe700 1 -- 172.21.15.8:0/1052231094 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004009550 con 0x7f3010107950 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300f7fe700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f301007db80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3004000c00 tx=0x7f3004011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30040039e0 con 0x7f3010107950 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3004003b40 con 0x7f3010107950 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301007c2b0 con 0x7f3010107950 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.657+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3004024820 con 0x7f3010107950 2024-02-19T16:13:15.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.658+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301007c750 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.658+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f3004024980 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.659+0000 7f300cff9700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 0x7f2ff803b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.659+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f300404cf90 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.659+0000 7f300effd700 1 -- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 msgr2=0x7f2ff803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.659+0000 7f300effd700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 0x7f2ff803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.659+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3010063610 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.664+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f3004024e60 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.860+0000 7f300effd700 1 -- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 msgr2=0x7f2ff803b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.860+0000 7f300effd700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 0x7f2ff803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.926+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f301010bd80 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.927+0000 7f300cff9700 1 -- 172.21.15.8:0/1052231094 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3004024c80 con 0x7f3010107950 2024-02-19T16:13:15.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 msgr2=0x7f2ff803b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 0x7f2ff803b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 msgr2=0x7f301007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f301007db80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3004000c00 tx=0x7f3004011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 shutdown_connections 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7f2ff8038f90 0x7f2ff803b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 --2- 172.21.15.8:0/1052231094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3010107950 0x7f301007db80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 >> 172.21.15.8:0/1052231094 conn(0x7f3010102f50 msgr2=0x7f3010103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 shutdown_connections 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:14.929+0000 7f3015ebd700 1 -- 172.21.15.8:0/1052231094 wait complete. 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for the mgr to restart... 2024-02-19T16:13:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr epoch 9... 2024-02-19T16:13:15.763 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1052231094' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-19T16:13:15.763 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.647+0000 7f1bb8b9a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:13:15.763 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.762+0000 7f1bb8b9a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:13:16.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.852+0000 7f1bb8b9a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:13:16.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:15 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:15.953+0000 7f1bb8b9a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:13:16.515 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:16 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:16.132+0000 7f1bb8b9a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:13:17.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:17.120+0000 7f1bb8b9a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:13:17.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:17.255+0000 7f1bb8b9a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:13:17.888 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:17.594+0000 7f1bb8b9a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:13:17.889 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:17.686+0000 7f1bb8b9a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:13:17.889 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:17.887+0000 7f1bb8b9a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:13:18.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:18 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:18.080+0000 7f1bb8b9a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:13:18.634 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:18 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:18.372+0000 7f1bb8b9a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:13:18.635 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:18 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:18.633+0000 7f1bb8b9a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:13:19.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:18 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:18.884+0000 7f1bb8b9a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:13:20.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:19 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:19.713+0000 7f1bb8b9a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:13:20.618 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:20.293+0000 7f1bb8b9a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:13:20.618 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:20.394+0000 7f1bb8b9a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:13:20.618 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:13:20.617+0000 7f1bb8b9a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:13:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: Active manager daemon x restarted 2024-02-19T16:13:21.135 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: Activating manager daemon x 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: osdmap e3: 0 total, 0 up, 0 in 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: mgrmap e10: x(active, starting, since 0.00327428s) 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: Manager daemon x is now available 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:21.136 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-19T16:13:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-19T16:13:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:21 smithi008 ceph-mon[32573]: mgrmap e11: x(active, since 1.0064s) 2024-02-19T16:13:22.262 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.908+0000 7fdfb61bd700 1 Processor -- start 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.908+0000 7fdfb61bd700 1 -- start start 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.908+0000 7fdfb61bd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb0106680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.908+0000 7fdfb61bd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb007be20 con 0x7fdfb0106260 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfaf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb0106680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfaf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb0106680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44490/0 (socket says 172.21.15.8:44490) 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfaf7fe700 1 -- 172.21.15.8:0/400590202 learned_addr learned my addr 172.21.15.8:0/400590202 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:22.263 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfaf7fe700 1 -- 172.21.15.8:0/400590202 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb007bf60 con 0x7fdfb0106260 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfaf7fe700 1 --2- 172.21.15.8:0/400590202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb0106680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdfa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b4b381bc19e22c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfae7fc700 1 -- 172.21.15.8:0/400590202 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0004160 con 0x7fdfb0106260 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfae7fc700 1 -- 172.21.15.8:0/400590202 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa00042c0 con 0x7fdfb0106260 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.909+0000 7fdfae7fc700 1 -- 172.21.15.8:0/400590202 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0011a80 con 0x7fdfb0106260 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 -- 172.21.15.8:0/400590202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 msgr2=0x7fdfb0106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/400590202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb0106680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdfa0009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.264 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 -- 172.21.15.8:0/400590202 shutdown_connections 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/400590202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb0106680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 -- 172.21.15.8:0/400590202 >> 172.21.15.8:0/400590202 conn(0x7fdfb0101e40 msgr2=0x7fdfb0104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 -- 172.21.15.8:0/400590202 shutdown_connections 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.910+0000 7fdfb61bd700 1 -- 172.21.15.8:0/400590202 wait complete. 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfb61bd700 1 Processor -- start 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfb61bd700 1 -- start start 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfb61bd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:22.265 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfb61bd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb019ddc0 con 0x7fdfb0106260 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfaf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfaf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44498/0 (socket says 172.21.15.8:44498) 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfaf7fe700 1 -- 172.21.15.8:0/4069005046 learned_addr learned my addr 172.21.15.8:0/4069005046 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.911+0000 7fdfaf7fe700 1 -- 172.21.15.8:0/4069005046 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0009550 con 0x7fdfb0106260 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.912+0000 7fdfaf7fe700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb019d880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0011740 tx=0x7fdfa0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.912+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0003b20 con 0x7fdfb0106260 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.912+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa001a910 con 0x7fdfb0106260 2024-02-19T16:13:22.266 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.912+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb019dfc0 con 0x7fdfb0106260 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.912+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0024900 con 0x7fdfb0106260 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.912+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb019e460 con 0x7fdfb0106260 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.913+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fdfa0024b70 con 0x7fdfb0106260 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.913+0000 7fdfacff9700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.913+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 --> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c03bab0 con 0x7fdf9c038ee0 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.913+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdfa004d290 con 0x7fdfb0106260 2024-02-19T16:13:22.267 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.913+0000 7fdfaeffd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 msgr2=0x7fdf9c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:15.913+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:16.114+0000 7fdfaeffd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 msgr2=0x7fdf9c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:16.114+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:16.515+0000 7fdfaeffd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 msgr2=0x7fdf9c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:16.515+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:17.316+0000 7fdfaeffd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 msgr2=0x7fdf9c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:17.316+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:18.917+0000 7fdfaeffd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 msgr2=0x7fdf9c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/886492473 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:18.917+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:20.632+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7fdfa001a430 con 0x7fdfb0106260 2024-02-19T16:13:22.268 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:20.632+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 msgr2=0x7fdf9c03b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:20.632+0000 7fdfacff9700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.633+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7fdfa002d3f0 con 0x7fdfb0106260 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.634+0000 7fdfacff9700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf9c03c960 0x7fdf9c03ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.634+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c03bab0 con 0x7fdf9c03c960 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.635+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf9c03c960 0x7fdf9c03ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.635+0000 7fdfaeffd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf9c03c960 0x7fdf9c03ed50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf98003a10 tx=0x7fdf980092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.636+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdf9c03bab0 con 0x7fdf9c03c960 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.647+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdfb0107940 con 0x7fdf9c03c960 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfacff9700 1 -- 172.21.15.8:0/4069005046 <== mgr.14152 v2:172.21.15.8:6800/354789085 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fdfb0107940 con 0x7fdf9c03c960 2024-02-19T16:13:22.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf9c03c960 msgr2=0x7fdf9c03ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf9c03c960 0x7fdf9c03ed50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf98003a10 tx=0x7fdf980092b0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 msgr2=0x7fdfb019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb019d880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0011740 tx=0x7fdfa0011a50 comp rx=0 tx=0).stop 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 shutdown_connections 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf9c03c960 0x7fdf9c03ed50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfb0106260 0x7fdfb019d880 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 --2- 172.21.15.8:0/4069005046 >> [v2:172.21.15.8:6800/886492473,v1:172.21.15.8:6801/886492473] conn(0x7fdf9c038ee0 0x7fdf9c03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 >> 172.21.15.8:0/4069005046 conn(0x7fdfb0101e40 msgr2=0x7fdfb0194680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 shutdown_connections 2024-02-19T16:13:22.270 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:21.648+0000 7fdfb61bd700 1 -- 172.21.15.8:0/4069005046 wait complete. 2024-02-19T16:13:22.271 INFO:teuthology.orchestra.run.smithi008.stdout:mgr epoch 9 is available 2024-02-19T16:13:22.271 INFO:teuthology.orchestra.run.smithi008.stdout:Generating a dashboard self-signed certificate... 2024-02-19T16:13:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:22 smithi008 ceph-mon[32573]: [19/Feb/2024:16:13:21] ENGINE Bus STARTING 2024-02-19T16:13:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:22 smithi008 ceph-mon[32573]: [19/Feb/2024:16:13:21] ENGINE Serving on https://172.21.15.8:7150 2024-02-19T16:13:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:22 smithi008 ceph-mon[32573]: [19/Feb/2024:16:13:22] ENGINE Serving on http://172.21.15.8:8765 2024-02-19T16:13:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:22 smithi008 ceph-mon[32573]: [19/Feb/2024:16:13:22] ENGINE Bus STARTED 2024-02-19T16:13:23.605 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-19T16:13:23.605 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.617+0000 7f2f21f5d700 1 Processor -- start 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.617+0000 7f2f21f5d700 1 -- start start 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f21f5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c107d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f21f5d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c108350 con 0x7f2f1c107960 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f1b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f1b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c107d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44588/0 (socket says 172.21.15.8:44588) 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f1b7fe700 1 -- 172.21.15.8:0/943464916 learned_addr learned my addr 172.21.15.8:0/943464916 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f1b7fe700 1 -- 172.21.15.8:0/943464916 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c108b60 con 0x7f2f1c107960 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f1b7fe700 1 --2- 172.21.15.8:0/943464916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c107d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0098a0 tx=0x7f2f0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f81290993571165 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.618+0000 7f2f1a7fc700 1 -- 172.21.15.8:0/943464916 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f0c004160 con 0x7f2f1c107960 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f1a7fc700 1 -- 172.21.15.8:0/943464916 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c0042c0 con 0x7f2f1c107960 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f1a7fc700 1 -- 172.21.15.8:0/943464916 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f0c011a80 con 0x7f2f1c107960 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 -- 172.21.15.8:0/943464916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 msgr2=0x7f2f1c107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:23.606 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 --2- 172.21.15.8:0/943464916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c107d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0098a0 tx=0x7f2f0c009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 -- 172.21.15.8:0/943464916 shutdown_connections 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 --2- 172.21.15.8:0/943464916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c107d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 -- 172.21.15.8:0/943464916 >> 172.21.15.8:0/943464916 conn(0x7f2f1c102fa0 msgr2=0x7f2f1c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 -- 172.21.15.8:0/943464916 shutdown_connections 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.619+0000 7f2f21f5d700 1 -- 172.21.15.8:0/943464916 wait complete. 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f21f5d700 1 Processor -- start 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f21f5d700 1 -- start start 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f21f5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f21f5d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c19bca0 con 0x7f2f1c107960 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f1b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f1b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44602/0 (socket says 172.21.15.8:44602) 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.620+0000 7f2f1b7fe700 1 -- 172.21.15.8:0/2267449289 learned_addr learned my addr 172.21.15.8:0/2267449289 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f1b7fe700 1 -- 172.21.15.8:0/2267449289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c009550 con 0x7f2f1c107960 2024-02-19T16:13:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f1b7fe700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c19b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c000c00 tx=0x7f2f0c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f0c0039e0 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c003b40 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f0c024820 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c19bea0 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.621+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c19c340 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.622+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f2f0c024980 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.622+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c10bd90 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.622+0000 7f2f18ff9700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f08038e80 0x7f2f0803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.622+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f0c04cff0 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.622+0000 7f2f1affd700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f08038e80 0x7f2f0803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.623+0000 7f2f1affd700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f08038e80 0x7f2f0803b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006fd0 tx=0x7f2f04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.627+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2f0c024c30 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.684+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f2f0c02d9b0 con 0x7f2f1c107960 2024-02-19T16:13:23.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:22.838+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f1c104160 con 0x7f2f08038e80 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.018+0000 7f2f18ff9700 1 -- 172.21.15.8:0/2267449289 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f2f1c104160 con 0x7f2f08038e80 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f08038e80 msgr2=0x7f2f0803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f08038e80 0x7f2f0803b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006fd0 tx=0x7f2f04006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 msgr2=0x7f2f1c19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c19b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c000c00 tx=0x7f2f0c011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 shutdown_connections 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f08038e80 0x7f2f0803b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 --2- 172.21.15.8:0/2267449289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f1c107960 0x7f2f1c19b760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 >> 172.21.15.8:0/2267449289 conn(0x7f2f1c102fa0 msgr2=0x7f2f1c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 shutdown_connections 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:23.021+0000 7f2f21f5d700 1 -- 172.21.15.8:0/2267449289 wait complete. 2024-02-19T16:13:23.609 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial admin user... 2024-02-19T16:13:23.909 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:23 smithi008 ceph-mon[32573]: mgrmap e12: x(active, since 2s) 2024-02-19T16:13:23.909 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:23 smithi008 ceph-mon[32573]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:23.909 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:23.909 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:23.909 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:25.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$NP157aeDaKucdGFwm0/Oee/wdr7VBIPlaCYFGrarZaYWs7Y2cfbd.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708359204, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-19T16:13:25.098 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.013+0000 7feae6113700 1 Processor -- start 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.013+0000 7feae6113700 1 -- start start 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.013+0000 7feae6113700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae0107d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.013+0000 7feae6113700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae0108340 con 0x7feae0107950 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.013+0000 7feadf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae0107d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feadf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae0107d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44608/0 (socket says 172.21.15.8:44608) 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feadf7fe700 1 -- 172.21.15.8:0/1070490861 learned_addr learned my addr 172.21.15.8:0/1070490861 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feadf7fe700 1 -- 172.21.15.8:0/1070490861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0108b50 con 0x7feae0107950 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feadf7fe700 1 --2- 172.21.15.8:0/1070490861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae0107d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fead00098a0 tx=0x7fead0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c609f9205c0749 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feade7fc700 1 -- 172.21.15.8:0/1070490861 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fead0004160 con 0x7feae0107950 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feade7fc700 1 -- 172.21.15.8:0/1070490861 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead00042c0 con 0x7feae0107950 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.014+0000 7feade7fc700 1 -- 172.21.15.8:0/1070490861 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fead0011a80 con 0x7feae0107950 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 -- 172.21.15.8:0/1070490861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 msgr2=0x7feae0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 --2- 172.21.15.8:0/1070490861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae0107d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fead00098a0 tx=0x7fead0009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:25.099 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 -- 172.21.15.8:0/1070490861 shutdown_connections 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 --2- 172.21.15.8:0/1070490861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae0107d70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 -- 172.21.15.8:0/1070490861 >> 172.21.15.8:0/1070490861 conn(0x7feae0102f50 msgr2=0x7feae0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 -- 172.21.15.8:0/1070490861 shutdown_connections 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 -- 172.21.15.8:0/1070490861 wait complete. 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.015+0000 7feae6113700 1 Processor -- start 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feae6113700 1 -- start start 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feae6113700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feae6113700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae007e0c0 con 0x7feae0107950 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feadf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feadf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44616/0 (socket says 172.21.15.8:44616) 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feadf7fe700 1 -- 172.21.15.8:0/777973348 learned_addr learned my addr 172.21.15.8:0/777973348 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feadf7fe700 1 -- 172.21.15.8:0/777973348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0009550 con 0x7feae0107950 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.016+0000 7feadf7fe700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae007db80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fead0003f20 tx=0x7fead0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.017+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fead00038b0 con 0x7feae0107950 2024-02-19T16:13:25.100 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.017+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae007c2b0 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.017+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead0003a10 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.017+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae007c750 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.017+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fead001a5c0 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.017+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fead001a860 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.018+0000 7feadcff9700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feac8038f40 0x7feac803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.018+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc005360 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.018+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fead004d1d0 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.018+0000 7feadeffd700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feac8038f40 0x7feac803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.018+0000 7feadeffd700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feac8038f40 0x7feac803b400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fead4006fd0 tx=0x7fead4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.023+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fead0018c20 con 0x7feae0107950 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.257+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7feacc002430 con 0x7feac8038f40 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.504+0000 7feadcff9700 1 -- 172.21.15.8:0/777973348 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7feacc002430 con 0x7feac8038f40 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feac8038f40 msgr2=0x7feac803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:25.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feac8038f40 0x7feac803b400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fead4006fd0 tx=0x7fead4006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 msgr2=0x7feae007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae007db80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fead0003f20 tx=0x7fead0011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 shutdown_connections 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feac8038f40 0x7feac803b400 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 --2- 172.21.15.8:0/777973348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feae0107950 0x7feae007db80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 >> 172.21.15.8:0/777973348 conn(0x7feae0102f50 msgr2=0x7feae0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 shutdown_connections 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:24.506+0000 7feae6113700 1 -- 172.21.15.8:0/777973348 wait complete. 2024-02-19T16:13:25.102 INFO:teuthology.orchestra.run.smithi008.stdout:Fetching dashboard port number... 2024-02-19T16:13:25.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:25 smithi008 ceph-mon[32573]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:25.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:25.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:27.360 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 8443 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.522+0000 7fbe79fd5700 1 Processor -- start 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.522+0000 7fbe79fd5700 1 -- start start 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.522+0000 7fbe79fd5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7410a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.522+0000 7fbe79fd5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7407be20 con 0x7fbe74108470 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7410a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7410a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40162/0 (socket says 172.21.15.8:40162) 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe737fe700 1 -- 172.21.15.8:0/1071420026 learned_addr learned my addr 172.21.15.8:0/1071420026 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe737fe700 1 -- 172.21.15.8:0/1071420026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7407bf60 con 0x7fbe74108470 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe737fe700 1 --2- 172.21.15.8:0/1071420026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7410a860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098a0 tx=0x7fbe64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb50e4a7ea67d413 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe727fc700 1 -- 172.21.15.8:0/1071420026 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64004160 con 0x7fbe74108470 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe727fc700 1 -- 172.21.15.8:0/1071420026 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe640042c0 con 0x7fbe74108470 2024-02-19T16:13:27.361 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.523+0000 7fbe727fc700 1 -- 172.21.15.8:0/1071420026 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64011a80 con 0x7fbe74108470 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 -- 172.21.15.8:0/1071420026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 msgr2=0x7fbe7410a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 --2- 172.21.15.8:0/1071420026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7410a860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098a0 tx=0x7fbe64009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 -- 172.21.15.8:0/1071420026 shutdown_connections 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 --2- 172.21.15.8:0/1071420026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7410a860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 -- 172.21.15.8:0/1071420026 >> 172.21.15.8:0/1071420026 conn(0x7fbe74101e40 msgr2=0x7fbe74104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 -- 172.21.15.8:0/1071420026 shutdown_connections 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 -- 172.21.15.8:0/1071420026 wait complete. 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.524+0000 7fbe79fd5700 1 Processor -- start 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe79fd5700 1 -- start start 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe79fd5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe79fd5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7419bbc0 con 0x7fbe74108470 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40174/0 (socket says 172.21.15.8:40174) 2024-02-19T16:13:27.362 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe737fe700 1 -- 172.21.15.8:0/850162280 learned_addr learned my addr 172.21.15.8:0/850162280 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe737fe700 1 -- 172.21.15.8:0/850162280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64009550 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.525+0000 7fbe737fe700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7419b680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe64011740 tx=0x7fbe64011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.526+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64003b20 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.526+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe6401a910 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.526+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7419bdc0 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.526+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe640248c0 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.526+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7419c260 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.526+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fbe6402d3f0 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.527+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7410bb70 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.527+0000 7fbe70ff9700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbe60038f90 0x7fbe6003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.527+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe6404d5b0 con 0x7fbe74108470 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.527+0000 7fbe72ffd700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbe60038f90 0x7fbe6003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:27.363 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.527+0000 7fbe72ffd700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbe60038f90 0x7fbe6003b450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c006fd0 tx=0x7fbe5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.532+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbe64019500 con 0x7fbe74108470 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.718+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fbe6401aaf0 con 0x7fbe74108470 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.734+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fbe7419eb40 con 0x7fbe74108470 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.734+0000 7fbe70ff9700 1 -- 172.21.15.8:0/850162280 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fbe6400ba20 con 0x7fbe74108470 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbe60038f90 msgr2=0x7fbe6003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbe60038f90 0x7fbe6003b450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c006fd0 tx=0x7fbe5c006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 msgr2=0x7fbe7419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7419b680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe64011740 tx=0x7fbe64011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 shutdown_connections 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbe60038f90 0x7fbe6003b450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 --2- 172.21.15.8:0/850162280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe74108470 0x7fbe7419b680 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.736+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 >> 172.21.15.8:0/850162280 conn(0x7fbe74101e40 msgr2=0x7fbe74102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.737+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 shutdown_connections 2024-02-19T16:13:27.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:26.737+0000 7fbe79fd5700 1 -- 172.21.15.8:0/850162280 wait complete. 2024-02-19T16:13:27.367 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T16:13:27.367 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-02-19T16:13:27.375 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T16:13:27.375 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-02-19T16:13:27.375 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-02-19T16:13:27.375 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-19T16:13:27.376 INFO:teuthology.orchestra.run.smithi008.stdout:Ceph Dashboard is now available at: 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout: URL: https://smithi008.front.sepia.ceph.com:8443/ 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout: User: admin 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout: Password: xrzd4dbeaa 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout:Saving cluster configuration to /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config directory 2024-02-19T16:13:27.377 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling autotune for osd_memory_target 2024-02-19T16:13:28.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:27 smithi008 ceph-mon[32573]: mgrmap e13: x(active, since 6s) 2024-02-19T16:13:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:27 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/850162280' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.805+0000 7fc8db1cf700 1 Processor -- start 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8db1cf700 1 -- start start 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8db1cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d410a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8db1cf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d407be60 con 0x7fc8d4108430 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8d8f6b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d410a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8d8f6b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d410a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40188/0 (socket says 172.21.15.8:40188) 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8d8f6b700 1 -- 172.21.15.8:0/2800584628 learned_addr learned my addr 172.21.15.8:0/2800584628 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8d8f6b700 1 -- 172.21.15.8:0/2800584628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d410ad60 con 0x7fc8d4108430 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8d8f6b700 1 --2- 172.21.15.8:0/2800584628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d410a820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80098a0 tx=0x7fc8c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac67abdfe2ee0cc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.806+0000 7fc8d37fe700 1 -- 172.21.15.8:0/2800584628 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c8004160 con 0x7fc8d4108430 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8d37fe700 1 -- 172.21.15.8:0/2800584628 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8c80042c0 con 0x7fc8d4108430 2024-02-19T16:13:28.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8d37fe700 1 -- 172.21.15.8:0/2800584628 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c8011a80 con 0x7fc8d4108430 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 -- 172.21.15.8:0/2800584628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 msgr2=0x7fc8d410a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 --2- 172.21.15.8:0/2800584628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d410a820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80098a0 tx=0x7fc8c8009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 -- 172.21.15.8:0/2800584628 shutdown_connections 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 --2- 172.21.15.8:0/2800584628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d410a820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 -- 172.21.15.8:0/2800584628 >> 172.21.15.8:0/2800584628 conn(0x7fc8d4101ea0 msgr2=0x7fc8d41042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 -- 172.21.15.8:0/2800584628 shutdown_connections 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.807+0000 7fc8db1cf700 1 -- 172.21.15.8:0/2800584628 wait complete. 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8db1cf700 1 Processor -- start 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8db1cf700 1 -- start start 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8db1cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8db1cf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d419bc20 con 0x7fc8d4108430 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8d8f6b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8d8f6b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40204/0 (socket says 172.21.15.8:40204) 2024-02-19T16:13:28.560 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8d8f6b700 1 -- 172.21.15.8:0/3952051614 learned_addr learned my addr 172.21.15.8:0/3952051614 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.808+0000 7fc8d8f6b700 1 -- 172.21.15.8:0/3952051614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c8009550 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.809+0000 7fc8d8f6b700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d419b6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8003f20 tx=0x7fc8c8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.809+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c80039f0 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.809+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d419be20 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.809+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8c8003b50 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.809+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c801a6d0 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.809+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d419c2c0 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.810+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fc8c801a990 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.810+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d410bba0 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.810+0000 7fc8d1ffb700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8c4038f80 0x7fc8c403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.810+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc8c804d6e0 con 0x7fc8d4108430 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.810+0000 7fc8d3fff700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8c4038f80 0x7fc8c403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:28.561 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.811+0000 7fc8d3fff700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8c4038f80 0x7fc8c403b440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006fd0 tx=0x7fc8c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:27.815+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fc8c8022330 con 0x7fc8d4108430 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.017+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc8d419eba0 con 0x7fc8d4108430 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.018+0000 7fc8d1ffb700 1 -- 172.21.15.8:0/3952051614 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc8c802c350 con 0x7fc8d4108430 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.020+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8c4038f80 msgr2=0x7fc8c403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.020+0000 7fc8db1cf700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8c4038f80 0x7fc8c403b440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006fd0 tx=0x7fc8c0006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.020+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 msgr2=0x7fc8d419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.020+0000 7fc8db1cf700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d419b6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8003f20 tx=0x7fc8c8011b00 comp rx=0 tx=0).stop 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.020+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 shutdown_connections 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.021+0000 7fc8db1cf700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8c4038f80 0x7fc8c403b440 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.021+0000 7fc8db1cf700 1 --2- 172.21.15.8:0/3952051614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8d4108430 0x7fc8d419b6e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.021+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 >> 172.21.15.8:0/3952051614 conn(0x7fc8d4101ea0 msgr2=0x7fc8d4102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.021+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 shutdown_connections 2024-02-19T16:13:28.562 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.021+0000 7fc8db1cf700 1 -- 172.21.15.8:0/3952051614 wait complete. 2024-02-19T16:13:29.815 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.984+0000 7f7a1c7f0700 1 Processor -- start 2024-02-19T16:13:29.815 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.984+0000 7f7a1c7f0700 1 -- start start 2024-02-19T16:13:29.815 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1c7f0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a14107180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:29.815 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1c7f0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14107750 con 0x7f7a14106d60 2024-02-19T16:13:29.815 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1a58c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a14107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1a58c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a14107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40218/0 (socket says 172.21.15.8:40218) 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1a58c700 1 -- 172.21.15.8:0/3238311905 learned_addr learned my addr 172.21.15.8:0/3238311905 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1a58c700 1 -- 172.21.15.8:0/3238311905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a14107a80 con 0x7f7a14106d60 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1a58c700 1 --2- 172.21.15.8:0/3238311905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a14107180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a100098a0 tx=0x7f7a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fac9ad2db5941f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1958a700 1 -- 172.21.15.8:0/3238311905 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a10004160 con 0x7f7a14106d60 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1958a700 1 -- 172.21.15.8:0/3238311905 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a100042c0 con 0x7f7a14106d60 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.985+0000 7f7a1958a700 1 -- 172.21.15.8:0/3238311905 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a10011a80 con 0x7f7a14106d60 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/3238311905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 msgr2=0x7f7a14107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 --2- 172.21.15.8:0/3238311905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a14107180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a100098a0 tx=0x7f7a10009bb0 comp rx=0 tx=0).stop 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/3238311905 shutdown_connections 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 --2- 172.21.15.8:0/3238311905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a14107180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/3238311905 >> 172.21.15.8:0/3238311905 conn(0x7f7a141015a0 msgr2=0x7f7a141039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:29.816 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/3238311905 shutdown_connections 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.986+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/3238311905 wait complete. 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1c7f0700 1 Processor -- start 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1c7f0700 1 -- start start 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1c7f0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a1419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1c7f0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1419bbd0 con 0x7f7a14106d60 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1a58c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a1419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1a58c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a1419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40226/0 (socket says 172.21.15.8:40226) 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1a58c700 1 -- 172.21.15.8:0/2268386068 learned_addr learned my addr 172.21.15.8:0/2268386068 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1a58c700 1 -- 172.21.15.8:0/2268386068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a10009550 con 0x7f7a14106d60 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.987+0000 7f7a1a58c700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a1419b690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a10011740 tx=0x7f7a10011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a10003b20 con 0x7f7a14106d60 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1419bdd0 con 0x7f7a14106d60 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a1001a910 con 0x7f7a14106d60 2024-02-19T16:13:29.817 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a10024900 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1419c270 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f7a10024a80 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.988+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a141958d0 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.989+0000 7f7a0b7fe700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a00038fe0 0x7f7a0003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.989+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a1004d400 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.989+0000 7f7a19d8b700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a00038fe0 0x7f7a0003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.989+0000 7f7a19d8b700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a00038fe0 0x7f7a0003b4a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a04006fd0 tx=0x7f7a04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:28.994+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7a10018350 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.316+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f7a1410ac80 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.318+0000 7f7a0b7fe700 1 -- 172.21.15.8:0/2268386068 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f7a1000b3e0 con 0x7f7a14106d60 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a00038fe0 msgr2=0x7f7a0003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a00038fe0 0x7f7a0003b4a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a04006fd0 tx=0x7f7a04006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 msgr2=0x7f7a1419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a1419b690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a10011740 tx=0x7f7a10011a50 comp rx=0 tx=0).stop 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 shutdown_connections 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a00038fe0 0x7f7a0003b4a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 --2- 172.21.15.8:0/2268386068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14106d60 0x7f7a1419b690 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 >> 172.21.15.8:0/2268386068 conn(0x7f7a141015a0 msgr2=0x7f7a14102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 shutdown_connections 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-02-19T16:13:29.321+0000 7f7a1c7f0700 1 -- 172.21.15.8:0/2268386068 wait complete. 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout:Or, if you are only running a single cluster on this host: 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-19T16:13:29.819 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout: ceph telemetry on 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout:For more information see: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:29.820 INFO:teuthology.orchestra.run.smithi008.stdout:Bootstrap complete. 2024-02-19T16:13:29.853 INFO:tasks.cephadm:Fetching config... 2024-02-19T16:13:29.853 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:13:29.853 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-19T16:13:29.868 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-19T16:13:29.868 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:13:29.868 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-19T16:13:29.923 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-19T16:13:29.923 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:13:29.923 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-19T16:13:29.988 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-19T16:13:29.989 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:13:29.989 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-19T16:13:30.043 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-19T16:13:30.043 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-19T16:13:30.122 INFO:teuthology.orchestra.run.smithi008.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:13:30.136 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-19T16:13:30.181 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:13:30.200 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-19T16:13:30.246 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Bsn/czibI/pUmtofVbylJOaVWph6dXtCzndkxAL1QEPGqNU5e+bTbR+Kbt/fenA/0g1N5+KSt/zTMNedvTrgFUZ/gIi2Z/83XG9OojjzWpxCApVhYxTCnytLyYvL3grqXGSF/gEsywOI39cFdxtvbIu9nirYsRQWVoTePKYWLSSd3tnLETvksXnI/jzinnQa6vWA5q63Q5Q1wQdqle0C6Njue2M6rVwgYHXeZHRwStbaZOZVfu77ms8aFXGgkYGu24O9k/MdOgmVdJJxld3RW0CzVeuewzVi+cV8IibyWPqWernYt09K/9lryTlWAgENbSz7BMOTJ2UEzmzq8fGdbon5oAjzGTOJ0gRBptOdb6BYMhOhgGnjq/yOzvag76Adskb5+5n7EWyoGwdl1HmgvJ8JSDN2SkQf17NkX6W0UWgk3hq/gNjXzEjxVeurOqGu8SOfzX5f/0xrgKvN1l+HuE4k5vHL0k+aLk2TEeMjrTSUJmcVAhPwDSPgbudnCPc= ceph-81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:13:30.261 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-19T16:13:30.500 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:30 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2268386068' entity='client.admin' 2024-02-19T16:13:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 -- 172.21.15.8:0/676712175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 msgr2=0x7f83a8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 --2- 172.21.15.8:0/676712175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8100d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83a000b0a0 tx=0x7f83a000b3b0 comp rx=0 tx=0).stop 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 -- 172.21.15.8:0/676712175 shutdown_connections 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 --2- 172.21.15.8:0/676712175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8100d30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 -- 172.21.15.8:0/676712175 >> 172.21.15.8:0/676712175 conn(0x7f83a80fbeb0 msgr2=0x7f83a80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 -- 172.21.15.8:0/676712175 shutdown_connections 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 -- 172.21.15.8:0/676712175 wait complete. 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 Processor -- start 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.798+0000 7f83aeb15700 1 -- start start 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83aeb15700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83aeb15700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a806a390 con 0x7f83a8100910 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83ac8b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83ac8b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8069e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40246/0 (socket says 172.21.15.8:40246) 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83ac8b1700 1 -- 172.21.15.8:0/2724236320 learned_addr learned my addr 172.21.15.8:0/2724236320 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83ac8b1700 1 -- 172.21.15.8:0/2724236320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009d00 con 0x7f83a8100910 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83ac8b1700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8069e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83a00093e0 tx=0x7f83a000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a0021070 con 0x7f83a8100910 2024-02-19T16:13:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83a0003c10 con 0x7f83a8100910 2024-02-19T16:13:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a000e040 con 0x7f83a8100910 2024-02-19T16:13:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.799+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a8199c70 con 0x7f83a8100910 2024-02-19T16:13:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.800+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a819a110 con 0x7f83a8100910 2024-02-19T16:13:32.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.800+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f83a0019070 con 0x7f83a8100910 2024-02-19T16:13:32.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.800+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a818f770 con 0x7f83a8100910 2024-02-19T16:13:32.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.801+0000 7f83a5ffb700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8390039020 0x7f839003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:32.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.801+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f83a004ca90 con 0x7f83a8100910 2024-02-19T16:13:32.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.801+0000 7f83a7fff700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8390039020 0x7f839003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:32.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.801+0000 7f83a7fff700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8390039020 0x7f839003b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:32.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:32.806+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f83a001c3d0 con 0x7f83a8100910 2024-02-19T16:13:33.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.006+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f83a8063610 con 0x7f83a8100910 2024-02-19T16:13:33.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.008+0000 7f83a5ffb700 1 -- 172.21.15.8:0/2724236320 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f83a8063610 con 0x7f83a8100910 2024-02-19T16:13:33.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8390039020 msgr2=0x7f839003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:33.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8390039020 0x7f839003b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:33.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 msgr2=0x7f83a8069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8069e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83a00093e0 tx=0x7f83a000bd40 comp rx=0 tx=0).stop 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 shutdown_connections 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8390039020 0x7f839003b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 --2- 172.21.15.8:0/2724236320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83a8100910 0x7f83a8069e50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 >> 172.21.15.8:0/2724236320 conn(0x7f83a80fbeb0 msgr2=0x7f83a80fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 shutdown_connections 2024-02-19T16:13:33.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:33.011+0000 7f83aeb15700 1 -- 172.21.15.8:0/2724236320 wait complete. 2024-02-19T16:13:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi008", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:13:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:33.611 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-19T16:13:33.612 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-19T16:13:33.855 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:34.258 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:34 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2724236320' entity='client.admin' 2024-02-19T16:13:34.258 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:34.258 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 -- 172.21.15.8:0/394852811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 msgr2=0x7ff394101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 --2- 172.21.15.8:0/394852811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394101e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff384009910 tx=0x7ff384009c20 comp rx=0 tx=0).stop 2024-02-19T16:13:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 -- 172.21.15.8:0/394852811 shutdown_connections 2024-02-19T16:13:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 --2- 172.21.15.8:0/394852811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394101e50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 -- 172.21.15.8:0/394852811 >> 172.21.15.8:0/394852811 conn(0x7ff3940fcfd0 msgr2=0x7ff3940ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 -- 172.21.15.8:0/394852811 shutdown_connections 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.363+0000 7ff39a2e1700 1 -- 172.21.15.8:0/394852811 wait complete. 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff39a2e1700 1 Processor -- start 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff39a2e1700 1 -- start start 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff39a2e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff39a2e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394076190 con 0x7ff394101a30 2024-02-19T16:13:35.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff393fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff393fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40270/0 (socket says 172.21.15.8:40270) 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.364+0000 7ff393fff700 1 -- 172.21.15.8:0/4217784411 learned_addr learned my addr 172.21.15.8:0/4217784411 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff393fff700 1 -- 172.21.15.8:0/4217784411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840095f0 con 0x7ff394101a30 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff393fff700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394075c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff384005850 tx=0x7ff384004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff38401d070 con 0x7ff394101a30 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff38400be40 con 0x7ff394101a30 2024-02-19T16:13:35.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3940743d0 con 0x7ff394101a30 2024-02-19T16:13:35.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff384022ea0 con 0x7ff394101a30 2024-02-19T16:13:35.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.365+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff394074870 con 0x7ff394101a30 2024-02-19T16:13:35.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.366+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7ff38400f4b0 con 0x7ff394101a30 2024-02-19T16:13:35.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.366+0000 7ff3917fa700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff37c038fe0 0x7ff37c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:35.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.366+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff38404d320 con 0x7ff394101a30 2024-02-19T16:13:35.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.366+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3941081c0 con 0x7ff394101a30 2024-02-19T16:13:35.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.366+0000 7ff3937fe700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff37c038fe0 0x7ff37c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:35.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.367+0000 7ff3937fe700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff37c038fe0 0x7ff37c03b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff388006fd0 tx=0x7ff388006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:35.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.371+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff384027020 con 0x7ff394101a30 2024-02-19T16:13:35.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.584+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff394107bb0 con 0x7ff37c038fe0 2024-02-19T16:13:35.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.587+0000 7ff3917fa700 1 -- 172.21.15.8:0/4217784411 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff394107bb0 con 0x7ff37c038fe0 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff37c038fe0 msgr2=0x7ff37c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff37c038fe0 0x7ff37c03b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff388006fd0 tx=0x7ff388006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 msgr2=0x7ff394075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394075c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff384005850 tx=0x7ff384004ff0 comp rx=0 tx=0).stop 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 shutdown_connections 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff37c038fe0 0x7ff37c03b4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 --2- 172.21.15.8:0/4217784411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff394101a30 0x7ff394075c50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 >> 172.21.15.8:0/4217784411 conn(0x7ff3940fcfd0 msgr2=0x7ff3940fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:35.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 shutdown_connections 2024-02-19T16:13:35.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:35.589+0000 7ff39a2e1700 1 -- 172.21.15.8:0/4217784411 wait complete. 2024-02-19T16:13:36.333 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2024-02-19T16:13:36.333 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:13:36.333 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2024-02-19T16:13:36.351 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:13:36.351 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:13:36.409 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2024-02-19T16:13:36.409 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch host add smithi156 2024-02-19T16:13:36.651 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:36.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:37.861 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:37 smithi008 ceph-mon[32573]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:13:37.861 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:37 smithi008 ceph-mon[32573]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:13:37.861 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:37 smithi008 ceph-mon[32573]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:13:38.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 -- 172.21.15.8:0/3324335889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 msgr2=0x7fe994106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:38.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 --2- 172.21.15.8:0/3324335889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe994106c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe984009910 tx=0x7fe984009c20 comp rx=0 tx=0).stop 2024-02-19T16:13:38.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 -- 172.21.15.8:0/3324335889 shutdown_connections 2024-02-19T16:13:38.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 --2- 172.21.15.8:0/3324335889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe994106c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:38.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 -- 172.21.15.8:0/3324335889 >> 172.21.15.8:0/3324335889 conn(0x7fe994077520 msgr2=0x7fe9940fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:38.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 -- 172.21.15.8:0/3324335889 shutdown_connections 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 -- 172.21.15.8:0/3324335889 wait complete. 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 Processor -- start 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.078+0000 7fe99ade7700 1 -- start start 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe99ade7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe9941961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe99ade7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941966e0 con 0x7fe9941068b0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe998b83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe9941961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe998b83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe9941961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58722/0 (socket says 172.21.15.8:58722) 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe998b83700 1 -- 172.21.15.8:0/3060364667 learned_addr learned my addr 172.21.15.8:0/3060364667 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe998b83700 1 -- 172.21.15.8:0/3060364667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9840095f0 con 0x7fe9941068b0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe998b83700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe9941961a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe984004f40 tx=0x7fe984005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98401d070 con 0x7fe9941068b0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe994196940 con 0x7fe9941068b0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe98400bbd0 con 0x7fe9941068b0 2024-02-19T16:13:38.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe984003900 con 0x7fe9941068b0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.079+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe994196dc0 con 0x7fe9941068b0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.080+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fe984003bc0 con 0x7fe9941068b0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.080+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe994101ff0 con 0x7fe9941068b0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.080+0000 7fe991ffb700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe97c038f80 0x7fe97c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.081+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe98404cec0 con 0x7fe9941068b0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.081+0000 7fe993fff700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe97c038f80 0x7fe97c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.081+0000 7fe993fff700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe97c038f80 0x7fe97c03b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe988006fd0 tx=0x7fe988006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:38.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.086+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe98402ae20 con 0x7fe9941068b0 2024-02-19T16:13:38.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:38.294+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7fe99410adc0 con 0x7fe97c038f80 2024-02-19T16:13:39.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:38 smithi008 ceph-mon[32573]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:40.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:39 smithi008 ceph-mon[32573]: Deploying cephadm binary to smithi156 2024-02-19T16:13:42.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:41 smithi008 ceph-mon[32573]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:42.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.654+0000 7fe991ffb700 1 -- 172.21.15.8:0/3060364667 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe99410adc0 con 0x7fe97c038f80 2024-02-19T16:13:42.655 INFO:teuthology.orchestra.run.smithi008.stdout:Added host 'smithi156' with addr '172.21.15.156' 2024-02-19T16:13:42.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe97c038f80 msgr2=0x7fe97c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:42.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe97c038f80 0x7fe97c03b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe988006fd0 tx=0x7fe988006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:42.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 msgr2=0x7fe9941961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:42.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe9941961a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe984004f40 tx=0x7fe984005020 comp rx=0 tx=0).stop 2024-02-19T16:13:42.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 shutdown_connections 2024-02-19T16:13:42.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe97c038f80 0x7fe97c03b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:42.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 --2- 172.21.15.8:0/3060364667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9941068b0 0x7fe9941961a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:42.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.657+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 >> 172.21.15.8:0/3060364667 conn(0x7fe994077520 msgr2=0x7fe994108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:42.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.658+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 shutdown_connections 2024-02-19T16:13:42.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:42.658+0000 7fe99ade7700 1 -- 172.21.15.8:0/3060364667 wait complete. 2024-02-19T16:13:43.502 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch host ls --format=json 2024-02-19T16:13:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:43 smithi008 ceph-mon[32573]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:43 smithi008 ceph-mon[32573]: Added host smithi156 2024-02-19T16:13:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:44.982 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:44 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 -- 172.21.15.8:0/3270629159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 msgr2=0x7fef50107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 --2- 172.21.15.8:0/3270629159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50107d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef40009910 tx=0x7fef40009c20 comp rx=0 tx=0).stop 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 -- 172.21.15.8:0/3270629159 shutdown_connections 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 --2- 172.21.15.8:0/3270629159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50107d90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 -- 172.21.15.8:0/3270629159 >> 172.21.15.8:0/3270629159 conn(0x7fef500fd4c0 msgr2=0x7fef500ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 -- 172.21.15.8:0/3270629159 shutdown_connections 2024-02-19T16:13:45.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.505+0000 7fef568f9700 1 -- 172.21.15.8:0/3270629159 wait complete. 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef568f9700 1 Processor -- start 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef568f9700 1 -- start start 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef568f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef568f9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50197850 con 0x7fef501079b0 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44164/0 (socket says 172.21.15.8:44164) 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef4ffff700 1 -- 172.21.15.8:0/381024380 learned_addr learned my addr 172.21.15.8:0/381024380 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.506+0000 7fef4ffff700 1 -- 172.21.15.8:0/381024380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400095f0 con 0x7fef501079b0 2024-02-19T16:13:45.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.507+0000 7fef4ffff700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50197310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef40004f40 tx=0x7fef40005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:45.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.507+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef4001d070 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.507+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4000bbd0 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.507+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef40003900 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.507+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef50197ab0 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.507+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef50197f30 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.508+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fef40003bc0 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.508+0000 7fef4d7fa700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fef3c038f80 0x7fef3c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.508+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef4004cdb0 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.508+0000 7fef4f7fe700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fef3c038f80 0x7fef3c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.508+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef50063610 con 0x7fef501079b0 2024-02-19T16:13:45.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.508+0000 7fef4f7fe700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fef3c038f80 0x7fef3c03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef38006fd0 tx=0x7fef38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:45.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.513+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fef40026080 con 0x7fef501079b0 2024-02-19T16:13:45.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.729+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef5010bdf0 con 0x7fef3c038f80 2024-02-19T16:13:45.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.730+0000 7fef4d7fa700 1 -- 172.21.15.8:0/381024380 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fef5010bdf0 con 0x7fef3c038f80 2024-02-19T16:13:45.731 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:45.731 INFO:teuthology.orchestra.run.smithi008.stdout:[{"addr": "172.21.15.8", "hostname": "smithi008", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2024-02-19T16:13:45.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.732+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fef3c038f80 msgr2=0x7fef3c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:45.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.732+0000 7fef568f9700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fef3c038f80 0x7fef3c03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef38006fd0 tx=0x7fef38006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:45.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.732+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 msgr2=0x7fef50197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.732+0000 7fef568f9700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50197310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef40004f40 tx=0x7fef40005020 comp rx=0 tx=0).stop 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.733+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 shutdown_connections 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.733+0000 7fef568f9700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fef3c038f80 0x7fef3c03b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.733+0000 7fef568f9700 1 --2- 172.21.15.8:0/381024380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef501079b0 0x7fef50197310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.733+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 >> 172.21.15.8:0/381024380 conn(0x7fef500fd4c0 msgr2=0x7fef500fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.733+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 shutdown_connections 2024-02-19T16:13:45.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:45.733+0000 7fef568f9700 1 -- 172.21.15.8:0/381024380 wait complete. 2024-02-19T16:13:46.456 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:46 smithi008 ceph-mon[32573]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:46.456 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:46 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:46.484 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-02-19T16:13:46.484 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:13:46.485 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-02-19T16:13:46.505 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:13:46.505 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:13:46.562 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-02-19T16:13:46.563 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch host add smithi191 2024-02-19T16:13:46.803 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:47.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:47 smithi008 ceph-mon[32573]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:13:48.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.299+0000 7fb8a3a91700 1 -- 172.21.15.8:0/3491273960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 msgr2=0x7fb89c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:48.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.299+0000 7fb8a3a91700 1 --2- 172.21.15.8:0/3491273960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c106c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb898009910 tx=0x7fb898009c20 comp rx=0 tx=0).stop 2024-02-19T16:13:48.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.299+0000 7fb8a3a91700 1 -- 172.21.15.8:0/3491273960 shutdown_connections 2024-02-19T16:13:48.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.299+0000 7fb8a3a91700 1 --2- 172.21.15.8:0/3491273960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c106c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:48.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.299+0000 7fb8a3a91700 1 -- 172.21.15.8:0/3491273960 >> 172.21.15.8:0/3491273960 conn(0x7fb89c0fc380 msgr2=0x7fb89c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:48.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a3a91700 1 -- 172.21.15.8:0/3491273960 shutdown_connections 2024-02-19T16:13:48.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a3a91700 1 -- 172.21.15.8:0/3491273960 wait complete. 2024-02-19T16:13:48.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a3a91700 1 Processor -- start 2024-02-19T16:13:48.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a3a91700 1 -- start start 2024-02-19T16:13:48.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a3a91700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:48.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a3a91700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c06ae40 con 0x7fb89c1068b0 2024-02-19T16:13:48.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.300+0000 7fb8a182d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb8a182d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44180/0 (socket says 172.21.15.8:44180) 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb8a182d700 1 -- 172.21.15.8:0/2421700770 learned_addr learned my addr 172.21.15.8:0/2421700770 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb8a182d700 1 -- 172.21.15.8:0/2421700770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980095f0 con 0x7fb89c1068b0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb8a182d700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c198100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb898004f40 tx=0x7fb898005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb89801d070 con 0x7fb89c1068b0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb89800bc30 con 0x7fb89c1068b0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb898003970 con 0x7fb89c1068b0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c198700 con 0x7fb89c1068b0 2024-02-19T16:13:48.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.301+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c198b80 con 0x7fb89c1068b0 2024-02-19T16:13:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.302+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fb898003be0 con 0x7fb89c1068b0 2024-02-19T16:13:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.302+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c101ff0 con 0x7fb89c1068b0 2024-02-19T16:13:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.302+0000 7fb88effd700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb888038fe0 0x7fb88803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.302+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb89804cdd0 con 0x7fb89c1068b0 2024-02-19T16:13:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.302+0000 7fb8a102c700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb888038fe0 0x7fb88803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.303+0000 7fb8a102c700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb888038fe0 0x7fb88803b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb890006fd0 tx=0x7fb890006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:48.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.307+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb898050030 con 0x7fb89c1068b0 2024-02-19T16:13:48.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:48 smithi008 ceph-mon[32573]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:48.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:48.522+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}) v1 -- 0x7fb89c198dd0 con 0x7fb888038fe0 2024-02-19T16:13:49.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:49.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:49.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:49.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:49.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:50 smithi008 ceph-mon[32573]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:50 smithi008 ceph-mon[32573]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:13:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:50 smithi008 ceph-mon[32573]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:13:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:50 smithi008 ceph-mon[32573]: Deploying cephadm binary to smithi191 2024-02-19T16:13:52.719 INFO:teuthology.orchestra.run.smithi008.stdout:Added host 'smithi191' with addr '172.21.15.191' 2024-02-19T16:13:52.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.718+0000 7fb88effd700 1 -- 172.21.15.8:0/2421700770 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb89c198dd0 con 0x7fb888038fe0 2024-02-19T16:13:52.720 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:52 smithi008 ceph-mon[32573]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb888038fe0 msgr2=0x7fb88803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb888038fe0 0x7fb88803b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb890006fd0 tx=0x7fb890006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 msgr2=0x7fb89c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c198100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb898004f40 tx=0x7fb898005020 comp rx=0 tx=0).stop 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 shutdown_connections 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb888038fe0 0x7fb88803b4a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 --2- 172.21.15.8:0/2421700770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb89c1068b0 0x7fb89c198100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 >> 172.21.15.8:0/2421700770 conn(0x7fb89c0fc380 msgr2=0x7fb89c1083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 shutdown_connections 2024-02-19T16:13:52.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:52.721+0000 7fb8a3a91700 1 -- 172.21.15.8:0/2421700770 wait complete. 2024-02-19T16:13:53.391 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch host ls --format=json 2024-02-19T16:13:53.633 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:54.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:53 smithi008 ceph-mon[32573]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:54.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:54.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:53 smithi008 ceph-mon[32573]: Added host smithi191 2024-02-19T16:13:54.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:13:54.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:54.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 -- 172.21.15.8:0/1168487381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 msgr2=0x7f7238107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 --2- 172.21.15.8:0/1168487381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238107da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f722c009910 tx=0x7f722c009c20 comp rx=0 tx=0).stop 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 -- 172.21.15.8:0/1168487381 shutdown_connections 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 --2- 172.21.15.8:0/1168487381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238107da0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 -- 172.21.15.8:0/1168487381 >> 172.21.15.8:0/1168487381 conn(0x7f72380fd4d0 msgr2=0x7f72380ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 -- 172.21.15.8:0/1168487381 shutdown_connections 2024-02-19T16:13:55.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.165+0000 7f723eabb700 1 -- 172.21.15.8:0/1168487381 wait complete. 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723eabb700 1 Processor -- start 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723eabb700 1 -- start start 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723eabb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723eabb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238197860 con 0x7f72381079c0 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723c857700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723c857700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44212/0 (socket says 172.21.15.8:44212) 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723c857700 1 -- 172.21.15.8:0/3996072704 learned_addr learned my addr 172.21.15.8:0/3996072704 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723c857700 1 -- 172.21.15.8:0/3996072704 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0095f0 con 0x7f72381079c0 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.166+0000 7f723c857700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238197320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f722c004f40 tx=0x7f722c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.167+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f722c01d070 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.167+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f722c00bbd0 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.167+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7238197ac0 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.167+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f722c003900 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.167+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7238197f40 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.168+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f722c003bc0 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.168+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7238063610 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.168+0000 7f7235ffb700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7228038f80 0x7f722803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.168+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f722c04cf20 con 0x7f72381079c0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.168+0000 7f7237fff700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7228038f80 0x7f722803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.168+0000 7f7237fff700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7228038f80 0x7f722803b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7224006fd0 tx=0x7f7224006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:55.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.173+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f722c029910 con 0x7f72381079c0 2024-02-19T16:13:55.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.384+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f723810bf40 con 0x7f7228038f80 2024-02-19T16:13:55.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.385+0000 7f7235ffb700 1 -- 172.21.15.8:0/3996072704 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f723810bf40 con 0x7f7228038f80 2024-02-19T16:13:55.386 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:13:55.386 INFO:teuthology.orchestra.run.smithi008.stdout:[{"addr": "172.21.15.8", "hostname": "smithi008", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7228038f80 msgr2=0x7f722803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7228038f80 0x7f722803b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7224006fd0 tx=0x7f7224006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 msgr2=0x7f7238197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238197320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f722c004f40 tx=0x7f722c005020 comp rx=0 tx=0).stop 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 shutdown_connections 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7228038f80 0x7f722803b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.387+0000 7f723eabb700 1 --2- 172.21.15.8:0/3996072704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72381079c0 0x7f7238197320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.388+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 >> 172.21.15.8:0/3996072704 conn(0x7f72380fd4d0 msgr2=0x7f72380fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:55.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.388+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 shutdown_connections 2024-02-19T16:13:55.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:55.388+0000 7f723eabb700 1 -- 172.21.15.8:0/3996072704 wait complete. 2024-02-19T16:13:55.994 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-19T16:13:55.994 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd crush tunables default 2024-02-19T16:13:56.230 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:13:56.551 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:56 smithi008 ceph-mon[32573]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:56.551 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:56.551 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:56 smithi008 ceph-mon[32573]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:13:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 -- 172.21.15.8:0/3686194672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 msgr2=0x7f5d50106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 --2- 172.21.15.8:0/3686194672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50106ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009910 tx=0x7f5d44009c20 comp rx=0 tx=0).stop 2024-02-19T16:13:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 -- 172.21.15.8:0/3686194672 shutdown_connections 2024-02-19T16:13:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 --2- 172.21.15.8:0/3686194672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50106ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 -- 172.21.15.8:0/3686194672 >> 172.21.15.8:0/3686194672 conn(0x7f5d500fc3d0 msgr2=0x7f5d500fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 -- 172.21.15.8:0/3686194672 shutdown_connections 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.648+0000 7f5d57159700 1 -- 172.21.15.8:0/3686194672 wait complete. 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d57159700 1 Processor -- start 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d57159700 1 -- start start 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d57159700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d57159700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50198660 con 0x7f5d501068c0 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d54ef5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d54ef5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45642/0 (socket says 172.21.15.8:45642) 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.649+0000 7f5d54ef5700 1 -- 172.21.15.8:0/3169889875 learned_addr learned my addr 172.21.15.8:0/3169889875 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d54ef5700 1 -- 172.21.15.8:0/3169889875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440095f0 con 0x7f5d501068c0 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d54ef5700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50198120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5d44006010 tx=0x7f5d44004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d4401d070 con 0x7f5d501068c0 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d4400bbd0 con 0x7f5d501068c0 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d501988c0 con 0x7f5d501068c0 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d44003880 con 0x7f5d501068c0 2024-02-19T16:13:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.650+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d50198d40 con 0x7f5d501068c0 2024-02-19T16:13:57.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.651+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f5d44003a70 con 0x7f5d501068c0 2024-02-19T16:13:57.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.651+0000 7f5d4dffb700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d40038fe0 0x7f5d4003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:13:57.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.651+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d4404cca0 con 0x7f5d501068c0 2024-02-19T16:13:57.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.651+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d50102000 con 0x7f5d501068c0 2024-02-19T16:13:57.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.651+0000 7f5d4ffff700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d40038fe0 0x7f5d4003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:13:57.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.652+0000 7f5d4ffff700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d40038fe0 0x7f5d4003b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c006fd0 tx=0x7f5d3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:13:57.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.656+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5d44026080 con 0x7f5d501068c0 2024-02-19T16:13:57.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:57.868+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5d5006a490 con 0x7f5d501068c0 2024-02-19T16:13:58.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.287+0000 7f5d4dffb700 1 -- 172.21.15.8:0/3169889875 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5d44029700 con 0x7f5d501068c0 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d40038fe0 msgr2=0x7f5d4003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d40038fe0 0x7f5d4003b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c006fd0 tx=0x7f5d3c006e40 comp rx=0 tx=0).stop 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 msgr2=0x7f5d50198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50198120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5d44006010 tx=0x7f5d44004ff0 comp rx=0 tx=0).stop 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 shutdown_connections 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d40038fe0 0x7f5d4003b4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 --2- 172.21.15.8:0/3169889875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d501068c0 0x7f5d50198120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:13:58.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 >> 172.21.15.8:0/3169889875 conn(0x7f5d500fc3d0 msgr2=0x7f5d501083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:13:58.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 shutdown_connections 2024-02-19T16:13:58.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:13:58.290+0000 7f5d57159700 1 -- 172.21.15.8:0/3169889875 wait complete. 2024-02-19T16:13:58.292 INFO:teuthology.orchestra.run.smithi008.stderr:adjusted tunables profile to default 2024-02-19T16:13:58.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:58 smithi008 ceph-mon[32573]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:13:58.662 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:58 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3169889875' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-19T16:13:59.015 INFO:tasks.cephadm:Adding mon.a on smithi008 2024-02-19T16:13:59.015 INFO:tasks.cephadm:Adding mon.b on smithi156 2024-02-19T16:13:59.015 INFO:tasks.cephadm:Adding mon.c on smithi191 2024-02-19T16:13:59.015 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch apply mon '3;smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c' 2024-02-19T16:13:59.250 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:13:59.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3169889875' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-19T16:13:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:13:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:13:59.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:13:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:00.607 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 -- 172.21.15.191:0/1659427695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 msgr2=0x7f3f50106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:00.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 --2- 172.21.15.191:0/1659427695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50106ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009910 tx=0x7f3f40009c20 comp rx=0 tx=0).stop 2024-02-19T16:14:00.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 -- 172.21.15.191:0/1659427695 shutdown_connections 2024-02-19T16:14:00.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 --2- 172.21.15.191:0/1659427695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50106ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:00.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 -- 172.21.15.191:0/1659427695 >> 172.21.15.191:0/1659427695 conn(0x7f3f500fc3b0 msgr2=0x7f3f500fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:00.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 -- 172.21.15.191:0/1659427695 shutdown_connections 2024-02-19T16:14:00.608 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.606+0000 7f3f54c22700 1 -- 172.21.15.191:0/1659427695 wait complete. 2024-02-19T16:14:00.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.607+0000 7f3f54c22700 1 Processor -- start 2024-02-19T16:14:00.609 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.607+0000 7f3f54c22700 1 -- start start 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.607+0000 7f3f54c22700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.607+0000 7f3f54c22700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5006adf0 con 0x7f3f501068c0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f4e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f4e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.191:59604/0 (socket says 172.21.15.191:59604) 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f4e59c700 1 -- 172.21.15.191:0/1349963387 learned_addr learned my addr 172.21.15.191:0/1349963387 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f4e59c700 1 -- 172.21.15.191:0/1349963387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400095f0 con 0x7f3f501068c0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f4e59c700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50198110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f40006010 tx=0x7f3f40004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f4001d070 con 0x7f3f501068c0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.608+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f50198710 con 0x7f3f501068c0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.609+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f4000bc30 con 0x7f3f501068c0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.609+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f40003930 con 0x7f3f501068c0 2024-02-19T16:14:00.610 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.609+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f50198b90 con 0x7f3f501068c0 2024-02-19T16:14:00.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.609+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f50063610 con 0x7f3f501068c0 2024-02-19T16:14:00.611 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.609+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f3f40003b30 con 0x7f3f501068c0 2024-02-19T16:14:00.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.610+0000 7f3f3f7fe700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3f38038f80 0x7f3f3803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:00.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.610+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3f4004ce20 con 0x7f3f501068c0 2024-02-19T16:14:00.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.610+0000 7f3f4dd9b700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3f38038f80 0x7f3f3803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:00.612 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.610+0000 7f3f4dd9b700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3f38038f80 0x7f3f3803b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006fd0 tx=0x7f3f44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:00.615 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.615+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f3f4000ba70 con 0x7f3f501068c0 2024-02-19T16:14:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:00 smithi008 ceph-mon[32573]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:00 smithi008 ceph-mon[32573]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:14:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:00 smithi008 ceph-mon[32573]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:14:00.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.825+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f50198e80 con 0x7f3f38038f80 2024-02-19T16:14:00.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.832+0000 7f3f3f7fe700 1 -- 172.21.15.191:0/1349963387 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3f50198e80 con 0x7f3f38038f80 2024-02-19T16:14:00.832 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mon update... 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3f38038f80 msgr2=0x7f3f3803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3f38038f80 0x7f3f3803b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006fd0 tx=0x7f3f44006e40 comp rx=0 tx=0).stop 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 msgr2=0x7f3f50198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50198110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f40006010 tx=0x7f3f40004ff0 comp rx=0 tx=0).stop 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 shutdown_connections 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3f38038f80 0x7f3f3803b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 --2- 172.21.15.191:0/1349963387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f501068c0 0x7f3f50198110 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 >> 172.21.15.191:0/1349963387 conn(0x7f3f500fc3b0 msgr2=0x7f3f50108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:00.835 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 shutdown_connections 2024-02-19T16:14:00.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:00.834+0000 7f3f54c22700 1 -- 172.21.15.191:0/1349963387 wait complete. 2024-02-19T16:14:01.393 DEBUG:teuthology.orchestra.run.smithi156:mon.b> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.b.service 2024-02-19T16:14:01.395 DEBUG:teuthology.orchestra.run.smithi191:mon.c> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.c.service 2024-02-19T16:14:01.397 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-19T16:14:01.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T16:14:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: Saving service mon spec with placement smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c;count:3 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:01 smithi008 ceph-mon[32573]: Deploying daemon mon.c on smithi191 2024-02-19T16:14:02.533 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:03 smithi008 ceph-mon[32573]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:05.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 -- 172.21.15.191:0/3046333143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 msgr2=0x7f6d5c107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:05.280 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 --2- 172.21.15.191:0/3046333143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c107d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009910 tx=0x7f6d50009c20 comp rx=0 tx=0).stop 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 -- 172.21.15.191:0/3046333143 shutdown_connections 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 --2- 172.21.15.191:0/3046333143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c107d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 -- 172.21.15.191:0/3046333143 >> 172.21.15.191:0/3046333143 conn(0x7f6d5c0fd4a0 msgr2=0x7f6d5c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 -- 172.21.15.191:0/3046333143 shutdown_connections 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.277+0000 7f6d63124700 1 -- 172.21.15.191:0/3046333143 wait complete. 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d63124700 1 Processor -- start 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d63124700 1 -- start start 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d63124700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d63124700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c197810 con 0x7f6d5c1079b0 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d60ec0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d60ec0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.191:48572/0 (socket says 172.21.15.191:48572) 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.278+0000 7f6d60ec0700 1 -- 172.21.15.191:0/3738094672 learned_addr learned my addr 172.21.15.191:0/3738094672 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:14:05.281 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d60ec0700 1 -- 172.21.15.191:0/3738094672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500095f0 con 0x7f6d5c1079b0 2024-02-19T16:14:05.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d60ec0700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c1972d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d50004f40 tx=0x7f6d50005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:05.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d5001d070 con 0x7f6d5c1079b0 2024-02-19T16:14:05.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c197a70 con 0x7f6d5c1079b0 2024-02-19T16:14:05.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d5000bbd0 con 0x7f6d5c1079b0 2024-02-19T16:14:05.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d50003900 con 0x7f6d5c1079b0 2024-02-19T16:14:05.282 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.279+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c197ef0 con 0x7f6d5c1079b0 2024-02-19T16:14:05.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.280+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f6d50003bc0 con 0x7f6d5c1079b0 2024-02-19T16:14:05.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.280+0000 7f6d59ffb700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6d4c038f80 0x7f6d4c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:05.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.280+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d5004ce50 con 0x7f6d5c1079b0 2024-02-19T16:14:05.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.280+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c063610 con 0x7f6d5c1079b0 2024-02-19T16:14:05.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.281+0000 7f6d5bfff700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6d4c038f80 0x7f6d4c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:05.283 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.281+0000 7f6d5bfff700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6d4c038f80 0x7f6d4c03b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006fd0 tx=0x7f6d48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:05.287 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.286+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f6d50026080 con 0x7f6d5c1079b0 2024-02-19T16:14:05.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.572+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d5c10bdf0 con 0x7f6d5c1079b0 2024-02-19T16:14:05.574 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.573+0000 7f6d59ffb700 1 -- 172.21.15.191:0/3738094672 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+741 (secure 0 0 0) 0x7f6d5002b440 con 0x7f6d5c1079b0 2024-02-19T16:14:05.574 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:14:05.574 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","modified":"2024-02-19T16:12:13.047065Z","created":"2024-02-19T16:12:13.047065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T16:14:05.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6d4c038f80 msgr2=0x7f6d4c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:05.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6d4c038f80 0x7f6d4c03b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006fd0 tx=0x7f6d48006e40 comp rx=0 tx=0).stop 2024-02-19T16:14:05.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 msgr2=0x7f6d5c1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:05.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c1972d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d50004f40 tx=0x7f6d50005020 comp rx=0 tx=0).stop 2024-02-19T16:14:05.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 shutdown_connections 2024-02-19T16:14:05.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6d4c038f80 0x7f6d4c03b440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:05.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 --2- 172.21.15.191:0/3738094672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d5c1079b0 0x7f6d5c1972d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:05.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 >> 172.21.15.191:0/3738094672 conn(0x7f6d5c0fd4a0 msgr2=0x7f6d5c1094b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:05.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 shutdown_connections 2024-02-19T16:14:05.577 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:05.575+0000 7f6d63124700 1 -- 172.21.15.191:0/3738094672 wait complete. 2024-02-19T16:14:05.590 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-19T16:14:05.697 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 podman[37171]: 2024-02-19 16:14:05.673848536 +0000 UTC m=+0.349193681 container start 29df719b1c8917a7f9df89dbc077fba793c9f61031045d641a1479b461999ff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T16:14:05.697 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: set uid:gid to 167:167 (ceph:ceph) 2024-02-19T16:14:05.697 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev), process ceph-mon, pid 2 2024-02-19T16:14:05.697 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pidfile_write: ignore empty --pid-file 2024-02-19T16:14:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: load: jerasure load: lrc 2024-02-19T16:14:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: RocksDB version: 7.9.2 2024-02-19T16:14:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Git sha 0 2024-02-19T16:14:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Compile date 2024-02-15 13:42:22 2024-02-19T16:14:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: DB SUMMARY 2024-02-19T16:14:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: DB Session ID: GZC1EBJNX8YENQS7NJKY 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: CURRENT file: CURRENT 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: IDENTITY file: IDENTITY 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.error_if_exists: 0 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.create_if_missing: 0 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.paranoid_checks: 1 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-19T16:14:06.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.env: 0x555faae69ae0 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.fs: PosixFileSystem 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.info_log: 0x555fac955900 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_file_opening_threads: 16 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.statistics: (nil) 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.use_fsync: 0 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_log_file_size: 0 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.keep_log_file_num: 1000 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.recycle_log_file_num: 0 2024-02-19T16:14:06.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.allow_fallocate: 1 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.allow_mmap_reads: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.allow_mmap_writes: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.use_direct_reads: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.create_missing_column_families: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.db_log_dir: 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.wal_dir: 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-19T16:14:06.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.advise_random_on_open: 1 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.db_write_buffer_size: 0 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.write_buffer_manager: 0x555fac929400 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.rate_limiter: (nil) 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.wal_recovery_mode: 2 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.enable_thread_tracking: 0 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.enable_pipelined_write: 0 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.unordered_write: 0 2024-02-19T16:14:06.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.row_cache: None 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.wal_filter: None 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.allow_ingest_behind: 0 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.two_write_queues: 0 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.manual_wal_flush: 0 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.wal_compression: 0 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.atomic_flush: 0 2024-02-19T16:14:06.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.log_readahead_size: 0 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.best_efforts_recovery: 0 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.allow_data_in_errors: 0 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.db_host_id: __hostname__ 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_background_jobs: 2 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_background_compactions: -1 2024-02-19T16:14:06.033 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_subcompactions: 1 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_total_wal_size: 0 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-19T16:14:06.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_open_files: -1 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bytes_per_sync: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_readahead_size: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_background_flushes: -1 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Compression algorithms supported: 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kZSTD supported: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kXpressCompression supported: 0 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kLZ4HCCompression supported: 1 2024-02-19T16:14:06.036 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kZlibCompression supported: 1 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kSnappyCompression supported: 1 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kLZ4Compression supported: 1 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: kBZip2Compression supported: 0 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.merge_operator: 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_filter: None 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_filter_factory: None 2024-02-19T16:14:06.037 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.sst_partitioner_factory: None 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555fac955a00) 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: cache_index_and_filter_blocks: 1 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: pin_top_level_index_and_filter: 1 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: index_type: 0 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: data_block_index_type: 0 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: index_shortening: 1 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: checksum: 4 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: no_block_cache: 0 2024-02-19T16:14:06.038 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_cache: 0x555fac959090 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_cache_name: BinnedLRUCache 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_cache_options: 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: capacity : 536870912 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: num_shard_bits : 4 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: strict_capacity_limit : 0 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: high_pri_pool_ratio: 0.000 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_cache_compressed: (nil) 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: persistent_cache: (nil) 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_size: 4096 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_size_deviation: 10 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_restart_interval: 16 2024-02-19T16:14:06.039 INFO:journalctl@ceph.mon.c.smithi191.stdout: index_block_restart_interval: 1 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: metadata_block_size: 4096 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: partition_filters: 0 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: use_delta_encoding: 1 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: filter_policy: bloomfilter 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: whole_key_filtering: 1 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: verify_compression: 0 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: read_amp_bytes_per_bit: 0 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: format_version: 5 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: enable_index_compression: 1 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: block_align: 0 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: max_auto_readahead_size: 262144 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: prepopulate_block_cache: 0 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: initial_auto_readahead_size: 8192 2024-02-19T16:14:06.040 INFO:journalctl@ceph.mon.c.smithi191.stdout: num_file_reads_for_auto_readahead: 2 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.write_buffer_size: 33554432 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_write_buffer_number: 2 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression: NoCompression 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression: Disabled 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.prefix_extractor: nullptr 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.num_levels: 7 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-19T16:14:06.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.level: 32767 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.strategy: 0 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.enabled: false 2024-02-19T16:14:06.042 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.target_file_size_base: 67108864 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-19T16:14:06.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.arena_block_size: 1048576 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.disable_auto_compactions: 0 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-19T16:14:06.044 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.table_properties_collectors: 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.inplace_update_support: 0 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-19T16:14:06.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.bloom_locality: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.max_successive_merges: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.paranoid_file_checks: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.force_consistency_checks: 1 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.report_bg_io_stats: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.ttl: 2592000 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-19T16:14:06.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.enable_blob_files: false 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.min_blob_size: 0 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.blob_file_size: 268435456 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.blob_file_starting_level: 0 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b65c797a-62ff-4763-9bd0-39a0e36215e5 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708359245703390, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-19T16:14:06.047 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708359245703959, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708359245, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b65c797a-62ff-4763-9bd0-39a0e36215e5", "db_session_id": "GZC1EBJNX8YENQS7NJKY", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708359245704072, "job": 1, "event": "recovery_finished"} 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555faca4a000 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: DB pointer 0x555faca34000 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: ** DB Stats ** 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-19T16:14:06.048 INFO:journalctl@ceph.mon.c.smithi191.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: ** Compaction Stats [default] ** 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: ** Compaction Stats [default] ** 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-19T16:14:06.049 INFO:journalctl@ceph.mon.c.smithi191.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Block cache BinnedLRUCache@0x555fac959090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-19T16:14:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: using public_addr v2:172.21.15.191:0/0 -> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: starting mon.c rank -1 at public addrs [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] at bind addrs [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(???) e0 preinit fsid 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).mds e1 new map 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout: e1 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout: legacy client fscid: -1 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout: No filesystems configured 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-19T16:14:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Deploying cephadm binary to smithi191 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Added host smithi191 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3169889875' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3169889875' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:06.055 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Saving service mon spec with placement smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c;count:3 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: Deploying daemon mon.c on smithi191 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:06.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:05 smithi191 ceph-mon[37203]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-19T16:14:06.360 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:06 smithi191 bash[37171]: 29df719b1c8917a7f9df89dbc077fba793c9f61031045d641a1479b461999ff4 2024-02-19T16:14:06.360 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:06 smithi191 systemd[1]: Started Ceph mon.c for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:14:07.520 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-19T16:14:07.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T16:14:07.766 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: mon.a calling monitor election 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: mon.c calling monitor election 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: monmap e2: 2 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: fsmap 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: mgrmap e13: x(active, since 50s) 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: overall HEALTH_OK 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:11.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: mon.a calling monitor election 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: mon.c calling monitor election 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: monmap e2: 2 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: fsmap 2024-02-19T16:14:11.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:11.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: mgrmap e13: x(active, since 50s) 2024-02-19T16:14:11.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: overall HEALTH_OK 2024-02-19T16:14:11.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:11.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:12.127 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:11 smithi191 ceph-mon[37203]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:12.127 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:12.127 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:12.128 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:12.128 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:11 smithi191 ceph-mon[37203]: Deploying daemon mon.b on smithi156 2024-02-19T16:14:12.128 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:12.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 -- 172.21.15.191:0/1121729682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2e00035c0 msgr2=0x7fe2e0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:12.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 --2- 172.21.15.191:0/1121729682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2e00035c0 0x7fe2e0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0069a10 tx=0x7fe2e400feb0 comp rx=0 tx=0).stop 2024-02-19T16:14:12.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 -- 172.21.15.191:0/1121729682 shutdown_connections 2024-02-19T16:14:12.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 --2- 172.21.15.191:0/1121729682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2e00035c0 0x7fe2e0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:12.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 -- 172.21.15.191:0/1121729682 >> 172.21.15.191:0/1121729682 conn(0x7fe2f00fc490 msgr2=0x7fe2f00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:12.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 -- 172.21.15.191:0/1121729682 shutdown_connections 2024-02-19T16:14:12.136 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.134+0000 7fe2f7124700 1 -- 172.21.15.191:0/1121729682 wait complete. 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f7124700 1 Processor -- start 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f7124700 1 -- start start 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f7124700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f7124700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2f01a3c80 0x7fe2f019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f7124700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f01a4210 con 0x7fe2f01a3c80 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f7124700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f01a4380 con 0x7fe2e00035c0 2024-02-19T16:14:12.139 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f4ec0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f4ec0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43038/0 (socket says 172.21.15.191:43038) 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2f4ec0700 1 -- 172.21.15.191:0/488099299 learned_addr learned my addr 172.21.15.191:0/488099299 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.135+0000 7fe2effff700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2f01a3c80 0x7fe2f019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2f4ec0700 1 -- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 msgr2=0x7fe2f01a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2f4ec0700 1 -- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 msgr2=0x7fe2f01a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2f4ec0700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2f4ec0700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2effff700 1 -- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 msgr2=0x7fe2f01a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2effff700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:12.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2effff700 1 -- 172.21.15.191:0/488099299 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e400f900 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2effff700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2f01a3c80 0x7fe2f019d9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00b4d0 tx=0x7fe2dc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe2dc011630 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc011c70 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f019dfa0 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe2dc010de0 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.136+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f019e4f0 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.137+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7fe2dc01a400 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.137+0000 7fe2edffb700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2d4038d20 0x7fe2d403b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.137+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe2dc04b750 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.137+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f0197940 con 0x7fe2f01a3c80 2024-02-19T16:14:12.141 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.138+0000 7fe2f4ec0700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2d4038d20 0x7fe2d403b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:12.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.138+0000 7fe2f4ec0700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2d4038d20 0x7fe2d403b1e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400f800 tx=0x7fe2e4005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:12.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.143+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe2dc0193b0 con 0x7fe2f01a3c80 2024-02-19T16:14:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:11 smithi008 ceph-mon[32573]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:11 smithi008 ceph-mon[32573]: Deploying daemon mon.b on smithi156 2024-02-19T16:14:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:12.417 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.416+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe2f019ee30 con 0x7fe2f01a3c80 2024-02-19T16:14:12.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.417+0000 7fe2edffb700 1 -- 172.21.15.191:0/488099299 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7fe2dc0172c0 con 0x7fe2f01a3c80 2024-02-19T16:14:12.418 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:14:12.418 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","modified":"2024-02-19T16:14:05.728694Z","created":"2024-02-19T16:12:13.047065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2d4038d20 msgr2=0x7fe2d403b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2d4038d20 0x7fe2d403b1e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400f800 tx=0x7fe2e4005c20 comp rx=0 tx=0).stop 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2f01a3c80 msgr2=0x7fe2f019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2f01a3c80 0x7fe2f019d9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00b4d0 tx=0x7fe2dc00b890 comp rx=0 tx=0).stop 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 shutdown_connections 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2d4038d20 0x7fe2d403b1e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00035c0 0x7fe2f01a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 --2- 172.21.15.191:0/488099299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2f01a3c80 0x7fe2f019d9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:12.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 >> 172.21.15.191:0/488099299 conn(0x7fe2f00fc490 msgr2=0x7fe2f00fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:12.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 shutdown_connections 2024-02-19T16:14:12.422 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:12.420+0000 7fe2f7124700 1 -- 172.21.15.191:0/488099299 wait complete. 2024-02-19T16:14:12.422 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-02-19T16:14:13.016 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:12 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/488099299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T16:14:13.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:12 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/488099299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T16:14:14.043 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-19T16:14:14.044 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T16:14:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:13 smithi008 ceph-mon[32573]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:13 smithi191 ceph-mon[37203]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:14.288 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:14:15.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:14 smithi156 systemd[1]: Started Ceph mon.b for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:14:20.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 -- 172.21.15.191:0/2897379834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e140035c0 msgr2=0x7f1e14005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:20.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 --2- 172.21.15.191:0/2897379834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e140035c0 0x7f1e14005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c1008e0 tx=0x7f1e24009e80 comp rx=0 tx=0).stop 2024-02-19T16:14:20.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 -- 172.21.15.191:0/2897379834 shutdown_connections 2024-02-19T16:14:20.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 --2- 172.21.15.191:0/2897379834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e140035c0 0x7f1e14005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 -- 172.21.15.191:0/2897379834 >> 172.21.15.191:0/2897379834 conn(0x7f1e2c0fc480 msgr2=0x7f1e2c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:20.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 -- 172.21.15.191:0/2897379834 shutdown_connections 2024-02-19T16:14:20.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.206+0000 7f1e34364700 1 -- 172.21.15.191:0/2897379834 wait complete. 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: mon.c calling monitor election 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: mon.a calling monitor election 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:20.209 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: fsmap 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:20.210 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: mgrmap e13: x(active, since 59s) 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.211 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 Processor -- start 2024-02-19T16:14:20.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 -- start start 2024-02-19T16:14:20.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:20.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e2c196920 0x7f1e2c19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 0x7f1e2c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c197010 con 0x7f1e2c196920 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1a2f60 con 0x7f1e2c19b1f0 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e34364700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c1a30a0 con 0x7f1e140035c0 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:50344/0 (socket says 172.21.15.191:50344) 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 -- 172.21.15.191:0/71875928 learned_addr learned my addr 172.21.15.191:0/71875928 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 msgr2=0x7f1e2c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 msgr2=0x7f1e2c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32100700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e318ff700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e2c196920 0x7f1e2c19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:20.215 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.207+0000 7f1e32901700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 0x7f1e2c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e32901700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 msgr2=0x7f1e2c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e32901700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 msgr2=0x7f1e2c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e32901700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 0x7f1e2c19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e32901700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 0x7f1e2c19b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e318ff700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 msgr2=0x7f1e2c1963e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e318ff700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e318ff700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 msgr2=0x7f1e2c19b6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e318ff700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 0x7f1e2c19b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e318ff700 1 -- 172.21.15.191:0/71875928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240098d0 con 0x7f1e2c196920 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e318ff700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e2c196920 0x7f1e2c19acb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b550 tx=0x7f1e2000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20004230 con 0x7f1e2c196920 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e20004390 con 0x7f1e2c196920 2024-02-19T16:14:20.216 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20011b90 con 0x7f1e2c196920 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c1a3380 con 0x7f1e2c196920 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.208+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c1a3830 con 0x7f1e2c196920 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.209+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c063610 con 0x7f1e2c196920 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.209+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f1e20011cf0 con 0x7f1e2c196920 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.210+0000 7f1e2b7fe700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1e18039410 0x7f1e1803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.210+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1e2004b5d0 con 0x7f1e2c196920 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.210+0000 7f1e32100700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1e18039410 0x7f1e1803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.210+0000 7f1e32100700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1e18039410 0x7f1e1803b8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e240098a0 tx=0x7f1e2401c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:20.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.215+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1e20004500 con 0x7f1e2c196920 2024-02-19T16:14:20.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: mon.c calling monitor election 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: mon.a calling monitor election 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: fsmap 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: mgrmap e13: x(active, since 59s) 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.514+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1e2c108860 con 0x7f1e2c196920 2024-02-19T16:14:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.514+0000 7f1e2b7fe700 1 -- 172.21.15.191:0/71875928 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f1e200151c0 con 0x7f1e2c196920 2024-02-19T16:14:20.515 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-19T16:14:20.516 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":3,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","modified":"2024-02-19T16:14:14.722951Z","created":"2024-02-19T16:12:13.047065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1e18039410 msgr2=0x7f1e1803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1e18039410 0x7f1e1803b8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e240098a0 tx=0x7f1e2401c040 comp rx=0 tx=0).stop 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e2c196920 msgr2=0x7f1e2c19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e2c196920 0x7f1e2c19acb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b550 tx=0x7f1e2000b910 comp rx=0 tx=0).stop 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 shutdown_connections 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1e18039410 0x7f1e1803b8d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e2c19b1f0 0x7f1e2c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e140035c0 0x7f1e2c1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 --2- 172.21.15.191:0/71875928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e2c196920 0x7f1e2c19acb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 >> 172.21.15.191:0/71875928 conn(0x7f1e2c0fc480 msgr2=0x7f1e2c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 shutdown_connections 2024-02-19T16:14:20.518 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:20.517+0000 7f1e34364700 1 -- 172.21.15.191:0/71875928 wait complete. 2024-02-19T16:14:20.519 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 3 2024-02-19T16:14:21.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:21.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:20 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/71875928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T16:14:21.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:21.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:21.102 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-19T16:14:21.102 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph config generate-minimal-conf 2024-02-19T16:14:21.132 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:21.132 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:20 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/71875928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T16:14:21.132 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:21.132 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:21.355 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:14:22.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:22.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:14:22.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: mon.c calling monitor election 2024-02-19T16:14:22.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: mon.a calling monitor election 2024-02-19T16:14:22.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:14:22.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:22.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: fsmap 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: mgrmap e13: x(active, since 59s) 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-19T16:14:22.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/71875928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.086 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: mon.b calling monitor election 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: mon.b calling monitor election 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: mon.c calling monitor election 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: mon.a calling monitor election 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: fsmap 2024-02-19T16:14:22.739 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: mgrmap e13: x(active, since 61s) 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Cluster is now healthy 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: overall HEALTH_OK 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Reconfiguring mon.a (unknown last config time)... 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:22.741 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:22 smithi008 ceph-mon[32573]: Reconfiguring daemon mon.a on smithi008 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: mon.b calling monitor election 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: mon.b calling monitor election 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: mon.c calling monitor election 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: mon.a calling monitor election 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: fsmap 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: mgrmap e13: x(active, since 61s) 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Cluster is now healthy 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: overall HEALTH_OK 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Reconfiguring mon.a (unknown last config time)... 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:23.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:22 smithi191 ceph-mon[37203]: Reconfiguring daemon mon.a on smithi008 2024-02-19T16:14:23.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 -- 172.21.15.8:0/1955007045 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35001069f0 msgr2=0x7f3500106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:23.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1955007045 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35001069f0 0x7f3500106dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009910 tx=0x7f34f0009c20 comp rx=0 tx=0).stop 2024-02-19T16:14:23.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 -- 172.21.15.8:0/1955007045 shutdown_connections 2024-02-19T16:14:23.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1955007045 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35001069f0 0x7f3500106dd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 -- 172.21.15.8:0/1955007045 >> 172.21.15.8:0/1955007045 conn(0x7f35000fbb30 msgr2=0x7f35000fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:23.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 -- 172.21.15.8:0/1955007045 shutdown_connections 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.066+0000 7f3505cbe700 1 -- 172.21.15.8:0/1955007045 wait complete. 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 Processor -- start 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 -- start start 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35001069f0 0x7f3500197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500197ad0 0x7f350019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 0x7f350019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35001981c0 con 0x7f350019c3a0 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35001a4110 con 0x7f3500197ad0 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f3505cbe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35001a4250 con 0x7f35001069f0 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f34fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 0x7f350019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f34fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 0x7f350019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41496/0 (socket says 172.21.15.8:41496) 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.067+0000 7f34fffff700 1 -- 172.21.15.8:0/1807209646 learned_addr learned my addr 172.21.15.8:0/1807209646 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:14:23.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34ff7fe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35001069f0 0x7f3500197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34feffd700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500197ad0 0x7f350019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fffff700 1 -- 172.21.15.8:0/1807209646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35001069f0 msgr2=0x7f3500197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fffff700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35001069f0 0x7f3500197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fffff700 1 -- 172.21.15.8:0/1807209646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500197ad0 msgr2=0x7f350019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fffff700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500197ad0 0x7f350019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fffff700 1 -- 172.21.15.8:0/1807209646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00095f0 con 0x7f350019c3a0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34ff7fe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35001069f0 0x7f3500197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fffff700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 0x7f350019c850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ba50 tx=0x7f34f800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800cab0 con 0x7f350019c3a0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34f800cc10 con 0x7f350019c3a0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35001a4530 con 0x7f350019c3a0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80075d0 con 0x7f350019c3a0 2024-02-19T16:14:23.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.068+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35001a49e0 con 0x7f350019c3a0 2024-02-19T16:14:23.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.069+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f34f80077b0 con 0x7f350019c3a0 2024-02-19T16:14:23.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.069+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3500063610 con 0x7f350019c3a0 2024-02-19T16:14:23.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.069+0000 7f34fcff9700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e8039410 0x7f34e803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:23.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.069+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34f804b740 con 0x7f350019c3a0 2024-02-19T16:14:23.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.069+0000 7f34ff7fe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e8039410 0x7f34e803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:23.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.070+0000 7f34ff7fe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e8039410 0x7f34e803b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f34f0004690 tx=0x7f34f000b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:23.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.074+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f34f801b8b0 con 0x7f350019c3a0 2024-02-19T16:14:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: mon.b calling monitor election 2024-02-19T16:14:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:14:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:14:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: mon.b calling monitor election 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: mon.c calling monitor election 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: mon.a calling monitor election 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: fsmap 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: mgrmap e13: x(active, since 61s) 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Cluster is now healthy 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: overall HEALTH_OK 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Reconfiguring mon.a (unknown last config time)... 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:23.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:23.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:22 smithi156 ceph-mon[36502]: Reconfiguring daemon mon.a on smithi008 2024-02-19T16:14:23.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.279+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f350010bef0 con 0x7f350019c3a0 2024-02-19T16:14:23.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.279+0000 7f34fcff9700 1 -- 172.21.15.8:0/1807209646 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f34f8019070 con 0x7f350019c3a0 2024-02-19T16:14:23.281 INFO:teuthology.orchestra.run.smithi008.stdout:# minimal ceph.conf for 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:14:23.281 INFO:teuthology.orchestra.run.smithi008.stdout:[global] 2024-02-19T16:14:23.281 INFO:teuthology.orchestra.run.smithi008.stdout: fsid = 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:14:23.282 INFO:teuthology.orchestra.run.smithi008.stdout: mon_host = [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-02-19T16:14:23.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e8039410 msgr2=0x7f34e803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:23.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e8039410 0x7f34e803b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f34f0004690 tx=0x7f34f000b560 comp rx=0 tx=0).stop 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 msgr2=0x7f350019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 0x7f350019c850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ba50 tx=0x7f34f800be10 comp rx=0 tx=0).stop 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 shutdown_connections 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e8039410 0x7f34e803b8d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f350019c3a0 0x7f350019c850 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35001069f0 0x7f3500197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 --2- 172.21.15.8:0/1807209646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3500197ad0 0x7f350019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.282+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 >> 172.21.15.8:0/1807209646 conn(0x7f35000fbb30 msgr2=0x7f3500103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.283+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 shutdown_connections 2024-02-19T16:14:23.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:23.283+0000 7f3505cbe700 1 -- 172.21.15.8:0/1807209646 wait complete. 2024-02-19T16:14:23.888 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:23 smithi008 ceph-mon[32573]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:23.888 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:23.888 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:23 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1807209646' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:23.915 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-19T16:14:23.915 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:14:23.915 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.conf 2024-02-19T16:14:23.987 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:14:23.988 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:14:24.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:23 smithi191 ceph-mon[37203]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:24.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:23 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:23 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1807209646' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:24.059 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:14:24.059 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2024-02-19T16:14:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:23 smithi156 ceph-mon[36502]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:23 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:14:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:23 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1807209646' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:24.095 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:14:24.095 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:14:24.165 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:14:24.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-02-19T16:14:24.201 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:14:24.201 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:14:24.270 INFO:tasks.cephadm:Adding mgr.x on smithi008 2024-02-19T16:14:24.270 INFO:tasks.cephadm:Adding mgr.y on smithi156 2024-02-19T16:14:24.271 INFO:tasks.cephadm:Adding mgr.z on smithi191 2024-02-19T16:14:24.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch apply mgr '3;smithi008=x;smithi156=y;smithi191=z' 2024-02-19T16:14:24.542 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:14:26.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: Reconfiguring mon.b (monmap changed)... 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:25 smithi191 ceph-mon[37203]: Reconfiguring daemon mon.b on smithi156 2024-02-19T16:14:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: Reconfiguring mon.b (monmap changed)... 2024-02-19T16:14:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:26.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:26.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:26.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:25 smithi156 ceph-mon[36502]: Reconfiguring daemon mon.b on smithi156 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 -- 172.21.15.191:0/2055415641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 msgr2=0x7f6098104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 --2- 172.21.15.191:0/2055415641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 0x7f6098104a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6094009910 tx=0x7f6094009c20 comp rx=0 tx=0).stop 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 -- 172.21.15.191:0/2055415641 shutdown_connections 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 --2- 172.21.15.191:0/2055415641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 0x7f6098104a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 -- 172.21.15.191:0/2055415641 >> 172.21.15.191:0/2055415641 conn(0x7f60980fbfc0 msgr2=0x7f60980fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 -- 172.21.15.191:0/2055415641 shutdown_connections 2024-02-19T16:14:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.084+0000 7f60a0422700 1 -- 172.21.15.191:0/2055415641 wait complete. 2024-02-19T16:14:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 Processor -- start 2024-02-19T16:14:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 -- start start 2024-02-19T16:14:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 0x7f6098197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 0x7f609819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6098197ae0 0x7f609819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609819f400 con 0x7f6098102630 2024-02-19T16:14:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609819f570 con 0x7f6098197ae0 2024-02-19T16:14:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f60a0422700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609819f730 con 0x7f6098197660 2024-02-19T16:14:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 0x7f609819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 0x7f609819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45620/0 (socket says 172.21.15.191:45620) 2024-02-19T16:14:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 -- 172.21.15.191:0/324925248 learned_addr learned my addr 172.21.15.191:0/324925248 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:14:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609e9bf700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6098197ae0 0x7f609819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609e1be700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 0x7f6098197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 -- 172.21.15.191:0/324925248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6098197ae0 msgr2=0x7f609819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6098197ae0 0x7f609819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 -- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 msgr2=0x7f6098197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 0x7f6098197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.085+0000 7f609d9bd700 1 -- 172.21.15.191:0/324925248 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940095f0 con 0x7f6098197660 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.086+0000 7f609d9bd700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 0x7f609819c6d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f608800b780 tx=0x7f608800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.086+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088004230 con 0x7f6098197660 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.086+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609819fa10 con 0x7f6098197660 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.086+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6088004390 con 0x7f6098197660 2024-02-19T16:14:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.086+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088011c00 con 0x7f6098197660 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.086+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609819ffe0 con 0x7f6098197660 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.087+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7f60880036a0 con 0x7f6098197660 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.087+0000 7f608f7fe700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6084039410 0x7f608403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.087+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60981909a0 con 0x7f6098197660 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.087+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f608804a8d0 con 0x7f6098197660 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.087+0000 7f609e1be700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6084039410 0x7f608403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.088+0000 7f609e1be700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6084039410 0x7f608403b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6094005230 tx=0x7f6094005300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:26.094 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.092+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f608801a3b0 con 0x7f6098197660 2024-02-19T16:14:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: Reconfiguring mon.b (monmap changed)... 2024-02-19T16:14:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:26.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:26.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:25 smithi008 ceph-mon[32573]: Reconfiguring daemon mon.b on smithi156 2024-02-19T16:14:26.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.308+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f609802da30 con 0x7f6084039410 2024-02-19T16:14:26.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.314+0000 7f608f7fe700 1 -- 172.21.15.191:0/324925248 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f609802da30 con 0x7f6084039410 2024-02-19T16:14:26.315 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mgr update... 2024-02-19T16:14:26.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6084039410 msgr2=0x7f608403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:26.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6084039410 0x7f608403b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6094005230 tx=0x7f6094005300 comp rx=0 tx=0).stop 2024-02-19T16:14:26.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 msgr2=0x7f609819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 0x7f609819c6d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f608800b780 tx=0x7f608800ba90 comp rx=0 tx=0).stop 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 shutdown_connections 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6084039410 0x7f608403b8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6098197ae0 0x7f609819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6098102630 0x7f6098197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 --2- 172.21.15.191:0/324925248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6098197660 0x7f609819c6d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 >> 172.21.15.191:0/324925248 conn(0x7f60980fbfc0 msgr2=0x7f60980fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 shutdown_connections 2024-02-19T16:14:26.318 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:14:26.316+0000 7f60a0422700 1 -- 172.21.15.191:0/324925248 wait complete. 2024-02-19T16:14:26.905 DEBUG:teuthology.orchestra.run.smithi156:mgr.y> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.y.service 2024-02-19T16:14:26.908 DEBUG:teuthology.orchestra.run.smithi191:mgr.z> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.z.service 2024-02-19T16:14:26.911 INFO:tasks.cephadm:Deploying OSDs... 2024-02-19T16:14:26.911 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:14:26.912 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/scratch_devs of=/dev/stdout 2024-02-19T16:14:26.932 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-19T16:14:26.932 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_1 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-02-19 16:13:30.820472181 +0000 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-02-19 16:03:51.395659327 +0000 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-02-19 16:03:51.395659327 +0000 2024-02-19T16:14:26.991 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2024-02-19 16:03:51.395659327 +0000 2024-02-19T16:14:26.992 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-19T16:14:27.062 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-02-19T16:14:27.062 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-02-19T16:14:27.063 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000126361 s, 4.1 MB/s 2024-02-19T16:14:27.064 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-19T16:14:27.122 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_2 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-02-19 16:13:30.856471803 +0000 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-02-19 16:03:51.735655884 +0000 2024-02-19T16:14:27.179 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-02-19 16:03:51.735655884 +0000 2024-02-19T16:14:27.180 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2024-02-19 16:03:51.735655884 +0000 2024-02-19T16:14:27.180 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-19T16:14:27.248 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-02-19T16:14:27.248 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-02-19T16:14:27.248 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000117572 s, 4.4 MB/s 2024-02-19T16:14:27.249 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-19T16:14:27.306 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_3 2024-02-19T16:14:27.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: Reconfiguring mon.c (monmap changed)... 2024-02-19T16:14:27.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:27.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:27.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:27.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: Reconfiguring daemon mon.c on smithi191 2024-02-19T16:14:27.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:27.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: Saving service mgr spec with placement smithi008=x;smithi156=y;smithi191=z;count:3 2024-02-19T16:14:27.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:27 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.365 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-19T16:14:27.365 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:27.365 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-19T16:14:27.365 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:27.366 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:27.366 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-02-19 16:13:30.893471414 +0000 2024-02-19T16:14:27.366 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-02-19 16:03:52.068652512 +0000 2024-02-19T16:14:27.367 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-02-19 16:03:52.068652512 +0000 2024-02-19T16:14:27.367 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2024-02-19 16:03:52.068652512 +0000 2024-02-19T16:14:27.367 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: Reconfiguring mon.c (monmap changed)... 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:27.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: Reconfiguring daemon mon.c on smithi191 2024-02-19T16:14:27.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:27.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: Saving service mgr spec with placement smithi008=x;smithi156=y;smithi191=z;count:3 2024-02-19T16:14:27.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:27 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: Reconfiguring mon.c (monmap changed)... 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: Reconfiguring daemon mon.c on smithi191 2024-02-19T16:14:27.429 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:27.430 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: Saving service mgr spec with placement smithi008=x;smithi156=y;smithi191=z;count:3 2024-02-19T16:14:27.430 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:27 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:27.431 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-02-19T16:14:27.431 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-02-19T16:14:27.431 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000107961 s, 4.7 MB/s 2024-02-19T16:14:27.433 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-19T16:14:27.489 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_4 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-02-19 16:13:30.928471047 +0000 2024-02-19T16:14:27.545 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-02-19 16:03:52.393649221 +0000 2024-02-19T16:14:27.546 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-02-19 16:03:52.393649221 +0000 2024-02-19T16:14:27.546 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2024-02-19 16:03:52.393649221 +0000 2024-02-19T16:14:27.546 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-19T16:14:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-02-19T16:14:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-02-19T16:14:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000110255 s, 4.6 MB/s 2024-02-19T16:14:27.613 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-19T16:14:27.670 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-19T16:14:27.670 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-02-19T16:14:27.689 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-19T16:14:27.689 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-02-19T16:14:27.747 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-19T16:14:27.747 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:27.747 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-19T16:14:27.747 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:27.747 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:27.748 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-19 16:14:14.192146955 +0000 2024-02-19T16:14:27.748 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-19 16:04:20.620538346 +0000 2024-02-19T16:14:27.748 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-19 16:04:20.620538346 +0000 2024-02-19T16:14:27.748 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-19 16:04:20.620538346 +0000 2024-02-19T16:14:27.748 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-19T16:14:27.816 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-19T16:14:27.816 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-19T16:14:27.816 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00013 s, 3.9 MB/s 2024-02-19T16:14:27.817 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-19T16:14:27.875 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-19 16:14:14.192146955 +0000 2024-02-19T16:14:27.932 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-19 16:04:20.952532888 +0000 2024-02-19T16:14:27.933 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-19 16:04:20.952532888 +0000 2024-02-19T16:14:27.933 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-19 16:04:20.952532888 +0000 2024-02-19T16:14:27.933 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-19T16:14:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-19T16:14:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-19T16:14:28.000 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.0001284 s, 4.0 MB/s 2024-02-19T16:14:28.001 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-19T16:14:28.059 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-02-19T16:14:28.115 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-19T16:14:28.115 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:28.115 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-02-19T16:14:28.116 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:28.116 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:28.116 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-19 16:14:14.193146939 +0000 2024-02-19T16:14:28.116 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-19 16:04:21.277527545 +0000 2024-02-19T16:14:28.116 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-19 16:04:21.277527545 +0000 2024-02-19T16:14:28.116 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-19 16:04:21.277527545 +0000 2024-02-19T16:14:28.116 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-19T16:14:28.183 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-19T16:14:28.183 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-19T16:14:28.183 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 9.7733e-05 s, 5.2 MB/s 2024-02-19T16:14:28.184 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-19T16:14:28.240 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-02-19T16:14:28.295 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-19T16:14:28.295 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-19 16:14:14.193146939 +0000 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-19 16:04:21.598522268 +0000 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-19 16:04:21.598522268 +0000 2024-02-19T16:14:28.296 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-19 16:04:21.598522268 +0000 2024-02-19T16:14:28.296 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-19T16:14:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-19T16:14:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-19T16:14:28.363 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000118455 s, 4.3 MB/s 2024-02-19T16:14:28.364 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-19T16:14:28.422 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-19T16:14:28.422 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-02-19T16:14:28.440 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-19T16:14:28.441 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-02-19T16:14:28.497 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-19T16:14:28.497 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-19 16:14:04.840892414 +0000 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-19 16:08:40.176582133 +0000 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-19 16:08:40.176582133 +0000 2024-02-19T16:14:28.498 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-19 16:08:40.176582133 +0000 2024-02-19T16:14:28.498 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-19T16:14:28.566 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:28 smithi191 ceph-mon[37203]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:28.566 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:28.566 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:28.566 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:28.567 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-19T16:14:28.567 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-19T16:14:28.567 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00026993 s, 1.9 MB/s 2024-02-19T16:14:28.568 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-19T16:14:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:28 smithi156 ceph-mon[36502]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:28.626 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-02-19T16:14:28.683 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-19T16:14:28.683 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:28.683 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-19T16:14:28.683 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:28.683 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:28.683 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-19 16:14:04.840892414 +0000 2024-02-19T16:14:28.684 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-19 16:08:40.521575964 +0000 2024-02-19T16:14:28.684 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-19 16:08:40.521575964 +0000 2024-02-19T16:14:28.684 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-19 16:08:40.521575964 +0000 2024-02-19T16:14:28.684 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-19T16:14:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-19T16:14:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-19T16:14:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000216508 s, 2.4 MB/s 2024-02-19T16:14:28.754 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-19T16:14:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:28 smithi008 ceph-mon[32573]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:28.809 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-02-19T16:14:28.864 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-19T16:14:28.864 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:28.864 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-02-19T16:14:28.864 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:28.864 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:28.864 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-19 16:14:04.840892414 +0000 2024-02-19T16:14:28.865 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-19 16:08:40.862569867 +0000 2024-02-19T16:14:28.865 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-19 16:08:40.862569867 +0000 2024-02-19T16:14:28.865 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-19 16:08:40.862569867 +0000 2024-02-19T16:14:28.865 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-19T16:14:28.930 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-19T16:14:28.930 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-19T16:14:28.930 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000172773 s, 3.0 MB/s 2024-02-19T16:14:28.931 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-19T16:14:28.987 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-19 16:14:04.841892396 +0000 2024-02-19T16:14:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-19 16:08:41.215563555 +0000 2024-02-19T16:14:29.045 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-19 16:08:41.215563555 +0000 2024-02-19T16:14:29.045 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-19 16:08:41.215563555 +0000 2024-02-19T16:14:29.045 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-19T16:14:29.109 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-19T16:14:29.109 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-19T16:14:29.109 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000240527 s, 2.1 MB/s 2024-02-19T16:14:29.111 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-19T16:14:29.166 INFO:tasks.cephadm:Deploying osd.0 on smithi008 with /dev/vg_nvme/lv_4... 2024-02-19T16:14:29.167 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-19T16:14:29.407 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:14:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:29.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T16:14:29.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:29.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:29.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:29.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:29.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:29.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:29.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T16:14:29.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:29.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:29.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:30.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:30 smithi156 ceph-mon[36502]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:30.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:30 smithi156 ceph-mon[36502]: Deploying daemon mgr.z on smithi191 2024-02-19T16:14:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:30 smithi191 ceph-mon[37203]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:30 smithi191 ceph-mon[37203]: Deploying daemon mgr.z on smithi191 2024-02-19T16:14:30.649 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:30 smithi008 ceph-mon[32573]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:30.650 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:30 smithi008 ceph-mon[32573]: Deploying daemon mgr.z on smithi191 2024-02-19T16:14:30.973 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:30 smithi191 systemd[1]: Starting Ceph mgr.z for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:14:31.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:30 smithi191 podman[39044]: 2024-02-19 16:14:30.875867711 +0000 UTC m=+0.017616410 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:14:31.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:30 smithi191 podman[39044]: 2024-02-19 16:14:30.99022626 +0000 UTC m=+0.131974985 container create 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-02-19T16:14:31.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 podman[39044]: 2024-02-19 16:14:31.176482715 +0000 UTC m=+0.318231397 container init 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:14:31.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 podman[39044]: 2024-02-19 16:14:31.179332785 +0000 UTC m=+0.321081463 container start 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:14:31.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 ceph-mgr[39064]: -- 172.21.15.191:0/1413904257 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a0dd961a0 con 0x558a0dd6cc00 2024-02-19T16:14:31.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 bash[39044]: 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 2024-02-19T16:14:31.319 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-19T16:14:31.319 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-19T16:14:31.319 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-02-19T16:14:31.319 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-02-19T16:14:31.319 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00825743 s, 1.3 GB/s 2024-02-19T16:14:31.319 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-02-19T16:14:31.515 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 systemd[1]: Started Ceph mgr.z for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:14:31.776 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:31.754+0000 7fdc19034200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:14:31.961 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_4 2024-02-19T16:14:32.205 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:14:32.276 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:31.849+0000 7fdc19034200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:14:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:32.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:32 smithi156 ceph-mon[36502]: Deploying daemon mgr.y on smithi156 2024-02-19T16:14:32.591 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:32.496+0000 7fdc19034200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:14:32.591 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:32.590+0000 7fdc19034200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:32.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:32.593 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-mon[37203]: Deploying daemon mgr.y on smithi156 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T16:14:32.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:32.618 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:32.618 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:32 smithi008 ceph-mon[32573]: Deploying daemon mgr.y on smithi156 2024-02-19T16:14:32.911 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:32.732+0000 7fdc19034200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:14:32.911 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:32.827+0000 7fdc19034200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:14:32.911 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:32.910+0000 7fdc19034200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:14:33.461 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:33.172+0000 7fdc19034200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:14:33.461 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:33.281+0000 7fdc19034200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:14:33.461 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:33.366+0000 7fdc19034200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:14:33.461 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:33.460+0000 7fdc19034200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:14:33.577 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 systemd[1]: Starting Ceph mgr.y for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:14:33.776 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:33.639+0000 7fdc19034200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:14:33.829 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 podman[37867]: 2024-02-19 16:14:33.576877149 +0000 UTC m=+0.112393115 container create 71c335939279572bbc545b5fc8affb44f241c135b363fb93c4d2d99e3108ff66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-19T16:14:33.829 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 podman[37867]: 2024-02-19 16:14:33.48214434 +0000 UTC m=+0.017660309 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:14:33.830 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 podman[37867]: 2024-02-19 16:14:33.762519005 +0000 UTC m=+0.298034974 container init 71c335939279572bbc545b5fc8affb44f241c135b363fb93c4d2d99e3108ff66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-19T16:14:33.830 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 podman[37867]: 2024-02-19 16:14:33.765202956 +0000 UTC m=+0.300718935 container start 71c335939279572bbc545b5fc8affb44f241c135b363fb93c4d2d99e3108ff66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:14:33.830 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 ceph-mgr[37887]: -- 172.21.15.156:0/4257934707 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581b01a41a0 con 0x5581b017b000 2024-02-19T16:14:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 -- 172.21.15.8:0/1217062302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf40749c0 msgr2=0x7efcf4074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 --2- 172.21.15.8:0/1217062302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf40749c0 0x7efcf4074e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009990 tx=0x7efcdc009ca0 comp rx=0 tx=0).stop 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 -- 172.21.15.8:0/1217062302 shutdown_connections 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 --2- 172.21.15.8:0/1217062302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf4109f30 0x7efcf410c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 --2- 172.21.15.8:0/1217062302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf4075f90 0x7efcf40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 --2- 172.21.15.8:0/1217062302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf40749c0 0x7efcf4074e40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 -- 172.21.15.8:0/1217062302 >> 172.21.15.8:0/1217062302 conn(0x7efcf40fd5f0 msgr2=0x7efcf40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 -- 172.21.15.8:0/1217062302 shutdown_connections 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 -- 172.21.15.8:0/1217062302 wait complete. 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 Processor -- start 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.891+0000 7efcfa53f700 1 -- start start 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcfa53f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf40749c0 0x7efcf419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcfa53f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf4075f90 0x7efcf419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcfa53f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 0x7efcf41a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcfa53f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41a1d90 con 0x7efcf4109f30 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcfa53f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41a1ed0 con 0x7efcf4075f90 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcfa53f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41a2060 con 0x7efcf40749c0 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 0x7efcf41a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 0x7efcf41a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44560/0 (socket says 172.21.15.8:44560) 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 -- 172.21.15.8:0/693483017 learned_addr learned my addr 172.21.15.8:0/693483017 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf3fff700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf40749c0 0x7efcf419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf37fe700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf4075f90 0x7efcf419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 -- 172.21.15.8:0/693483017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf40749c0 msgr2=0x7efcf419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf40749c0 0x7efcf419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 -- 172.21.15.8:0/693483017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf4075f90 msgr2=0x7efcf419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf4075f90 0x7efcf419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.892+0000 7efcf8adc700 1 -- 172.21.15.8:0/693483017 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0095f0 con 0x7efcf4109f30 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.893+0000 7efcf8adc700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 0x7efcf41a17c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efcec007f10 tx=0x7efcec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.893+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00dd20 con 0x7efcf4109f30 2024-02-19T16:14:33.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.893+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcec00f040 con 0x7efcf4109f30 2024-02-19T16:14:33.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.893+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec015400 con 0x7efcf4109f30 2024-02-19T16:14:33.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.893+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf41a2340 con 0x7efcf4109f30 2024-02-19T16:14:33.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.893+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf41a2840 con 0x7efcf4109f30 2024-02-19T16:14:33.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.894+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45987+0+0 (secure 0 0 0) 0x7efcec015560 con 0x7efcf4109f30 2024-02-19T16:14:33.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.894+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf4108d40 con 0x7efcf4109f30 2024-02-19T16:14:33.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.894+0000 7efcf17fa700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efce00391c0 0x7efce003b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:33.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.894+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efcec04b450 con 0x7efcf4109f30 2024-02-19T16:14:33.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.894+0000 7efcf3fff700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efce00391c0 0x7efce003b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:33.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.894+0000 7efcf3fff700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efce00391c0 0x7efce003b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efce4005950 tx=0x7efce400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:33.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:33.899+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7efcec0138f0 con 0x7efcf4109f30 2024-02-19T16:14:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:34.135+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efcf402d9c0 con 0x7efce00391c0 2024-02-19T16:14:34.289 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 bash[37867]: 71c335939279572bbc545b5fc8affb44f241c135b363fb93c4d2d99e3108ff66 2024-02-19T16:14:34.289 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:33 smithi156 systemd[1]: Started Ceph mgr.y for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:14:34.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:34.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:34.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:14:34.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:14:34.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:34.579 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:34.335+0000 7f0f793fe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:34.435+0000 7f0f793fe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:34.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:14:34.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:14:34.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:34.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:14:34.639 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:34.942 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:34.637+0000 7fdc19034200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:14:34.942 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:34 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:34.776+0000 7fdc19034200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:14:35.211 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:35.119+0000 7fdc19034200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:14:35.211 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:35.210+0000 7fdc19034200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:14:35.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.124+0000 7f0f793fe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:14:35.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.220+0000 7f0f793fe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:14:35.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:35 smithi191 ceph-mon[37203]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:35.527 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:35.414+0000 7fdc19034200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:14:35.555 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-mon[36502]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:35.555 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.367+0000 7f0f793fe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:14:35.555 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.469+0000 7f0f793fe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:14:35.556 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.554+0000 7f0f793fe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:14:35.755 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:35 smithi008 ceph-mon[32573]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:14:35.885 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:35.603+0000 7fdc19034200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:14:36.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.817+0000 7f0f793fe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:14:36.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:35.926+0000 7f0f793fe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:14:36.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:36.011+0000 7f0f793fe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:14:36.148 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:35.884+0000 7fdc19034200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:14:36.148 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:36.147+0000 7fdc19034200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:14:36.452 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:36.379+0000 7fdc19034200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:14:36.453 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:36.105+0000 7f0f793fe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:14:36.453 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:36.282+0000 7f0f793fe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:14:36.704 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: Reconfiguring mgr.x (unknown last config time)... 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:36.705 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:36.706 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: Reconfiguring daemon mgr.x on smithi008 2024-02-19T16:14:36.706 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: Reconfiguring mgr.x (unknown last config time)... 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: Reconfiguring daemon mgr.x on smithi008 2024-02-19T16:14:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: Reconfiguring mgr.x (unknown last config time)... 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: Reconfiguring daemon mgr.x on smithi008 2024-02-19T16:14:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:37.456 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:37.171+0000 7fdc19034200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:14:37.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1418877246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb7652b6-3998-473c-bd04-422f89e95197"}]: dispatch 2024-02-19T16:14:37.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1418877246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb7652b6-3998-473c-bd04-422f89e95197"}]': finished 2024-02-19T16:14:37.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-mon[36502]: osdmap e5: 1 total, 0 up, 1 in 2024-02-19T16:14:37.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:37.580 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:37.254+0000 7f0f793fe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:14:37.580 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:37.383+0000 7f0f793fe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:14:37.739 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1418877246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb7652b6-3998-473c-bd04-422f89e95197"}]: dispatch 2024-02-19T16:14:37.739 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1418877246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb7652b6-3998-473c-bd04-422f89e95197"}]': finished 2024-02-19T16:14:37.739 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-mon[37203]: osdmap e5: 1 total, 0 up, 1 in 2024-02-19T16:14:37.739 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:37.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1418877246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb7652b6-3998-473c-bd04-422f89e95197"}]: dispatch 2024-02-19T16:14:37.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1418877246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb7652b6-3998-473c-bd04-422f89e95197"}]': finished 2024-02-19T16:14:37.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:37 smithi008 ceph-mon[32573]: osdmap e5: 1 total, 0 up, 1 in 2024-02-19T16:14:37.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:37 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:38.005 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:37.720+0000 7f0f793fe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:14:38.005 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:37.809+0000 7f0f793fe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:14:38.005 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:38.003+0000 7f0f793fe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:14:38.026 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:37.738+0000 7fdc19034200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:14:38.027 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:37.835+0000 7fdc19034200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:14:38.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:38.188+0000 7f0f793fe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:14:38.458 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:14:38.047+0000 7fdc19034200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:14:38.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:38.466+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 14) v1 ==== 91349+0+0 (secure 0 0 0) 0x7efcec004020 con 0x7efcf4109f30 2024-02-19T16:14:38.732 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3023078043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: Standby manager daemon z started 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:14:38.733 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:38.470+0000 7f0f793fe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:14:38.734 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:38.731+0000 7f0f793fe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:14:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3023078043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:14:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: Standby manager daemon z started 2024-02-19T16:14:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-19T16:14:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:14:38.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-19T16:14:38.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:38 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:14:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3023078043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:14:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: Standby manager daemon z started 2024-02-19T16:14:38.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-19T16:14:38.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:14:38.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-19T16:14:38.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:38 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/4048838735' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:14:39.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:38.964+0000 7f0f793fe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:14:39.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:39 smithi156 ceph-mon[36502]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-19T16:14:39.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-19T16:14:39.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:39 smithi008 ceph-mon[32573]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-19T16:14:39.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-19T16:14:39.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:39 smithi191 ceph-mon[37203]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-19T16:14:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-19T16:14:40.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:39 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:39.761+0000 7f0f793fe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:14:40.642 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:40 smithi156 ceph-mon[36502]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:40.642 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:40.331+0000 7f0f793fe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:14:40.642 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:40.427+0000 7f0f793fe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:14:40.642 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:14:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:14:40.640+0000 7f0f793fe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:14:40.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:40 smithi008 ceph-mon[32573]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:40.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:40 smithi191 ceph-mon[37203]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:41.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:41.480+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7efcec01cc30 con 0x7efcf4109f30 2024-02-19T16:14:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:41 smithi008 ceph-mon[32573]: Standby manager daemon y started 2024-02-19T16:14:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:41 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-19T16:14:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:41 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:14:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:41 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-19T16:14:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:41 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:14:41.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:41 smithi191 ceph-mon[37203]: Standby manager daemon y started 2024-02-19T16:14:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:41 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-19T16:14:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:41 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:14:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:41 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-19T16:14:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:41 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:14:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:41 smithi156 ceph-mon[36502]: Standby manager daemon y started 2024-02-19T16:14:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:41 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-19T16:14:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:41 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:14:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:41 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-19T16:14:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:41 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/1791150761' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:14:42.621 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:42 smithi008 ceph-mon[32573]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:42.621 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:42 smithi008 ceph-mon[32573]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-02-19T16:14:42.622 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:42 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-19T16:14:42.622 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:42 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:42.622 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:42 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:42.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:42 smithi191 ceph-mon[37203]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:42.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:42 smithi191 ceph-mon[37203]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-02-19T16:14:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:42 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-19T16:14:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:42 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:42 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:42 smithi156 ceph-mon[36502]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:42 smithi156 ceph-mon[36502]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-02-19T16:14:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:42 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-19T16:14:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:42 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:42 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:43.691 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:43.691 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:43.691 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:43.691 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:44.742 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:44 smithi008 ceph-mon[32573]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:44.750 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:44 smithi156 ceph-mon[36502]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:44.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:44 smithi191 ceph-mon[37203]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:45.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T16:14:45.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T16:14:45.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T16:14:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:46.619 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:46 smithi008 ceph-mon[32573]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:46.619 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:46 smithi008 ceph-mon[32573]: Deploying daemon osd.0 on smithi008 2024-02-19T16:14:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:46 smithi191 ceph-mon[37203]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:46 smithi191 ceph-mon[37203]: Deploying daemon osd.0 on smithi008 2024-02-19T16:14:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:46 smithi156 ceph-mon[36502]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:46 smithi156 ceph-mon[36502]: Deploying daemon osd.0 on smithi008 2024-02-19T16:14:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:48 smithi008 ceph-mon[32573]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:48.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:48 smithi191 ceph-mon[37203]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:48.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:48 smithi156 ceph-mon[36502]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:50 smithi008 ceph-mon[32573]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:50.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:50 smithi191 ceph-mon[37203]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:50.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:50.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:50.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:50 smithi156 ceph-mon[36502]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:50.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:50.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:50.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:52.757 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:52 smithi008 ceph-mon[32573]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:52.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:52 smithi191 ceph-mon[37203]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:52.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:52 smithi156 ceph-mon[36502]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:53.934 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:53 smithi008 ceph-mon[32573]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:53.934 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:53.934 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:53.934 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:53.934 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:53.935 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:53.992 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 0 on host 'smithi008' 2024-02-19T16:14:53.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.991+0000 7efcf17fa700 1 -- 172.21.15.8:0/693483017 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efcf402d9c0 con 0x7efce00391c0 2024-02-19T16:14:53.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efce00391c0 msgr2=0x7efce003b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:53.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efce00391c0 0x7efce003b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efce4005950 tx=0x7efce400b410 comp rx=0 tx=0).stop 2024-02-19T16:14:53.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 msgr2=0x7efcf41a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:53.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 0x7efcf41a17c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efcec007f10 tx=0x7efcec00d3b0 comp rx=0 tx=0).stop 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 shutdown_connections 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efce00391c0 0x7efce003b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcf4109f30 0x7efcf41a17c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf40749c0 0x7efcf419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 --2- 172.21.15.8:0/693483017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf4075f90 0x7efcf419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 >> 172.21.15.8:0/693483017 conn(0x7efcf40fd5f0 msgr2=0x7efcf4105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 shutdown_connections 2024-02-19T16:14:53.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:14:53.993+0000 7efcfa53f700 1 -- 172.21.15.8:0/693483017 wait complete. 2024-02-19T16:14:54.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:53 smithi156 ceph-mon[36502]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:54.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:54.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:54.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:54.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:54.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:54.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:53 smithi191 ceph-mon[37203]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:54.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:54.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:54.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:54.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:54.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:54.613 DEBUG:teuthology.orchestra.run.smithi008:osd.0> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.0.service 2024-02-19T16:14:54.618 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_4... 2024-02-19T16:14:54.618 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-19T16:14:54.862 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:14:55.175 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:55.175 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:55.175 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:55.175 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:54 smithi008 ceph-mon[32573]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T16:14:55.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:55.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:55.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:55.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:54 smithi156 ceph-mon[36502]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T16:14:55.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:55.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:55.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:14:55.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:54 smithi191 ceph-mon[37203]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T16:14:56.112 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:55 smithi156 ceph-mon[36502]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:56.112 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:55 smithi156 ceph-mon[36502]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T16:14:56.112 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:55 smithi156 ceph-mon[36502]: osdmap e6: 1 total, 0 up, 1 in 2024-02-19T16:14:56.113 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:55 smithi156 ceph-mon[36502]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:14:56.113 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:56.163 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:55 smithi008 ceph-mon[32573]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:56.164 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:55 smithi008 ceph-mon[32573]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T16:14:56.164 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:55 smithi008 ceph-mon[32573]: osdmap e6: 1 total, 0 up, 1 in 2024-02-19T16:14:56.164 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:55 smithi008 ceph-mon[32573]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:14:56.164 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:56.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:55 smithi191 ceph-mon[37203]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:56.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:55 smithi191 ceph-mon[37203]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T16:14:56.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:55 smithi191 ceph-mon[37203]: osdmap e6: 1 total, 0 up, 1 in 2024-02-19T16:14:56.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:55 smithi191 ceph-mon[37203]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:14:56.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:56.660 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-19T16:14:56.660 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-19T16:14:56.660 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-19T16:14:56.660 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-19T16:14:56.660 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00856062 s, 1.2 GB/s 2024-02-19T16:14:56.660 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-19T16:14:56.769 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0[45163]: 2024-02-19T16:14:56.448+0000 7feb239e2700 -1 osd.0 0 waiting for initial osdmap 2024-02-19T16:14:56.769 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0[45163]: 2024-02-19T16:14:56.452+0000 7feb1afcc700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:14:57.220 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2024-02-19T16:14:57.246 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:56 smithi156 ceph-mon[36502]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:14:57.246 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:56 smithi156 ceph-mon[36502]: osdmap e7: 1 total, 0 up, 1 in 2024-02-19T16:14:57.246 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:56 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:57.246 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:56 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:57.246 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:56 smithi156 ceph-mon[36502]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' 2024-02-19T16:14:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-mon[32573]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:14:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-mon[32573]: osdmap e7: 1 total, 0 up, 1 in 2024-02-19T16:14:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:56 smithi008 ceph-mon[32573]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' 2024-02-19T16:14:57.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:56 smithi191 ceph-mon[37203]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:14:57.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:56 smithi191 ceph-mon[37203]: osdmap e7: 1 total, 0 up, 1 in 2024-02-19T16:14:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:56 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:56 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:56 smithi191 ceph-mon[37203]: from='osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077]' entity='osd.0' 2024-02-19T16:14:57.471 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:14:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:14:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:14:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] boot 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: osdmap e8: 1 total, 1 up, 1 in 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:14:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:14:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] boot 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: osdmap e8: 1 total, 1 up, 1 in 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: osd.0 [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] boot 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: osdmap e8: 1 total, 1 up, 1 in 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:14:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 -- 172.21.15.156:0/1827713159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50102000 msgr2=0x7faf50102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 --2- 172.21.15.156:0/1827713159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50102000 0x7faf50102420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf440099c0 tx=0x7faf44009cd0 comp rx=0 tx=0).stop 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 -- 172.21.15.156:0/1827713159 shutdown_connections 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 --2- 172.21.15.156:0/1827713159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf50103ba0 0x7faf5010a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 --2- 172.21.15.156:0/1827713159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50102000 0x7faf50102420 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 --2- 172.21.15.156:0/1827713159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50103200 0x7faf50103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 -- 172.21.15.156:0/1827713159 >> 172.21.15.156:0/1827713159 conn(0x7faf500fd5c0 msgr2=0x7faf500ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 -- 172.21.15.156:0/1827713159 shutdown_connections 2024-02-19T16:14:58.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.852+0000 7faf56e64700 1 -- 172.21.15.156:0/1827713159 wait complete. 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 Processor -- start 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 -- start start 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50102000 0x7faf50075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf50103200 0x7faf500743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 0x7faf50074910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50075090 con 0x7faf50102000 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50075200 con 0x7faf50103ba0 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf56e64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501a2020 con 0x7faf50103200 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 0x7faf50074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 0x7faf50074910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49036/0 (socket says 172.21.15.156:49036) 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 -- 172.21.15.156:0/3882269222 learned_addr learned my addr 172.21.15.156:0/3882269222 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf54c00700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50102000 0x7faf50075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:58.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 -- 172.21.15.156:0/3882269222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf50103200 msgr2=0x7faf500743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf50103200 0x7faf500743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 -- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50102000 msgr2=0x7faf50075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50102000 0x7faf50075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 -- 172.21.15.156:0/3882269222 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440095f0 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf54c00700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50102000 0x7faf50075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.853+0000 7faf55401700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 0x7faf50074910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf4800ada0 tx=0x7faf4800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.854+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800f070 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.854+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf48007b30 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.854+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf501a2300 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.854+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48005500 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.854+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf501a2800 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.855+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf501094e0 con 0x7faf50103ba0 2024-02-19T16:14:58.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.856+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7faf48004ad0 con 0x7faf50103ba0 2024-02-19T16:14:58.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.856+0000 7faf4dffb700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faf400a20c0 0x7faf400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:14:58.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.856+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7faf480cb940 con 0x7faf50103ba0 2024-02-19T16:14:58.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.857+0000 7faf54c00700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faf400a20c0 0x7faf400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:14:58.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.857+0000 7faf54c00700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faf400a20c0 0x7faf400a4580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf4400b580 tx=0x7faf44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:14:58.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:58.861+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7faf480963f0 con 0x7faf50103ba0 2024-02-19T16:14:59.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:14:59.070+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7faf5002da30 con 0x7faf400a20c0 2024-02-19T16:14:59.071 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:58 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi008 2024-02-19T16:14:59.071 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:58 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi008 to 17093M 2024-02-19T16:14:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:58 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi008 2024-02-19T16:14:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:58 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi008 to 17093M 2024-02-19T16:14:59.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:58 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi008 2024-02-19T16:14:59.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:58 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi008 to 17093M 2024-02-19T16:15:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:59 smithi008 ceph-mon[32573]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:59 smithi008 ceph-mon[32573]: osdmap e9: 1 total, 1 up, 1 in 2024-02-19T16:15:00.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:59 smithi008 ceph-mon[32573]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:00.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:00.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:00.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:14:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:00.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:59 smithi191 ceph-mon[37203]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:00.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:59 smithi191 ceph-mon[37203]: osdmap e9: 1 total, 1 up, 1 in 2024-02-19T16:15:00.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:59 smithi191 ceph-mon[37203]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:00.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:00.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:00.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:14:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:59 smithi156 ceph-mon[36502]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:59 smithi156 ceph-mon[36502]: osdmap e9: 1 total, 1 up, 1 in 2024-02-19T16:15:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:59 smithi156 ceph-mon[36502]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:14:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:02.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:01 smithi156 ceph-mon[36502]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:02.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:01 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/4139125403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]: dispatch 2024-02-19T16:15:02.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:01 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]: dispatch 2024-02-19T16:15:02.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:01 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]': finished 2024-02-19T16:15:02.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:01 smithi156 ceph-mon[36502]: osdmap e10: 2 total, 1 up, 2 in 2024-02-19T16:15:02.243 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:01 smithi008 ceph-mon[32573]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:01 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/4139125403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]: dispatch 2024-02-19T16:15:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:01 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]: dispatch 2024-02-19T16:15:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:01 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]': finished 2024-02-19T16:15:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:01 smithi008 ceph-mon[32573]: osdmap e10: 2 total, 1 up, 2 in 2024-02-19T16:15:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:01 smithi191 ceph-mon[37203]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:01 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/4139125403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]: dispatch 2024-02-19T16:15:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:01 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]: dispatch 2024-02-19T16:15:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:01 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaca12a9-0ccf-4c84-b071-32648d5dcca8"}]': finished 2024-02-19T16:15:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:01 smithi191 ceph-mon[37203]: osdmap e10: 2 total, 1 up, 2 in 2024-02-19T16:15:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:03.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:02 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/2758791921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:03.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:02 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/2758791921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:03.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:02 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/2758791921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:03 smithi008 ceph-mon[32573]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:04.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:03 smithi191 ceph-mon[37203]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:04.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:03 smithi156 ceph-mon[36502]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:05 smithi008 ceph-mon[32573]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:06.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:05 smithi191 ceph-mon[37203]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:05 smithi156 ceph-mon[36502]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:08 smithi008 ceph-mon[32573]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:08 smithi191 ceph-mon[37203]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:08.301 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:08 smithi156 ceph-mon[36502]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:09 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T16:15:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:09 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:09 smithi008 ceph-mon[32573]: Deploying daemon osd.1 on smithi156 2024-02-19T16:15:09.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:09 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T16:15:09.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:09 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:09.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:09 smithi191 ceph-mon[37203]: Deploying daemon osd.1 on smithi156 2024-02-19T16:15:09.327 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:09 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T16:15:09.327 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:09 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:09.327 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:09 smithi156 ceph-mon[36502]: Deploying daemon osd.1 on smithi156 2024-02-19T16:15:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:10 smithi008 ceph-mon[32573]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:10.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:10 smithi191 ceph-mon[37203]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:10 smithi156 ceph-mon[36502]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:12.210 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:12 smithi156 ceph-mon[36502]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:12 smithi008 ceph-mon[32573]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:12.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:12 smithi191 ceph-mon[37203]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:14 smithi008 ceph-mon[32573]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:14 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:14 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:14 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:14 smithi191 ceph-mon[37203]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:14 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:14 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:14 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:14 smithi156 ceph-mon[36502]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:14 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:14 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:14 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:16.220 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:16 smithi156 ceph-mon[36502]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:16.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:16 smithi008 ceph-mon[32573]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:16.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:16 smithi191 ceph-mon[37203]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:17.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.281+0000 7faf4dffb700 1 -- 172.21.15.156:0/3882269222 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faf5002da30 con 0x7faf400a20c0 2024-02-19T16:15:17.283 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2024-02-19T16:15:17.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faf400a20c0 msgr2=0x7faf400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:17.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faf400a20c0 0x7faf400a4580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf4400b580 tx=0x7faf44005fb0 comp rx=0 tx=0).stop 2024-02-19T16:15:17.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 msgr2=0x7faf50074910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 0x7faf50074910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf4800ada0 tx=0x7faf4800c330 comp rx=0 tx=0).stop 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 shutdown_connections 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faf400a20c0 0x7faf400a4580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf50103ba0 0x7faf50074910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf50102000 0x7faf50075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 --2- 172.21.15.156:0/3882269222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf50103200 0x7faf500743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 >> 172.21.15.156:0/3882269222 conn(0x7faf500fd5c0 msgr2=0x7faf50106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 shutdown_connections 2024-02-19T16:15:17.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:15:17.284+0000 7faf56e64700 1 -- 172.21.15.156:0/3882269222 wait complete. 2024-02-19T16:15:17.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:17.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:17.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:17.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:17 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:17 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:17 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:17.986 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.1.service 2024-02-19T16:15:18.029 INFO:tasks.cephadm:Deploying osd.2 on smithi191 with /dev/vg_nvme/lv_4... 2024-02-19T16:15:18.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-19T16:15:18.272 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:15:18.546 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:18 smithi156 ceph-mon[36502]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:18.546 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:18 smithi156 ceph-mon[36502]: from='osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T16:15:18.546 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:18 smithi156 ceph-mon[36502]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T16:15:18.657 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:18 smithi191 ceph-mon[37203]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:18.657 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:18 smithi191 ceph-mon[37203]: from='osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T16:15:18.657 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:18 smithi191 ceph-mon[37203]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T16:15:18.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:18 smithi008 ceph-mon[32573]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:18.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:18 smithi008 ceph-mon[32573]: from='osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T16:15:18.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:18 smithi008 ceph-mon[32573]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T16:15:19.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-mon[36502]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T16:15:19.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-mon[36502]: from='osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:15:19.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-mon[36502]: osdmap e11: 2 total, 1 up, 2 in 2024-02-19T16:15:19.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:19.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-mon[36502]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:15:19.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-mon[36502]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:15:19.666 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:19 smithi191 ceph-mon[37203]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T16:15:19.666 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:19 smithi191 ceph-mon[37203]: from='osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:15:19.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:19 smithi191 ceph-mon[37203]: osdmap e11: 2 total, 1 up, 2 in 2024-02-19T16:15:19.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:19.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:19 smithi191 ceph-mon[37203]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:15:19.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:19 smithi191 ceph-mon[37203]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:15:19.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:19 smithi008 ceph-mon[32573]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T16:15:19.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:19 smithi008 ceph-mon[32573]: from='osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:15:19.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:19 smithi008 ceph-mon[32573]: osdmap e11: 2 total, 1 up, 2 in 2024-02-19T16:15:19.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:19.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:19 smithi008 ceph-mon[32573]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:15:19.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:19 smithi008 ceph-mon[32573]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:15:20.224 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-19T16:15:20.224 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-19T16:15:20.224 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-19T16:15:20.224 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-19T16:15:20.224 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127554 s, 822 MB/s 2024-02-19T16:15:20.224 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-19T16:15:20.238 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1[41099]: 2024-02-19T16:15:19.908+0000 7f376cf57700 -1 osd.1 0 waiting for initial osdmap 2024-02-19T16:15:20.238 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:19 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1[41099]: 2024-02-19T16:15:19.911+0000 7f3764d42700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:15:20.504 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:20 smithi191 ceph-mon[37203]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:20.504 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:20 smithi191 ceph-mon[37203]: osdmap e12: 2 total, 1 up, 2 in 2024-02-19T16:15:20.504 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.504 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.504 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:20 smithi191 ceph-mon[37203]: from='osd.1 ' entity='osd.1' 2024-02-19T16:15:20.504 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.555 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:20 smithi156 ceph-mon[36502]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:20.555 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:20 smithi156 ceph-mon[36502]: osdmap e12: 2 total, 1 up, 2 in 2024-02-19T16:15:20.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:20 smithi156 ceph-mon[36502]: from='osd.1 ' entity='osd.1' 2024-02-19T16:15:20.556 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:20 smithi008 ceph-mon[32573]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:20 smithi008 ceph-mon[32573]: osdmap e12: 2 total, 1 up, 2 in 2024-02-19T16:15:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:20 smithi008 ceph-mon[32573]: from='osd.1 ' entity='osd.1' 2024-02-19T16:15:20.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:20.806 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_4 2024-02-19T16:15:21.050 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:15:21.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:15:21.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:15:21.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] boot 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: osdmap e13: 2 total, 2 up, 2 in 2024-02-19T16:15:21.477 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:21.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:15:21.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:15:21.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] boot 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: osdmap e13: 2 total, 2 up, 2 in 2024-02-19T16:15:21.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:21.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: osd.1 [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] boot 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: osdmap e13: 2 total, 2 up, 2 in 2024-02-19T16:15:21.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 -- 172.21.15.191:0/3398740678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 msgr2=0x7f52b4101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 --2- 172.21.15.191:0/3398740678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b4101330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009990 tx=0x7f52b0009ca0 comp rx=0 tx=0).stop 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 -- 172.21.15.191:0/3398740678 shutdown_connections 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 --2- 172.21.15.191:0/3398740678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b4102a30 0x7f52b4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 --2- 172.21.15.191:0/3398740678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b4101330 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 --2- 172.21.15.191:0/3398740678 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 0x7f52b41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 -- 172.21.15.191:0/3398740678 >> 172.21.15.191:0/3398740678 conn(0x7f52b40fc4b0 msgr2=0x7f52b40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 -- 172.21.15.191:0/3398740678 shutdown_connections 2024-02-19T16:15:22.556 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.552+0000 7f52bb355700 1 -- 172.21.15.191:0/3398740678 wait complete. 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 Processor -- start 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 -- start start 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 0x7f52b419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b4102a30 0x7f52b41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41a1d70 con 0x7f52b4102070 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41a1ee0 con 0x7f52b4102a30 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52bb355700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41a20a0 con 0x7f52b4100f10 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:22.557 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b406b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:37350/0 (socket says 172.21.15.191:37350) 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 -- 172.21.15.191:0/4274119136 learned_addr learned my addr 172.21.15.191:0/4274119136 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b98f2700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b4102a30 0x7f52b41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b88f0700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 0x7f52b419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 -- 172.21.15.191:0/4274119136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b4102a30 msgr2=0x7f52b41a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b4102a30 0x7f52b41a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 -- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 msgr2=0x7f52b419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 0x7f52b419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 -- 172.21.15.191:0/4274119136 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00095f0 con 0x7f52b4100f10 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b88f0700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 0x7f52b419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.553+0000 7f52b90f1700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b406b1e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52b000f690 tx=0x7f52b000f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.554+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b001d070 con 0x7f52b4100f10 2024-02-19T16:15:22.558 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.554+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52b0004cc0 con 0x7f52b4100f10 2024-02-19T16:15:22.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.554+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b41a2320 con 0x7f52b4100f10 2024-02-19T16:15:22.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.554+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00056e0 con 0x7f52b4100f10 2024-02-19T16:15:22.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.554+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b41a2810 con 0x7f52b4100f10 2024-02-19T16:15:22.559 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.554+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b41082c0 con 0x7f52b4100f10 2024-02-19T16:15:22.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.556+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7f52b00047e0 con 0x7f52b4100f10 2024-02-19T16:15:22.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.556+0000 7f52aa7fc700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f52a00a20c0 0x7f52a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:22.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.556+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f52b00cda90 con 0x7f52b4100f10 2024-02-19T16:15:22.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.557+0000 7f52b88f0700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f52a00a20c0 0x7f52a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:22.560 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.557+0000 7f52b88f0700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f52a00a20c0 0x7f52a00a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52b406b760 tx=0x7f52a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:15:22.561 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.560+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f52b0098340 con 0x7f52b4100f10 2024-02-19T16:15:22.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:22.763+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f52b4062420 con 0x7f52a00a20c0 2024-02-19T16:15:22.764 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:22 smithi191 ceph-mon[37203]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:22.764 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:22 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi156 2024-02-19T16:15:22.764 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:22 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi156 to 17080M 2024-02-19T16:15:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:22 smithi008 ceph-mon[32573]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:22 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi156 2024-02-19T16:15:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:22 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi156 to 17080M 2024-02-19T16:15:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:22 smithi156 ceph-mon[36502]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-19T16:15:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:22 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi156 2024-02-19T16:15:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:22 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi156 to 17080M 2024-02-19T16:15:23.627 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:23 smithi191 ceph-mon[37203]: osdmap e14: 2 total, 2 up, 2 in 2024-02-19T16:15:23.627 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:23 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:23.627 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:23 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:23.628 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:23 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:23.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:23 smithi008 ceph-mon[32573]: osdmap e14: 2 total, 2 up, 2 in 2024-02-19T16:15:23.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:23.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:23.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:23.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:23 smithi156 ceph-mon[36502]: osdmap e14: 2 total, 2 up, 2 in 2024-02-19T16:15:23.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:23 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:23.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:23 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:23.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:23 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:24.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:24 smithi191 ceph-mon[37203]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:24.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:24 smithi191 ceph-mon[37203]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:24.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:24 smithi008 ceph-mon[32573]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:24.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:24 smithi008 ceph-mon[32573]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:24.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:24 smithi156 ceph-mon[36502]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:24.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:24 smithi156 ceph-mon[36502]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:25.709 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:25 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/954068902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]: dispatch 2024-02-19T16:15:25.709 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:25 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]: dispatch 2024-02-19T16:15:25.709 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:25 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]': finished 2024-02-19T16:15:25.709 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:25 smithi191 ceph-mon[37203]: osdmap e15: 3 total, 2 up, 3 in 2024-02-19T16:15:25.709 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:25.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:25 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/954068902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]: dispatch 2024-02-19T16:15:25.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:25 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]: dispatch 2024-02-19T16:15:25.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:25 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]': finished 2024-02-19T16:15:25.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:25 smithi008 ceph-mon[32573]: osdmap e15: 3 total, 2 up, 3 in 2024-02-19T16:15:25.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:25.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:25 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/954068902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]: dispatch 2024-02-19T16:15:25.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:25 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]: dispatch 2024-02-19T16:15:25.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:25 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5bc2023-4102-4aec-a85d-4c8f175a3926"}]': finished 2024-02-19T16:15:25.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:25 smithi156 ceph-mon[36502]: osdmap e15: 3 total, 2 up, 3 in 2024-02-19T16:15:25.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:26.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:26 smithi008 ceph-mon[32573]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:26.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:26 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/1214958609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:26.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:26 smithi191 ceph-mon[37203]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:26.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:26 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/1214958609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:26.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:26 smithi156 ceph-mon[36502]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:26.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:26 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/1214958609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:28 smithi008 ceph-mon[32573]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:28.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:28 smithi191 ceph-mon[37203]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:28 smithi156 ceph-mon[36502]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:30.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:30 smithi191 ceph-mon[37203]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:30.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:30 smithi008 ceph-mon[32573]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:30 smithi156 ceph-mon[36502]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:32.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:32 smithi008 ceph-mon[32573]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:32.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T16:15:32.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:32.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:32 smithi191 ceph-mon[37203]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:32.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T16:15:32.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:32 smithi156 ceph-mon[36502]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T16:15:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:33.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:33 smithi191 ceph-mon[37203]: Deploying daemon osd.2 on smithi191 2024-02-19T16:15:33.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:33 smithi008 ceph-mon[32573]: Deploying daemon osd.2 on smithi191 2024-02-19T16:15:33.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:33 smithi156 ceph-mon[36502]: Deploying daemon osd.2 on smithi191 2024-02-19T16:15:34.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:34 smithi008 ceph-mon[32573]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:34.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:34 smithi191 ceph-mon[37203]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:34.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:34 smithi156 ceph-mon[36502]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:36 smithi008 ceph-mon[32573]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:36.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:36 smithi191 ceph-mon[37203]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:36 smithi156 ceph-mon[36502]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:38.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:38 smithi008 ceph-mon[32573]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:38.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:38 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:38.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:38 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:38.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:38 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:38.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:38 smithi191 ceph-mon[37203]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:38 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:38 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:38 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:38 smithi156 ceph-mon[36502]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:38 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:38 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:38 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:40.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:40 smithi008 ceph-mon[32573]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:40.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:40 smithi191 ceph-mon[37203]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:40 smithi156 ceph-mon[36502]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:41.127 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.126+0000 7f52aa7fc700 1 -- 172.21.15.191:0/4274119136 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f52b4062420 con 0x7f52a00a20c0 2024-02-19T16:15:41.127 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 2 on host 'smithi191' 2024-02-19T16:15:41.129 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.128+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f52a00a20c0 msgr2=0x7f52a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.128+0000 7f52bb355700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f52a00a20c0 0x7f52a00a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52b406b760 tx=0x7f52a4008040 comp rx=0 tx=0).stop 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.128+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 msgr2=0x7f52b406b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.128+0000 7f52bb355700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b406b1e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52b000f690 tx=0x7f52b000f6c0 comp rx=0 tx=0).stop 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 shutdown_connections 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f52a00a20c0 0x7f52a00a4580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52b4102a30 0x7f52b41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b4100f10 0x7f52b406b1e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 --2- 172.21.15.191:0/4274119136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52b4102070 0x7f52b419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 >> 172.21.15.191:0/4274119136 conn(0x7f52b40fc4b0 msgr2=0x7f52b4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 shutdown_connections 2024-02-19T16:15:41.130 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:15:41.129+0000 7f52bb355700 1 -- 172.21.15.191:0/4274119136 wait complete. 2024-02-19T16:15:41.753 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.754 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:41.779 DEBUG:teuthology.orchestra.run.smithi191:osd.2> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.2.service 2024-02-19T16:15:41.781 INFO:tasks.cephadm:Deploying osd.3 on smithi008 with /dev/vg_nvme/lv_3... 2024-02-19T16:15:41.781 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-19T16:15:41.810 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.810 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.810 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:41.810 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:41.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:15:42.031 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:15:42.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:42 smithi008 ceph-mon[32573]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:42.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:42 smithi008 ceph-mon[32573]: from='osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T16:15:42.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:42 smithi008 ceph-mon[32573]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T16:15:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:42 smithi156 ceph-mon[36502]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:42 smithi156 ceph-mon[36502]: from='osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T16:15:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:42 smithi156 ceph-mon[36502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T16:15:42.977 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:42 smithi191 ceph-mon[37203]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:42.977 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:42 smithi191 ceph-mon[37203]: from='osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T16:15:42.977 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:42 smithi191 ceph-mon[37203]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T16:15:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:43 smithi008 ceph-mon[32573]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T16:15:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:43 smithi008 ceph-mon[32573]: osdmap e16: 3 total, 2 up, 3 in 2024-02-19T16:15:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:43 smithi008 ceph-mon[32573]: from='osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:15:43.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:43 smithi008 ceph-mon[32573]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:15:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:43 smithi156 ceph-mon[36502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T16:15:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:43 smithi156 ceph-mon[36502]: osdmap e16: 3 total, 2 up, 3 in 2024-02-19T16:15:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:43 smithi156 ceph-mon[36502]: from='osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:15:43.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:43 smithi156 ceph-mon[36502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:15:44.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:43 smithi191 ceph-mon[37203]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T16:15:44.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:43 smithi191 ceph-mon[37203]: osdmap e16: 3 total, 2 up, 3 in 2024-02-19T16:15:44.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:43 smithi191 ceph-mon[37203]: from='osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:15:44.021 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:43 smithi191 ceph-mon[37203]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:15:44.066 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-19T16:15:44.066 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-19T16:15:44.066 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-02-19T16:15:44.066 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-02-19T16:15:44.066 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00841222 s, 1.2 GB/s 2024-02-19T16:15:44.067 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-02-19T16:15:44.277 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2[42376]: 2024-02-19T16:15:44.018+0000 7fbb35f0a700 -1 osd.2 0 waiting for initial osdmap 2024-02-19T16:15:44.277 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2[42376]: 2024-02-19T16:15:44.022+0000 7fbb2dcf5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:15:44.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:44 smithi008 ceph-mon[32573]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:44.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:44 smithi008 ceph-mon[32573]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:15:44.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:44 smithi008 ceph-mon[32573]: osdmap e17: 3 total, 2 up, 3 in 2024-02-19T16:15:44.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:44 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:44 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:44 smithi008 ceph-mon[32573]: from='osd.2 ' entity='osd.2' 2024-02-19T16:15:44.736 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_3 2024-02-19T16:15:44.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:44 smithi156 ceph-mon[36502]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:44.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:44 smithi156 ceph-mon[36502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:15:44.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:44 smithi156 ceph-mon[36502]: osdmap e17: 3 total, 2 up, 3 in 2024-02-19T16:15:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:44 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:44 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:44 smithi156 ceph-mon[36502]: from='osd.2 ' entity='osd.2' 2024-02-19T16:15:44.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-mon[37203]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:44.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-mon[37203]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:15:44.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-mon[37203]: osdmap e17: 3 total, 2 up, 3 in 2024-02-19T16:15:44.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:44.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:44 smithi191 ceph-mon[37203]: from='osd.2 ' entity='osd.2' 2024-02-19T16:15:45.024 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:15:45.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] boot 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: osdmap e18: 3 total, 3 up, 3 in 2024-02-19T16:15:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] boot 2024-02-19T16:15:45.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: osdmap e18: 3 total, 3 up, 3 in 2024-02-19T16:15:45.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: osd.2 [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] boot 2024-02-19T16:15:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: osdmap e18: 3 total, 3 up, 3 in 2024-02-19T16:15:46.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:15:46.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 -- 172.21.15.8:0/3514863110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 msgr2=0x7ff520102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 --2- 172.21.15.8:0/3514863110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520102430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5100099c0 tx=0x7ff510009cd0 comp rx=0 tx=0).stop 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 -- 172.21.15.8:0/3514863110 shutdown_connections 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 --2- 172.21.15.8:0/3514863110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103bb0 0x7ff52010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 --2- 172.21.15.8:0/3514863110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520102430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 --2- 172.21.15.8:0/3514863110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103210 0x7ff520103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 -- 172.21.15.8:0/3514863110 >> 172.21.15.8:0/3514863110 conn(0x7ff5200fd5f0 msgr2=0x7ff5200ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.670+0000 7ff52522f700 1 -- 172.21.15.8:0/3514863110 shutdown_connections 2024-02-19T16:15:46.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.671+0000 7ff52522f700 1 -- 172.21.15.8:0/3514863110 wait complete. 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.671+0000 7ff52522f700 1 Processor -- start 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.671+0000 7ff52522f700 1 -- start start 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff52522f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff52522f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103210 0x7ff5201985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff52522f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103bb0 0x7ff52019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff52522f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52019fc20 con 0x7ff520102010 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff52522f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52019fd60 con 0x7ff520103210 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff52522f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52019fef0 con 0x7ff520103bb0 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51624/0 (socket says 172.21.15.8:51624) 2024-02-19T16:15:46.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 -- 172.21.15.8:0/840370421 learned_addr learned my addr 172.21.15.8:0/840370421 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51f59e700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103bb0 0x7ff52019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51e59c700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103210 0x7ff5201985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 -- 172.21.15.8:0/840370421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103bb0 msgr2=0x7ff52019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103bb0 0x7ff52019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 -- 172.21.15.8:0/840370421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103210 msgr2=0x7ff5201985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103210 0x7ff5201985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 -- 172.21.15.8:0/840370421 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100095f0 con 0x7ff520102010 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51f59e700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103bb0 0x7ff52019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51e59c700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103210 0x7ff5201985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.672+0000 7ff51ed9d700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520198090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff510003790 tx=0x7ff510003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.673+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51001d070 con 0x7ff520102010 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.673+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff510022470 con 0x7ff520102010 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.673+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5201a0170 con 0x7ff520102010 2024-02-19T16:15:46.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.673+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51000f6c0 con 0x7ff520102010 2024-02-19T16:15:46.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.673+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201a0690 con 0x7ff520102010 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.674+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520109450 con 0x7ff520102010 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.675+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 15) v1 ==== 136711+0+0 (secure 0 0 0) 0x7ff5100225e0 con 0x7ff520102010 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.675+0000 7ff507fff700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff5080a20c0 0x7ff5080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.675+0000 7ff51e59c700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff5080a20c0 0x7ff5080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.675+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7ff5100ce2c0 con 0x7ff520102010 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.676+0000 7ff51e59c700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff5080a20c0 0x7ff5080a4580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5140099d0 tx=0x7ff514008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:15:46.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.679+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff510098950 con 0x7ff520102010 2024-02-19T16:15:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:46 smithi008 ceph-mon[32573]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:46 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi191 2024-02-19T16:15:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:46 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi191 to 17080M 2024-02-19T16:15:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:46 smithi008 ceph-mon[32573]: osdmap e19: 3 total, 3 up, 3 in 2024-02-19T16:15:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:46 smithi156 ceph-mon[36502]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:46 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi191 2024-02-19T16:15:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:46 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi191 to 17080M 2024-02-19T16:15:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:46 smithi156 ceph-mon[36502]: osdmap e19: 3 total, 3 up, 3 in 2024-02-19T16:15:46.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:46.895+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff520062420 con 0x7ff5080a20c0 2024-02-19T16:15:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:46 smithi191 ceph-mon[37203]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-02-19T16:15:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:46 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi191 2024-02-19T16:15:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:46 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi191 to 17080M 2024-02-19T16:15:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:46 smithi191 ceph-mon[37203]: osdmap e19: 3 total, 3 up, 3 in 2024-02-19T16:15:47.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:15:47.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:47.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:47.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:47.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:15:47.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:47.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:47.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:48.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:15:48.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:15:48.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:15:48.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:48.891 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:48 smithi008 ceph-mon[32573]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:48.891 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:48 smithi008 ceph-mon[32573]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:48.891 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-19T16:15:48.892 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:48 smithi008 ceph-mon[32573]: osdmap e20: 3 total, 3 up, 3 in 2024-02-19T16:15:48.892 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:15:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:48 smithi191 ceph-mon[37203]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:48 smithi191 ceph-mon[37203]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-19T16:15:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:48 smithi191 ceph-mon[37203]: osdmap e20: 3 total, 3 up, 3 in 2024-02-19T16:15:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:15:49.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:48 smithi156 ceph-mon[36502]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:49.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:48 smithi156 ceph-mon[36502]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:15:49.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-19T16:15:49.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:48 smithi156 ceph-mon[36502]: osdmap e20: 3 total, 3 up, 3 in 2024-02-19T16:15:49.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:15:49.198 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:48 smithi008 sudo[50044]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T16:15:49.198 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:48 smithi008 sudo[50044]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:49.198 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:48 smithi008 sudo[50044]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:49.198 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:48 smithi008 sudo[50044]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:49.519 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:49 smithi008 sudo[50052]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T16:15:49.519 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:49 smithi008 sudo[50052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:49.519 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:49 smithi008 sudo[50052]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:49.520 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:15:49 smithi008 sudo[50052]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:49.723 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44815]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T16:15:49.723 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:49.723 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44815]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:49.723 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44815]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:49.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:49 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-19T16:15:49.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:49 smithi156 ceph-mon[36502]: osdmap e21: 3 total, 3 up, 3 in 2024-02-19T16:15:49.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:49 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:49.988 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:49 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-19T16:15:49.988 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:49 smithi191 ceph-mon[37203]: osdmap e21: 3 total, 3 up, 3 in 2024-02-19T16:15:49.988 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:49 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:50.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-19T16:15:50.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:49 smithi008 ceph-mon[32573]: osdmap e21: 3 total, 3 up, 3 in 2024-02-19T16:15:50.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:50.079 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44818]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T16:15:50.079 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:50.079 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44818]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:50.079 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:15:49 smithi156 sudo[44818]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:50.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:15:50.235+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff510098d20 con 0x7ff520102010 2024-02-19T16:15:50.242 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:49 smithi191 sudo[46016]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T16:15:50.242 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:49 smithi191 sudo[46016]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:50.242 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:49 smithi191 sudo[46016]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:50.243 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:50 smithi191 sudo[46016]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:50.526 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:50 smithi191 sudo[46019]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T16:15:50.527 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:50 smithi191 sudo[46019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:50.527 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:50 smithi191 sudo[46019]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:50.527 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:15:50 smithi191 sudo[46019]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:50.784 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50131]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T16:15:50.784 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50131]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:50.784 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50131]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:50.784 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50131]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:51.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:51.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: osdmap e22: 3 total, 3 up, 3 in 2024-02-19T16:15:51.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3754675262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8fecdd8-fbd2-4e72-8607-dedc9f0ff018"}]: dispatch 2024-02-19T16:15:51.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3754675262' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8fecdd8-fbd2-4e72-8607-dedc9f0ff018"}]': finished 2024-02-19T16:15:51.045 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: osdmap e23: 4 total, 3 up, 4 in 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:50 smithi191 ceph-mon[37203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46022]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46022]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46022]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:51.046 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46022]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: osdmap e22: 3 total, 3 up, 3 in 2024-02-19T16:15:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3754675262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8fecdd8-fbd2-4e72-8607-dedc9f0ff018"}]: dispatch 2024-02-19T16:15:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3754675262' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8fecdd8-fbd2-4e72-8607-dedc9f0ff018"}]': finished 2024-02-19T16:15:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: osdmap e23: 4 total, 3 up, 4 in 2024-02-19T16:15:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:15:51.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:51.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-19T16:15:51.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:51.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:50 smithi156 ceph-mon[36502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50134]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T16:15:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50134]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50134]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 sudo[50134]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: osdmap e22: 3 total, 3 up, 3 in 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3754675262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8fecdd8-fbd2-4e72-8607-dedc9f0ff018"}]: dispatch 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3754675262' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8fecdd8-fbd2-4e72-8607-dedc9f0ff018"}]': finished 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: osdmap e23: 4 total, 3 up, 4 in 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:15:51.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:50 smithi008 ceph-mon[32573]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:51.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46025]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T16:15:51.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:51.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46025]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:51.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 sudo[46025]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:51.771 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44821]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T16:15:51.771 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:51.771 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44821]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:51.771 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44821]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:51.772 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44824]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T16:15:51.772 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T16:15:51.772 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44824]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T16:15:51.772 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 sudo[44824]: pam_unix(sudo:session): session closed for user root 2024-02-19T16:15:52.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:52.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3210826345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:51 smithi156 ceph-mon[36502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3210826345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:51 smithi008 ceph-mon[32573]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3210826345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:15:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T16:15:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:15:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:15:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:15:52.280 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:51 smithi191 ceph-mon[37203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T16:15:54.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:53 smithi156 ceph-mon[36502]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:54.159 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:53 smithi008 ceph-mon[32573]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:54.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:53 smithi191 ceph-mon[37203]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:56.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:55 smithi156 ceph-mon[36502]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:56.097 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:55 smithi008 ceph-mon[32573]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:56.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:55 smithi191 ceph-mon[37203]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:15:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:57 smithi156 ceph-mon[36502]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-02-19T16:15:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T16:15:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:57 smithi156 ceph-mon[36502]: Deploying daemon osd.3 on smithi008 2024-02-19T16:15:58.214 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:57 smithi008 ceph-mon[32573]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-02-19T16:15:58.214 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T16:15:58.214 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:58.214 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:57 smithi008 ceph-mon[32573]: Deploying daemon osd.3 on smithi008 2024-02-19T16:15:58.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:57 smithi191 ceph-mon[37203]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-02-19T16:15:58.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T16:15:58.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:15:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:57 smithi191 ceph-mon[37203]: Deploying daemon osd.3 on smithi008 2024-02-19T16:15:59.948 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:15:59 smithi008 ceph-mon[32573]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-19T16:16:00.079 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:15:59 smithi191 ceph-mon[37203]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-19T16:16:00.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:15:59 smithi156 ceph-mon[36502]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-02-19T16:16:00.946 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:00 smithi008 ceph-mon[32573]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-19T16:16:01.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:00 smithi156 ceph-mon[36502]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-19T16:16:01.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:00 smithi191 ceph-mon[37203]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-02-19T16:16:03.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:02 smithi008 ceph-mon[32573]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-19T16:16:03.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:02 smithi191 ceph-mon[37203]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-19T16:16:03.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:02 smithi156 ceph-mon[36502]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-19T16:16:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:04.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:04.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:04.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:04.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:04.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:04.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:05.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:05 smithi008 ceph-mon[32573]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-19T16:16:05.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:05 smithi191 ceph-mon[37203]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-19T16:16:05.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:05 smithi156 ceph-mon[36502]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-19T16:16:07.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.141+0000 7ff507fff700 1 -- 172.21.15.8:0/840370421 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff520062420 con 0x7ff5080a20c0 2024-02-19T16:16:07.142 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 3 on host 'smithi008' 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff5080a20c0 msgr2=0x7ff5080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff5080a20c0 0x7ff5080a4580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5140099d0 tx=0x7ff514008040 comp rx=0 tx=0).stop 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 msgr2=0x7ff520198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520198090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff510003790 tx=0x7ff510003960 comp rx=0 tx=0).stop 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 shutdown_connections 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff5080a20c0 0x7ff5080a4580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520103bb0 0x7ff52019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff520102010 0x7ff520198090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:07.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 --2- 172.21.15.8:0/840370421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff520103210 0x7ff5201985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:07.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 >> 172.21.15.8:0/840370421 conn(0x7ff5200fd5f0 msgr2=0x7ff520106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:16:07.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 shutdown_connections 2024-02-19T16:16:07.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:16:07.143+0000 7ff52522f700 1 -- 172.21.15.8:0/840370421 wait complete. 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:07.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:07 smithi008 ceph-mon[32573]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:07.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:07 smithi191 ceph-mon[37203]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T16:16:07.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:07.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:07.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-02-19T16:16:07.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:07.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:07.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:07 smithi156 ceph-mon[36502]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T16:16:07.883 DEBUG:teuthology.orchestra.run.smithi008:osd.3> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.3.service 2024-02-19T16:16:07.885 INFO:tasks.cephadm:Deploying osd.4 on smithi156 with /dev/vg_nvme/lv_3... 2024-02-19T16:16:07.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-19T16:16:08.130 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:16:09.442 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:09 smithi156 ceph-mon[36502]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T16:16:09.442 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:09 smithi156 ceph-mon[36502]: osdmap e24: 4 total, 3 up, 4 in 2024-02-19T16:16:09.442 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:09 smithi156 ceph-mon[36502]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:16:09.442 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:09 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:09.442 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:09 smithi156 ceph-mon[36502]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:16:09.446 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-mon[32573]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T16:16:09.446 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-mon[32573]: osdmap e24: 4 total, 3 up, 4 in 2024-02-19T16:16:09.446 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-mon[32573]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:16:09.446 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:09.446 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-mon[32573]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:16:09.446 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3[52114]: 2024-02-19T16:16:09.147+0000 7f841c912700 -1 osd.3 0 waiting for initial osdmap 2024-02-19T16:16:09.447 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:16:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3[52114]: 2024-02-19T16:16:09.153+0000 7f8418705700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:16:09.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:09 smithi191 ceph-mon[37203]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T16:16:09.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:09 smithi191 ceph-mon[37203]: osdmap e24: 4 total, 3 up, 4 in 2024-02-19T16:16:09.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:09 smithi191 ceph-mon[37203]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:16:09.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:09 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:09.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:09 smithi191 ceph-mon[37203]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:16:09.985 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-19T16:16:09.985 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-19T16:16:09.986 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-19T16:16:09.986 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-19T16:16:09.986 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832646 s, 1.3 GB/s 2024-02-19T16:16:09.986 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-19T16:16:10.515 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:10 smithi156 ceph-mon[36502]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:16:10.516 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:10 smithi156 ceph-mon[36502]: osdmap e25: 4 total, 3 up, 4 in 2024-02-19T16:16:10.516 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:10.516 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:10 smithi008 ceph-mon[32573]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:16:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:10 smithi008 ceph-mon[32573]: osdmap e25: 4 total, 3 up, 4 in 2024-02-19T16:16:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:10.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:10 smithi191 ceph-mon[37203]: from='osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:16:10.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:10 smithi191 ceph-mon[37203]: osdmap e25: 4 total, 3 up, 4 in 2024-02-19T16:16:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:10.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2024-02-19T16:16:10.780 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:16:11.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:16:11.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] boot 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: osdmap e26: 4 total, 4 up, 4 in 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:11.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:11 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:16:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:16:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] boot 2024-02-19T16:16:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: osdmap e26: 4 total, 4 up, 4 in 2024-02-19T16:16:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:11 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: osd.3 [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] boot 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: osdmap e26: 4 total, 4 up, 4 in 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:11.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:11.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:11.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:11 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:12.373 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 -- 172.21.15.156:0/3194640119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103370 msgr2=0x7f19e41037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 --2- 172.21.15.156:0/3194640119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103370 0x7f19e41037d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009990 tx=0x7f19d4009ca0 comp rx=0 tx=0).stop 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 -- 172.21.15.156:0/3194640119 shutdown_connections 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 --2- 172.21.15.156:0/3194640119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e4103d10 0x7f19e410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 --2- 172.21.15.156:0/3194640119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 0x7f19e4100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 --2- 172.21.15.156:0/3194640119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103370 0x7f19e41037d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 -- 172.21.15.156:0/3194640119 >> 172.21.15.156:0/3194640119 conn(0x7f19e40fbc50 msgr2=0x7f19e40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 -- 172.21.15.156:0/3194640119 shutdown_connections 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 -- 172.21.15.156:0/3194640119 wait complete. 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 Processor -- start 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.371+0000 7f19eb496700 1 -- start start 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19eb496700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 0x7f19e4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:12.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19eb496700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e4103370 0x7f19e41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19eb496700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 0x7f19e419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19eb496700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419fb30 con 0x7f19e41002f0 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19eb496700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419fc70 con 0x7f19e4103d10 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19eb496700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419fe00 con 0x7f19e4103370 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 0x7f19e419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 0x7f19e419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50190/0 (socket says 172.21.15.156:50190) 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9232700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 0x7f19e4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 -- 172.21.15.156:0/172001548 learned_addr learned my addr 172.21.15.156:0/172001548 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e8a31700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e4103370 0x7f19e41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 -- 172.21.15.156:0/172001548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e4103370 msgr2=0x7f19e41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e4103370 0x7f19e41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:12.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 -- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 msgr2=0x7f19e4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 0x7f19e4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 -- 172.21.15.156:0/172001548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40095f0 con 0x7f19e4103d10 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9232700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 0x7f19e4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19e9a33700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 0x7f19e419f560 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00ef30 tx=0x7f19dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc018070 con 0x7f19e4103d10 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19dc00f040 con 0x7f19e4103d10 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.372+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00ce50 con 0x7f19e4103d10 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.373+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e41a0080 con 0x7f19e4103d10 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.373+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e41a0580 con 0x7f19e4103d10 2024-02-19T16:16:12.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.373+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4109530 con 0x7f19e4103d10 2024-02-19T16:16:12.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.374+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f19dc003a00 con 0x7f19e4103d10 2024-02-19T16:16:12.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.375+0000 7f19da7fc700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f19d00a21d0 0x7f19d00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:12.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.375+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f19dc0ccc00 con 0x7f19e4103d10 2024-02-19T16:16:12.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.375+0000 7f19e9232700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f19d00a21d0 0x7f19d00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:12.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.376+0000 7f19e9232700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f19d00a21d0 0x7f19d00a4690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f19e40699d0 tx=0x7f19e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:16:12.379 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.379+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f19dc096e60 con 0x7f19e4103d10 2024-02-19T16:16:12.458 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:12 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi008 2024-02-19T16:16:12.459 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:12 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi008 to 8546M 2024-02-19T16:16:12.459 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:12 smithi008 ceph-mon[32573]: osdmap e27: 4 total, 4 up, 4 in 2024-02-19T16:16:12.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:12 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi008 2024-02-19T16:16:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:12 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi008 to 8546M 2024-02-19T16:16:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:12 smithi191 ceph-mon[37203]: osdmap e27: 4 total, 4 up, 4 in 2024-02-19T16:16:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:12 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi008 2024-02-19T16:16:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:12 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi008 to 8546M 2024-02-19T16:16:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:12 smithi156 ceph-mon[36502]: osdmap e27: 4 total, 4 up, 4 in 2024-02-19T16:16:12.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:12.582+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f19e402da30 con 0x7f19d00a21d0 2024-02-19T16:16:13.427 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:13 smithi008 ceph-mon[32573]: osdmap e28: 4 total, 4 up, 4 in 2024-02-19T16:16:13.428 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:13 smithi008 ceph-mon[32573]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:16:13.428 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:16:13.428 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:16:13.428 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:13.428 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:13 smithi008 ceph-mon[32573]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:13 smithi156 ceph-mon[36502]: osdmap e28: 4 total, 4 up, 4 in 2024-02-19T16:16:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:13 smithi156 ceph-mon[36502]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:16:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:16:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:16:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:13 smithi156 ceph-mon[36502]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:13 smithi191 ceph-mon[37203]: osdmap e28: 4 total, 4 up, 4 in 2024-02-19T16:16:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:13 smithi191 ceph-mon[37203]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:16:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:16:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:16:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:13 smithi191 ceph-mon[37203]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:14.458 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:14 smithi156 ceph-mon[36502]: osdmap e29: 4 total, 4 up, 4 in 2024-02-19T16:16:14.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:14 smithi008 ceph-mon[32573]: osdmap e29: 4 total, 4 up, 4 in 2024-02-19T16:16:14.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:14 smithi191 ceph-mon[37203]: osdmap e29: 4 total, 4 up, 4 in 2024-02-19T16:16:15.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:15 smithi008 ceph-mon[32573]: osdmap e30: 4 total, 4 up, 4 in 2024-02-19T16:16:15.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:15 smithi008 ceph-mon[32573]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:15.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:15 smithi191 ceph-mon[37203]: osdmap e30: 4 total, 4 up, 4 in 2024-02-19T16:16:15.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:15 smithi191 ceph-mon[37203]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:15.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:15 smithi156 ceph-mon[36502]: osdmap e30: 4 total, 4 up, 4 in 2024-02-19T16:16:15.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:15 smithi156 ceph-mon[36502]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:16.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:16 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/2086863835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]: dispatch 2024-02-19T16:16:16.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:16 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]: dispatch 2024-02-19T16:16:16.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:16 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]': finished 2024-02-19T16:16:16.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:16 smithi008 ceph-mon[32573]: osdmap e31: 5 total, 4 up, 5 in 2024-02-19T16:16:16.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:16.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:16 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/697241899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:16:16.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:16 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/2086863835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]: dispatch 2024-02-19T16:16:16.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:16 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]: dispatch 2024-02-19T16:16:16.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:16 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]': finished 2024-02-19T16:16:16.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:16 smithi191 ceph-mon[37203]: osdmap e31: 5 total, 4 up, 5 in 2024-02-19T16:16:16.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:16.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:16 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/697241899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:16:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:16 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/2086863835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]: dispatch 2024-02-19T16:16:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:16 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]: dispatch 2024-02-19T16:16:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:16 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356b4cc1-0311-4770-9e1f-02809310c1ac"}]': finished 2024-02-19T16:16:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:16 smithi156 ceph-mon[36502]: osdmap e31: 5 total, 4 up, 5 in 2024-02-19T16:16:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:16 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/697241899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:16:17.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:17 smithi008 ceph-mon[32573]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-19T16:16:17.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:17 smithi191 ceph-mon[37203]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-19T16:16:17.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:17 smithi156 ceph-mon[36502]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-19T16:16:19.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:18 smithi008 ceph-mon[32573]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:16:19.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:18 smithi191 ceph-mon[37203]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:16:19.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:18 smithi156 ceph-mon[36502]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:16:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:20 smithi156 ceph-mon[36502]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-19T16:16:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:20 smithi008 ceph-mon[32573]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-19T16:16:21.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:20 smithi191 ceph-mon[37203]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-19T16:16:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T16:16:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:22.910 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T16:16:22.910 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:23.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T16:16:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:23.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:23 smithi156 ceph-mon[36502]: Deploying daemon osd.4 on smithi156 2024-02-19T16:16:23.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:23 smithi156 ceph-mon[36502]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-19T16:16:23.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:23 smithi008 ceph-mon[32573]: Deploying daemon osd.4 on smithi156 2024-02-19T16:16:23.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:23 smithi008 ceph-mon[32573]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-19T16:16:24.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:23 smithi191 ceph-mon[37203]: Deploying daemon osd.4 on smithi156 2024-02-19T16:16:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:23 smithi191 ceph-mon[37203]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-19T16:16:25.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:24 smithi008 ceph-mon[32573]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:16:25.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:24 smithi191 ceph-mon[37203]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:16:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:24 smithi156 ceph-mon[36502]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:16:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:26 smithi008 ceph-mon[32573]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-19T16:16:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:26 smithi191 ceph-mon[37203]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-19T16:16:27.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:26 smithi156 ceph-mon[36502]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-19T16:16:29.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:29.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:29.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:29.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:29 smithi156 ceph-mon[36502]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:29.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:29.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:29.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:29.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:29 smithi008 ceph-mon[32573]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:29.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:29.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:29.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:29.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:29 smithi191 ceph-mon[37203]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:31.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:30 smithi008 ceph-mon[32573]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:31.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:30 smithi191 ceph-mon[37203]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:31.058 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:30 smithi156 ceph-mon[36502]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:31.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.747+0000 7f19da7fc700 1 -- 172.21.15.156:0/172001548 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f19e402da30 con 0x7f19d00a21d0 2024-02-19T16:16:31.750 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 4 on host 'smithi156' 2024-02-19T16:16:31.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f19d00a21d0 msgr2=0x7f19d00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f19d00a21d0 0x7f19d00a4690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f19e40699d0 tx=0x7f19e0009260 comp rx=0 tx=0).stop 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 msgr2=0x7f19e419f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 0x7f19e419f560 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00ef30 tx=0x7f19dc00c3a0 comp rx=0 tx=0).stop 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 shutdown_connections 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f19d00a21d0 0x7f19d00a4690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e4103d10 0x7f19e419f560 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19e41002f0 0x7f19e4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 --2- 172.21.15.156:0/172001548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e4103370 0x7f19e41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 >> 172.21.15.156:0/172001548 conn(0x7f19e40fbc50 msgr2=0x7f19e4106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 shutdown_connections 2024-02-19T16:16:31.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:16:31.750+0000 7f19eb496700 1 -- 172.21.15.156:0/172001548 wait complete. 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:32.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:32.438 DEBUG:teuthology.orchestra.run.smithi156:osd.4> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.4.service 2024-02-19T16:16:32.440 INFO:tasks.cephadm:Deploying osd.5 on smithi191 with /dev/vg_nvme/lv_3... 2024-02-19T16:16:32.440 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-19T16:16:32.685 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:16:32.961 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:32 smithi156 ceph-mon[36502]: from='osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T16:16:32.962 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:32 smithi156 ceph-mon[36502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T16:16:32.962 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:32 smithi156 ceph-mon[36502]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:33.067 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:32 smithi191 ceph-mon[37203]: from='osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T16:16:33.067 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:32 smithi191 ceph-mon[37203]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T16:16:33.068 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:32 smithi191 ceph-mon[37203]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:32 smithi008 ceph-mon[32573]: from='osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T16:16:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:32 smithi008 ceph-mon[32573]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T16:16:33.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:32 smithi008 ceph-mon[32573]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:33 smithi156 ceph-mon[36502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T16:16:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:33 smithi156 ceph-mon[36502]: from='osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:16:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:33 smithi156 ceph-mon[36502]: osdmap e32: 5 total, 4 up, 5 in 2024-02-19T16:16:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:33 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:33 smithi156 ceph-mon[36502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:16:34.126 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:33 smithi191 ceph-mon[37203]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T16:16:34.126 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:33 smithi191 ceph-mon[37203]: from='osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:16:34.126 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:33 smithi191 ceph-mon[37203]: osdmap e32: 5 total, 4 up, 5 in 2024-02-19T16:16:34.126 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:33 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:34.126 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:33 smithi191 ceph-mon[37203]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:16:34.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:33 smithi008 ceph-mon[32573]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T16:16:34.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:33 smithi008 ceph-mon[32573]: from='osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:16:34.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:33 smithi008 ceph-mon[32573]: osdmap e32: 5 total, 4 up, 5 in 2024-02-19T16:16:34.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:33 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:34.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:33 smithi008 ceph-mon[32573]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:16:34.580 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:16:34.412+0000 7fc2f24ec700 -1 osd.4 0 waiting for initial osdmap 2024-02-19T16:16:34.580 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:16:34.421+0000 7fc2ea2d7700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:16:34.674 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-19T16:16:34.674 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-19T16:16:34.674 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-19T16:16:34.674 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-19T16:16:34.674 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112687 s, 931 MB/s 2024-02-19T16:16:34.674 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-19T16:16:35.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:34 smithi008 ceph-mon[32573]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:16:35.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:34 smithi008 ceph-mon[32573]: osdmap e33: 5 total, 4 up, 5 in 2024-02-19T16:16:35.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:34 smithi008 ceph-mon[32573]: from='osd.4 ' entity='osd.4' 2024-02-19T16:16:35.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:34 smithi008 ceph-mon[32573]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:35.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:34 smithi191 ceph-mon[37203]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:16:35.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:34 smithi191 ceph-mon[37203]: osdmap e33: 5 total, 4 up, 5 in 2024-02-19T16:16:35.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:34 smithi191 ceph-mon[37203]: from='osd.4 ' entity='osd.4' 2024-02-19T16:16:35.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:34 smithi191 ceph-mon[37203]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:35.318 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-mon[36502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:16:35.318 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-mon[36502]: osdmap e33: 5 total, 4 up, 5 in 2024-02-19T16:16:35.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-mon[36502]: from='osd.4 ' entity='osd.4' 2024-02-19T16:16:35.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:34 smithi156 ceph-mon[36502]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-02-19T16:16:35.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_3 2024-02-19T16:16:35.598 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:16:35.892 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] boot 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: osdmap e34: 5 total, 5 up, 5 in 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:35.893 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:35.894 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:16:36.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:16:36.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:36.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] boot 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: osdmap e34: 5 total, 5 up, 5 in 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:16:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:16:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: osd.4 [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] boot 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: osdmap e34: 5 total, 5 up, 5 in 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:16:36.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:36.980 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:36 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi156 2024-02-19T16:16:36.980 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:36 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi156 to 8540M 2024-02-19T16:16:36.981 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:36 smithi191 ceph-mon[37203]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:36.981 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:36 smithi191 ceph-mon[37203]: osdmap e35: 5 total, 5 up, 5 in 2024-02-19T16:16:37.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 -- 172.21.15.191:0/3465412424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 msgr2=0x7efd54101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:37.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/3465412424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54101330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd48009990 tx=0x7efd48009ca0 comp rx=0 tx=0).stop 2024-02-19T16:16:37.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 -- 172.21.15.191:0/3465412424 shutdown_connections 2024-02-19T16:16:37.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/3465412424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd54102a30 0x7efd54109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:37.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/3465412424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54101330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:37.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/3465412424 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 0x7efd541024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:37.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 -- 172.21.15.191:0/3465412424 >> 172.21.15.191:0/3465412424 conn(0x7efd540fc4b0 msgr2=0x7efd540fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:16:37.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 -- 172.21.15.191:0/3465412424 shutdown_connections 2024-02-19T16:16:37.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.207+0000 7efd5a0b2700 1 -- 172.21.15.191:0/3465412424 wait complete. 2024-02-19T16:16:37.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 Processor -- start 2024-02-19T16:16:37.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 -- start start 2024-02-19T16:16:37.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:37.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 0x7efd54198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:37.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd54102a30 0x7efd5419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:37.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419fac0 con 0x7efd54102070 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419fc00 con 0x7efd54102a30 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd5a0b2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419fd90 con 0x7efd54100f10 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.208+0000 7efd537fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:49068/0 (socket says 172.21.15.191:49068) 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 -- 172.21.15.191:0/4098204399 learned_addr learned my addr 172.21.15.191:0/4098204399 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd52ffd700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 0x7efd54198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd53fff700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd54102a30 0x7efd5419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 -- 172.21.15.191:0/4098204399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd54102a30 msgr2=0x7efd5419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd54102a30 0x7efd5419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 -- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 msgr2=0x7efd54198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 0x7efd54198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:37.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 -- 172.21.15.191:0/4098204399 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480095f0 con 0x7efd54100f10 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd52ffd700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 0x7efd54198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd537fe700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54197f30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd480038c0 tx=0x7efd4800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4801d070 con 0x7efd54100f10 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd48005530 con 0x7efd54100f10 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd541a0010 con 0x7efd54100f10 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48004cb0 con 0x7efd54100f10 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.209+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd541a0480 con 0x7efd54100f10 2024-02-19T16:16:37.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.210+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd54068ca0 con 0x7efd54100f10 2024-02-19T16:16:37.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.211+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7efd48005050 con 0x7efd54100f10 2024-02-19T16:16:37.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.212+0000 7efd50ff9700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd3c0a2150 0x7efd3c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:16:37.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.212+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7efd480cddb0 con 0x7efd54100f10 2024-02-19T16:16:37.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.212+0000 7efd52ffd700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd3c0a2150 0x7efd3c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:16:37.214 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.212+0000 7efd52ffd700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd3c0a2150 0x7efd3c0a4610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd44005950 tx=0x7efd440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:16:37.217 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.216+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7efd48097e80 con 0x7efd54100f10 2024-02-19T16:16:37.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:36 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi156 2024-02-19T16:16:37.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:36 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi156 to 8540M 2024-02-19T16:16:37.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:36 smithi008 ceph-mon[32573]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:37.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:36 smithi008 ceph-mon[32573]: osdmap e35: 5 total, 5 up, 5 in 2024-02-19T16:16:37.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:36 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi156 2024-02-19T16:16:37.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:36 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi156 to 8540M 2024-02-19T16:16:37.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:36 smithi156 ceph-mon[36502]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:37.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:36 smithi156 ceph-mon[36502]: osdmap e35: 5 total, 5 up, 5 in 2024-02-19T16:16:37.420 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:37.419+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7efd540fe660 con 0x7efd3c0a2150 2024-02-19T16:16:38.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:37 smithi191 ceph-mon[37203]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:16:38.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:37 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:16:38.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:37 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:16:38.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:37 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:37 smithi008 ceph-mon[32573]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:16:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:37 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:16:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:37 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:16:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:37 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:38.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:37 smithi156 ceph-mon[36502]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:16:38.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:37 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:16:38.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:37 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:16:38.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:37 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:39.171 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:38 smithi191 ceph-mon[37203]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:39.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:38 smithi008 ceph-mon[32573]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:39.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:38 smithi156 ceph-mon[36502]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:40.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/3727872376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]: dispatch 2024-02-19T16:16:40.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:40 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]: dispatch 2024-02-19T16:16:40.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:40 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]': finished 2024-02-19T16:16:40.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:40 smithi191 ceph-mon[37203]: osdmap e36: 6 total, 5 up, 6 in 2024-02-19T16:16:40.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:40 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:16:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/3727872376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]: dispatch 2024-02-19T16:16:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:40 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]: dispatch 2024-02-19T16:16:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:40 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]': finished 2024-02-19T16:16:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:40 smithi008 ceph-mon[32573]: osdmap e36: 6 total, 5 up, 6 in 2024-02-19T16:16:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:40 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:16:40.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/3727872376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]: dispatch 2024-02-19T16:16:40.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:40 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]: dispatch 2024-02-19T16:16:40.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:40 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca5e23ef-499f-4cca-ac7a-d85f36ffbedb"}]': finished 2024-02-19T16:16:40.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:40 smithi156 ceph-mon[36502]: osdmap e36: 6 total, 5 up, 6 in 2024-02-19T16:16:40.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:40 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:16:41.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/2816644974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:16:41.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/2816644974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:16:41.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/2816644974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:16:42.436 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:42 smithi008 ceph-mon[32573]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:42.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:42 smithi191 ceph-mon[37203]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:42.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:42 smithi156 ceph-mon[36502]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:44.439 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:44 smithi191 ceph-mon[37203]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:44.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:44 smithi008 ceph-mon[32573]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:44.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:44 smithi156 ceph-mon[36502]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:46.330 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:46 smithi191 ceph-mon[37203]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:46.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:46 smithi008 ceph-mon[32573]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:46.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:46 smithi156 ceph-mon[36502]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:48.514 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:48 smithi191 ceph-mon[37203]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:48.514 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T16:16:48.514 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:48.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:48 smithi008 ceph-mon[32573]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:48.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T16:16:48.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:48.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:48 smithi156 ceph-mon[36502]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:48.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T16:16:48.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:16:49.503 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:49 smithi191 ceph-mon[37203]: Deploying daemon osd.5 on smithi191 2024-02-19T16:16:49.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:49 smithi008 ceph-mon[32573]: Deploying daemon osd.5 on smithi191 2024-02-19T16:16:49.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:49 smithi156 ceph-mon[36502]: Deploying daemon osd.5 on smithi191 2024-02-19T16:16:50.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:50 smithi008 ceph-mon[32573]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:50.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:50 smithi191 ceph-mon[37203]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:50.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:50 smithi156 ceph-mon[36502]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:52 smithi008 ceph-mon[32573]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:52.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:52 smithi191 ceph-mon[37203]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:52.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:52 smithi156 ceph-mon[36502]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:54 smithi008 ceph-mon[32573]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:54.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:54 smithi191 ceph-mon[37203]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:54.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:54 smithi156 ceph-mon[36502]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:54.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:54.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:54.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:16:56.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:56 smithi191 ceph-mon[37203]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:56 smithi008 ceph-mon[32573]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:56.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:56 smithi156 ceph-mon[36502]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:56.938 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.936+0000 7efd50ff9700 1 -- 172.21.15.191:0/4098204399 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efd540fe660 con 0x7efd3c0a2150 2024-02-19T16:16:56.938 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 5 on host 'smithi191' 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd3c0a2150 msgr2=0x7efd3c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd3c0a2150 0x7efd3c0a4610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd44005950 tx=0x7efd440058e0 comp rx=0 tx=0).stop 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 msgr2=0x7efd54197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54197f30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd480038c0 tx=0x7efd4800f740 comp rx=0 tx=0).stop 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 shutdown_connections 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd3c0a2150 0x7efd3c0a4610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd54102a30 0x7efd5419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd54100f10 0x7efd54197f30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 --2- 172.21.15.191:0/4098204399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd54102070 0x7efd54198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 >> 172.21.15.191:0/4098204399 conn(0x7efd540fc4b0 msgr2=0x7efd540fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 shutdown_connections 2024-02-19T16:16:56.940 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:16:56.939+0000 7efd5a0b2700 1 -- 172.21.15.191:0/4098204399 wait complete. 2024-02-19T16:16:57.711 DEBUG:teuthology.orchestra.run.smithi191:osd.5> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.5.service 2024-02-19T16:16:57.713 INFO:tasks.cephadm:Deploying osd.6 on smithi008 with /dev/vg_nvme/lv_2... 2024-02-19T16:16:57.713 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-19T16:16:57.976 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:16:58.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:58.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: from='osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T16:16:58.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T16:16:58.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:57 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: from='osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T16:16:58.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T16:16:58.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:57 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:16:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: from='osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T16:16:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T16:16:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:57 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:16:59.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-mon[37203]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T16:16:59.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-mon[37203]: osdmap e37: 6 total, 5 up, 6 in 2024-02-19T16:16:59.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-mon[37203]: from='osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:16:59.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-mon[37203]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:16:59.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:16:59.027 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:16:58.935+0000 7f0d895e1700 -1 osd.5 0 waiting for initial osdmap 2024-02-19T16:16:59.027 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:16:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:16:58.941+0000 7f0d853d4700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:16:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:58 smithi008 ceph-mon[32573]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T16:16:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:58 smithi008 ceph-mon[32573]: osdmap e37: 6 total, 5 up, 6 in 2024-02-19T16:16:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:58 smithi008 ceph-mon[32573]: from='osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:16:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:58 smithi008 ceph-mon[32573]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:16:59.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:16:59.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:58 smithi156 ceph-mon[36502]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T16:16:59.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:58 smithi156 ceph-mon[36502]: osdmap e37: 6 total, 5 up, 6 in 2024-02-19T16:16:59.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:58 smithi156 ceph-mon[36502]: from='osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:16:59.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:58 smithi156 ceph-mon[36502]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:16:59.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:16:59.997 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-19T16:16:59.997 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-19T16:16:59.997 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-02-19T16:16:59.997 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-02-19T16:16:59.997 INFO:teuthology.orchestra.run.smithi008.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00816602 s, 1.3 GB/s 2024-02-19T16:16:59.997 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 2024-02-19T16:16:59.998 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: osdmap e38: 6 total, 5 up, 6 in 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] boot 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: osdmap e39: 6 total, 6 up, 6 in 2024-02-19T16:17:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:16:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:17:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:17:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: osdmap e38: 6 total, 5 up, 6 in 2024-02-19T16:17:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] boot 2024-02-19T16:17:00.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: osdmap e39: 6 total, 6 up, 6 in 2024-02-19T16:17:00.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:16:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: osdmap e38: 6 total, 5 up, 6 in 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: osd.5 [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] boot 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: osdmap e39: 6 total, 6 up, 6 in 2024-02-19T16:17:00.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:16:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:17:00.713 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_2 2024-02-19T16:17:00.956 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:17:01.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:00 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:17:01.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:00 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:17:01.205 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:00 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:17:01.205 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:00 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:17:01.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:00 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:17:01.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:00 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:17:02.109 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:02.109 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: osdmap e40: 6 total, 6 up, 6 in 2024-02-19T16:17:02.109 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi191 2024-02-19T16:17:02.109 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi191 to 8540M 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:02.110 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.230 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: osdmap e40: 6 total, 6 up, 6 in 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi191 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi191 to 8540M 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.231 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:02.232 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:02.232 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:02.232 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: osdmap e40: 6 total, 6 up, 6 in 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi191 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi191 to 8540M 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:02.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:02.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:02.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 -- 172.21.15.8:0/3777606912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 msgr2=0x7f21f410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 --2- 172.21.15.8:0/3777606912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f410a170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f21e400b0a0 tx=0x7f21e400b3b0 comp rx=0 tx=0).stop 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 -- 172.21.15.8:0/3777606912 shutdown_connections 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 --2- 172.21.15.8:0/3777606912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f410a170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 --2- 172.21.15.8:0/3777606912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21f4102b00 0x7f21f4104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 --2- 172.21.15.8:0/3777606912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21f4105430 0x7f21f4107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:02.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 -- 172.21.15.8:0/3777606912 >> 172.21.15.8:0/3777606912 conn(0x7f21f40fc4b0 msgr2=0x7f21f40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 -- 172.21.15.8:0/3777606912 shutdown_connections 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.689+0000 7f21fc17b700 1 -- 172.21.15.8:0/3777606912 wait complete. 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 Processor -- start 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 -- start start 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21f4102b00 0x7f21f4102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21f4105430 0x7f21f4100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f4100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4101770 con 0x7f21f4107d60 2024-02-19T16:17:02.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41018e0 con 0x7f21f4102b00 2024-02-19T16:17:02.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fc17b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41a20d0 con 0x7f21f4105430 2024-02-19T16:17:02.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f4100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:02.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f4100ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54150/0 (socket says 172.21.15.8:54150) 2024-02-19T16:17:02.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 -- 172.21.15.8:0/1767598000 learned_addr learned my addr 172.21.15.8:0/1767598000 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:17:02.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21f9716700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21f4105430 0x7f21f4100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:02.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21f9f17700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21f4102b00 0x7f21f4102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:02.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 -- 172.21.15.8:0/1767598000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21f4105430 msgr2=0x7f21f4100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:02.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21f4105430 0x7f21f4100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:02.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 -- 172.21.15.8:0/1767598000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21f4102b00 msgr2=0x7f21f4102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:02.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21f4102b00 0x7f21f4102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:02.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.690+0000 7f21fa718700 1 -- 172.21.15.8:0/1767598000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4009d00 con 0x7f21f4107d60 2024-02-19T16:17:02.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.691+0000 7f21fa718700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f4100ff0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009850 tx=0x7f21e40079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:17:02.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.691+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e400e040 con 0x7f21f4107d60 2024-02-19T16:17:02.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.691+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21e4004c50 con 0x7f21f4107d60 2024-02-19T16:17:02.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.691+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40052a0 con 0x7f21f4107d60 2024-02-19T16:17:02.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.691+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f41a2350 con 0x7f21f4107d60 2024-02-19T16:17:02.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.691+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f41a27c0 con 0x7f21f4107d60 2024-02-19T16:17:02.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.692+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f40fe030 con 0x7f21f4107d60 2024-02-19T16:17:02.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.693+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f21e401d470 con 0x7f21f4107d60 2024-02-19T16:17:02.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.693+0000 7f21f2ffd700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f21dc0a2150 0x7f21dc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:02.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.694+0000 7f21f9f17700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f21dc0a2150 0x7f21dc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:02.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.694+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f21e40ce6b0 con 0x7f21f4107d60 2024-02-19T16:17:02.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.694+0000 7f21f9f17700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f21dc0a2150 0x7f21dc0a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009c60 tx=0x7f21ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:17:02.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.697+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f21e4098510 con 0x7f21f4107d60 2024-02-19T16:17:02.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:02.910+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f21f402d990 con 0x7f21dc0a2150 2024-02-19T16:17:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:02 smithi191 ceph-mon[37203]: osdmap e41: 6 total, 6 up, 6 in 2024-02-19T16:17:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:02 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:02 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:02 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:03.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:02 smithi008 ceph-mon[32573]: osdmap e41: 6 total, 6 up, 6 in 2024-02-19T16:17:03.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:02 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:03.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:02 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:03.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:02 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:03.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:02 smithi156 ceph-mon[36502]: osdmap e41: 6 total, 6 up, 6 in 2024-02-19T16:17:03.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:02 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:03.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:02 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:03.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:02 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:03 smithi008 ceph-mon[32573]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:03 smithi008 ceph-mon[32573]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:03 smithi008 ceph-mon[32573]: osdmap e42: 6 total, 6 up, 6 in 2024-02-19T16:17:04.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:03 smithi191 ceph-mon[37203]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:04.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:03 smithi191 ceph-mon[37203]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:04.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:03 smithi191 ceph-mon[37203]: osdmap e42: 6 total, 6 up, 6 in 2024-02-19T16:17:04.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:03 smithi156 ceph-mon[36502]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:04.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:03 smithi156 ceph-mon[36502]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:04.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:03 smithi156 ceph-mon[36502]: osdmap e42: 6 total, 6 up, 6 in 2024-02-19T16:17:05.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:04 smithi008 ceph-mon[32573]: osdmap e43: 6 total, 6 up, 6 in 2024-02-19T16:17:05.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:04 smithi191 ceph-mon[37203]: osdmap e43: 6 total, 6 up, 6 in 2024-02-19T16:17:05.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:04 smithi156 ceph-mon[36502]: osdmap e43: 6 total, 6 up, 6 in 2024-02-19T16:17:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:05 smithi008 ceph-mon[32573]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:05 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2845115129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84d121f0-81a1-4702-bd47-4c34f40b5b9f"}]: dispatch 2024-02-19T16:17:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:05 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2845115129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84d121f0-81a1-4702-bd47-4c34f40b5b9f"}]': finished 2024-02-19T16:17:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:05 smithi008 ceph-mon[32573]: osdmap e44: 7 total, 6 up, 7 in 2024-02-19T16:17:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:05 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:06.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:05 smithi191 ceph-mon[37203]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:06.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2845115129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84d121f0-81a1-4702-bd47-4c34f40b5b9f"}]: dispatch 2024-02-19T16:17:06.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2845115129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84d121f0-81a1-4702-bd47-4c34f40b5b9f"}]': finished 2024-02-19T16:17:06.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:05 smithi191 ceph-mon[37203]: osdmap e44: 7 total, 6 up, 7 in 2024-02-19T16:17:06.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:05 smithi156 ceph-mon[36502]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:05 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2845115129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84d121f0-81a1-4702-bd47-4c34f40b5b9f"}]: dispatch 2024-02-19T16:17:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:05 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2845115129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84d121f0-81a1-4702-bd47-4c34f40b5b9f"}]': finished 2024-02-19T16:17:06.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:05 smithi156 ceph-mon[36502]: osdmap e44: 7 total, 6 up, 7 in 2024-02-19T16:17:06.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:05 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:07.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:06 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3811120057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:07.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:06 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3811120057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:07.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:06 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3811120057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:07 smithi008 ceph-mon[32573]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-19T16:17:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:07 smithi191 ceph-mon[37203]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-19T16:17:08.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:07 smithi156 ceph-mon[36502]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-19T16:17:10.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:09 smithi008 ceph-mon[32573]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:17:10.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:09 smithi191 ceph-mon[37203]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:17:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:09 smithi156 ceph-mon[36502]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:17:12.243 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:11 smithi008 ceph-mon[32573]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-19T16:17:12.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:11 smithi191 ceph-mon[37203]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-19T16:17:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:11 smithi156 ceph-mon[36502]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-19T16:17:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:13 smithi191 ceph-mon[37203]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-02-19T16:17:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-19T16:17:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:13 smithi191 ceph-mon[37203]: Deploying daemon osd.6 on smithi008 2024-02-19T16:17:14.174 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:13 smithi008 ceph-mon[32573]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-02-19T16:17:14.174 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-19T16:17:14.175 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:14.175 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:13 smithi008 ceph-mon[32573]: Deploying daemon osd.6 on smithi008 2024-02-19T16:17:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:13 smithi156 ceph-mon[36502]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-02-19T16:17:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-19T16:17:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:14.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:13 smithi156 ceph-mon[36502]: Deploying daemon osd.6 on smithi008 2024-02-19T16:17:16.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:15 smithi008 ceph-mon[32573]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:17:16.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:15 smithi191 ceph-mon[37203]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:17:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:15 smithi156 ceph-mon[36502]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:17:18.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:17 smithi008 ceph-mon[32573]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-02-19T16:17:18.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:17 smithi191 ceph-mon[37203]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-02-19T16:17:18.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:17 smithi156 ceph-mon[36502]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-02-19T16:17:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:20 smithi008 ceph-mon[32573]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:20.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:20 smithi191 ceph-mon[37203]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:20.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:20.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:20.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:20 smithi156 ceph-mon[36502]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:22.685 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 6 on host 'smithi008' 2024-02-19T16:17:22.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.684+0000 7f21f2ffd700 1 -- 172.21.15.8:0/1767598000 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f21f402d990 con 0x7f21dc0a2150 2024-02-19T16:17:22.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f21dc0a2150 msgr2=0x7f21dc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:22.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f21dc0a2150 0x7f21dc0a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009c60 tx=0x7f21ec009260 comp rx=0 tx=0).stop 2024-02-19T16:17:22.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 msgr2=0x7f21f4100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:22.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f4100ff0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009850 tx=0x7f21e40079b0 comp rx=0 tx=0).stop 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 shutdown_connections 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f21dc0a2150 0x7f21dc0a4610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21f4107d60 0x7f21f4100ff0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21f4102b00 0x7f21f4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 --2- 172.21.15.8:0/1767598000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21f4105430 0x7f21f4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 >> 172.21.15.8:0/1767598000 conn(0x7f21f40fc4b0 msgr2=0x7f21f4103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.686+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 shutdown_connections 2024-02-19T16:17:22.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:17:22.687+0000 7f21fc17b700 1 -- 172.21.15.8:0/1767598000 wait complete. 2024-02-19T16:17:22.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:22 smithi008 ceph-mon[32573]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:22.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:22 smithi191 ceph-mon[37203]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:22 smithi156 ceph-mon[36502]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:23.653 DEBUG:teuthology.orchestra.run.smithi008:osd.6> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.6.service 2024-02-19T16:17:23.656 INFO:tasks.cephadm:Deploying osd.7 on smithi156 with /dev/vg_nvme/lv_2... 2024-02-19T16:17:23.656 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-19T16:17:23.903 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:17:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:23 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:23 smithi008 ceph-mon[32573]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-19T16:17:24.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:23 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:23 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:23 smithi156 ceph-mon[36502]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-19T16:17:24.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:23 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:23 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:23 smithi191 ceph-mon[37203]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-19T16:17:24.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:24.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-19T16:17:24.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: osdmap e45: 7 total, 6 up, 7 in 2024-02-19T16:17:24.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:17:24.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:24.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:24.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: osdmap e45: 7 total, 6 up, 7 in 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-19T16:17:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: osdmap e45: 7 total, 6 up, 7 in 2024-02-19T16:17:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:17:25.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:25.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:25.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:25.444 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:17:25.144+0000 7f1b8047e700 -1 osd.6 0 waiting for initial osdmap 2024-02-19T16:17:25.445 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:17:25.155+0000 7f1b77a68700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:17:25.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-mon[32573]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:17:25.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-mon[32573]: osdmap e46: 7 total, 6 up, 7 in 2024-02-19T16:17:25.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:25.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:25.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:25 smithi008 ceph-mon[32573]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' 2024-02-19T16:17:25.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:25 smithi156 ceph-mon[36502]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:17:25.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:25 smithi156 ceph-mon[36502]: osdmap e46: 7 total, 6 up, 7 in 2024-02-19T16:17:25.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:25.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:25 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:25.851 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:25 smithi156 ceph-mon[36502]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' 2024-02-19T16:17:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-19T16:17:26.017 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-19T16:17:26.018 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-19T16:17:26.018 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-19T16:17:26.018 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00840965 s, 1.2 GB/s 2024-02-19T16:17:26.018 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-19T16:17:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:25 smithi191 ceph-mon[37203]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:17:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:25 smithi191 ceph-mon[37203]: osdmap e46: 7 total, 6 up, 7 in 2024-02-19T16:17:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:25 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:26.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:25 smithi191 ceph-mon[37203]: from='osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517]' entity='osd.6' 2024-02-19T16:17:26.643 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2024-02-19T16:17:26.887 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] boot 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: osdmap e47: 7 total, 7 up, 7 in 2024-02-19T16:17:27.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:26 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:17:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:17:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:27.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] boot 2024-02-19T16:17:27.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: osdmap e47: 7 total, 7 up, 7 in 2024-02-19T16:17:27.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:26 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:17:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:17:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-02-19T16:17:27.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:27.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: osd.6 [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] boot 2024-02-19T16:17:27.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: osdmap e47: 7 total, 7 up, 7 in 2024-02-19T16:17:27.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:26 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: osdmap e48: 7 total, 7 up, 7 in 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:28.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: osdmap e48: 7 total, 7 up, 7 in 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:28.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:28.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:28.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:28.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:28.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.523 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 -- 172.21.15.156:0/1178705363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 msgr2=0x7fc92c1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:28.523 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 --2- 172.21.15.156:0/1178705363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1025e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9200099c0 tx=0x7fc920009cd0 comp rx=0 tx=0).stop 2024-02-19T16:17:28.523 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 -- 172.21.15.156:0/1178705363 shutdown_connections 2024-02-19T16:17:28.523 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 --2- 172.21.15.156:0/1178705363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc92c102b20 0x7fc92c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 --2- 172.21.15.156:0/1178705363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc92c100f80 0x7fc92c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 --2- 172.21.15.156:0/1178705363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1025e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 -- 172.21.15.156:0/1178705363 >> 172.21.15.156:0/1178705363 conn(0x7fc92c0fc540 msgr2=0x7fc92c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 -- 172.21.15.156:0/1178705363 shutdown_connections 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.518+0000 7fc931efa700 1 -- 172.21.15.156:0/1178705363 wait complete. 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 Processor -- start 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 -- start start 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc92c100f80 0x7fc92c196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:28.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc92c102b20 0x7fc92c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c19ea30 con 0x7fc92c100f80 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c19eb70 con 0x7fc92c102180 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc931efa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c19ed00 con 0x7fc92c102b20 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc92affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc92affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49864/0 (socket says 172.21.15.156:49864) 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc92affd700 1 -- 172.21.15.156:0/2204553832 learned_addr learned my addr 172.21.15.156:0/2204553832 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.519+0000 7fc92b7fe700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc92c100f80 0x7fc92c196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92bfff700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc92c102b20 0x7fc92c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92affd700 1 -- 172.21.15.156:0/2204553832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc92c102b20 msgr2=0x7fc92c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92affd700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc92c102b20 0x7fc92c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92affd700 1 -- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc92c100f80 msgr2=0x7fc92c196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92affd700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc92c100f80 0x7fc92c196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92affd700 1 -- 172.21.15.156:0/2204553832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200095f0 con 0x7fc92c102180 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc92affd700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1973e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc920003790 tx=0x7fc920005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:17:28.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92001d070 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c06a7a0 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc920004110 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920022d20 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.520+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c06ac40 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.521+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c068ca0 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.522+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc920022470 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.523+0000 7fc928ff9700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9140a2150 0x7fc9140a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.523+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fc9200ceaa0 con 0x7fc92c102180 2024-02-19T16:17:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.523+0000 7fc92b7fe700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9140a2150 0x7fc9140a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: osdmap e48: 7 total, 7 up, 7 in 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:28.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:28.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:28.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:28.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:28.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:28.528 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.524+0000 7fc92b7fe700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9140a2150 0x7fc9140a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc92c101fe0 tx=0x7fc91c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:17:28.528 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.527+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fc920098890 con 0x7fc92c102180 2024-02-19T16:17:28.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:28.729+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc92c107f00 con 0x7fc9140a2150 2024-02-19T16:17:29.470 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:29 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi008 2024-02-19T16:17:29.470 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:29 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi008 to 5697M 2024-02-19T16:17:29.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:29.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:29.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:29 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi008 2024-02-19T16:17:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:29 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi008 to 5697M 2024-02-19T16:17:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:29.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:29.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:29 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi008 2024-02-19T16:17:29.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:29 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi008 to 5697M 2024-02-19T16:17:29.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:29.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:29.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:30.324 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:30 smithi156 ceph-mon[36502]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:30.325 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:30 smithi156 ceph-mon[36502]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:30 smithi008 ceph-mon[32573]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:30 smithi008 ceph-mon[32573]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:30.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:30 smithi191 ceph-mon[37203]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:30.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:30 smithi191 ceph-mon[37203]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:32.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:32.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/1654998049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]: dispatch 2024-02-19T16:17:32.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]: dispatch 2024-02-19T16:17:32.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]': finished 2024-02-19T16:17:32.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: osdmap e49: 8 total, 7 up, 8 in 2024-02-19T16:17:32.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:32.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:32 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/4062936628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:32.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:32.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/1654998049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]: dispatch 2024-02-19T16:17:32.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]: dispatch 2024-02-19T16:17:32.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]': finished 2024-02-19T16:17:32.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: osdmap e49: 8 total, 7 up, 8 in 2024-02-19T16:17:32.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:32.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:32 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/4062936628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/1654998049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]: dispatch 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]: dispatch 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20a26fbd-a93b-472b-a894-7693384be848"}]': finished 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: osdmap e49: 8 total, 7 up, 8 in 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:32 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/4062936628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:34.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:34 smithi008 ceph-mon[32573]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:34.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:34 smithi191 ceph-mon[37203]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:34.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:34 smithi156 ceph-mon[36502]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:36.462 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:36 smithi156 ceph-mon[36502]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:36.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:36 smithi008 ceph-mon[32573]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:36.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:36 smithi191 ceph-mon[37203]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:38.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:38 smithi156 ceph-mon[36502]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:38.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:38 smithi008 ceph-mon[32573]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:38.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:38 smithi191 ceph-mon[37203]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:39.493 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-19T16:17:39.493 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:39.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-19T16:17:39.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:39.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-19T16:17:39.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:40.500 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:40 smithi156 ceph-mon[36502]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:40.500 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:40 smithi156 ceph-mon[36502]: Deploying daemon osd.7 on smithi156 2024-02-19T16:17:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:40 smithi008 ceph-mon[32573]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:40.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:40 smithi008 ceph-mon[32573]: Deploying daemon osd.7 on smithi156 2024-02-19T16:17:40.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:40 smithi191 ceph-mon[37203]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:40.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:40 smithi191 ceph-mon[37203]: Deploying daemon osd.7 on smithi156 2024-02-19T16:17:42.306 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:42 smithi156 ceph-mon[36502]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:42.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:42 smithi008 ceph-mon[32573]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:42.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:42 smithi191 ceph-mon[37203]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:44.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:44 smithi008 ceph-mon[32573]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:44.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:44 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:44.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:44 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:44.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:44 smithi191 ceph-mon[37203]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:44.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:44 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:44.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:44 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:44.563 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:44 smithi156 ceph-mon[36502]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:44.563 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:44 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:44.563 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:44 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:45.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:45.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:45.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:46.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:46 smithi156 ceph-mon[36502]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:46.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:46 smithi008 ceph-mon[32573]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:46.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:46 smithi191 ceph-mon[37203]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:47.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.644+0000 7fc928ff9700 1 -- 172.21.15.156:0/2204553832 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc92c107f00 con 0x7fc9140a2150 2024-02-19T16:17:47.646 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 7 on host 'smithi156' 2024-02-19T16:17:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9140a2150 msgr2=0x7fc9140a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9140a2150 0x7fc9140a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc92c101fe0 tx=0x7fc91c00b410 comp rx=0 tx=0).stop 2024-02-19T16:17:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 msgr2=0x7fc92c1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1973e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc920003790 tx=0x7fc920005b00 comp rx=0 tx=0).stop 2024-02-19T16:17:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 shutdown_connections 2024-02-19T16:17:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9140a2150 0x7fc9140a4610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:47.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc92c102b20 0x7fc92c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:47.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc92c100f80 0x7fc92c196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:47.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 --2- 172.21.15.156:0/2204553832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c102180 0x7fc92c1973e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:47.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.646+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 >> 172.21.15.156:0/2204553832 conn(0x7fc92c0fc540 msgr2=0x7fc92c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:17:47.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.647+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 shutdown_connections 2024-02-19T16:17:47.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:17:47.647+0000 7fc931efa700 1 -- 172.21.15.156:0/2204553832 wait complete. 2024-02-19T16:17:48.377 DEBUG:teuthology.orchestra.run.smithi156:osd.7> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.7.service 2024-02-19T16:17:48.379 INFO:tasks.cephadm:Deploying osd.8 on smithi191 with /dev/vg_nvme/lv_2... 2024-02-19T16:17:48.379 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-19T16:17:48.625 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:17:48.746 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:48.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: from='osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-19T16:17:48.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:48 smithi191 ceph-mon[37203]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-19T16:17:48.877 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:48.877 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.877 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.877 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.877 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:48.878 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: from='osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-19T16:17:48.878 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:48 smithi156 ceph-mon[36502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-19T16:17:49.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:49.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: from='osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-19T16:17:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:48 smithi008 ceph-mon[32573]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-19T16:17:50.088 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:49 smithi191 ceph-mon[37203]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:50.088 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:49 smithi191 ceph-mon[37203]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-19T16:17:50.088 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:49 smithi191 ceph-mon[37203]: osdmap e50: 8 total, 7 up, 8 in 2024-02-19T16:17:50.088 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:49 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:50.088 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:49 smithi191 ceph-mon[37203]: from='osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:17:50.088 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:49 smithi191 ceph-mon[37203]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:17:50.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:49 smithi008 ceph-mon[32573]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:50.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:49 smithi008 ceph-mon[32573]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-19T16:17:50.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:49 smithi008 ceph-mon[32573]: osdmap e50: 8 total, 7 up, 8 in 2024-02-19T16:17:50.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:49 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:50.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:49 smithi008 ceph-mon[32573]: from='osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:17:50.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:49 smithi008 ceph-mon[32573]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:17:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-mon[36502]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-mon[36502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-19T16:17:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-mon[36502]: osdmap e50: 8 total, 7 up, 8 in 2024-02-19T16:17:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-mon[36502]: from='osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:17:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-mon[36502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:17:50.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:17:49.947+0000 7f25408c1700 -1 osd.7 0 waiting for initial osdmap 2024-02-19T16:17:50.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:17:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:17:49.955+0000 7f253c6b4700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:17:50.657 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-19T16:17:50.657 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-19T16:17:50.657 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-19T16:17:50.657 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-19T16:17:50.657 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142496 s, 736 MB/s 2024-02-19T16:17:50.657 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-19T16:17:51.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:50 smithi191 ceph-mon[37203]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:17:51.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:50 smithi191 ceph-mon[37203]: osdmap e51: 8 total, 7 up, 8 in 2024-02-19T16:17:51.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:51.262 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:50 smithi008 ceph-mon[32573]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:17:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:50 smithi008 ceph-mon[32573]: osdmap e51: 8 total, 7 up, 8 in 2024-02-19T16:17:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:51.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:51.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_2 2024-02-19T16:17:51.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:50 smithi156 ceph-mon[36502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:17:51.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:50 smithi156 ceph-mon[36502]: osdmap e51: 8 total, 7 up, 8 in 2024-02-19T16:17:51.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:51.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:51.535 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:17:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:17:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] boot 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: osdmap e52: 8 total, 8 up, 8 in 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: osdmap e53: 8 total, 8 up, 8 in 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi156 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi156 to 5693M 2024-02-19T16:17:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:52.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:51 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] boot 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: osdmap e52: 8 total, 8 up, 8 in 2024-02-19T16:17:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: osdmap e53: 8 total, 8 up, 8 in 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi156 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi156 to 5693M 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:52.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:52.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:51 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:17:52.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:17:52.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-02-19T16:17:52.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:52.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: osd.7 [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] boot 2024-02-19T16:17:52.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: osdmap e52: 8 total, 8 up, 8 in 2024-02-19T16:17:52.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:17:52.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: osdmap e53: 8 total, 8 up, 8 in 2024-02-19T16:17:52.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi156 2024-02-19T16:17:52.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi156 to 5693M 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:52.333 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:17:52.334 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:52.334 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:17:52.334 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:51 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 -- 172.21.15.191:0/2113073174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7141000c0 msgr2=0x7fe714100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2113073174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7141000c0 0x7fe714100520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe708009990 tx=0x7fe708009ca0 comp rx=0 tx=0).stop 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 -- 172.21.15.191:0/2113073174 shutdown_connections 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2113073174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe714100a60 0x7fe71410a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2113073174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe714103640 0x7fe714103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2113073174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7141000c0 0x7fe714100520 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 -- 172.21.15.191:0/2113073174 >> 172.21.15.191:0/2113073174 conn(0x7fe7140fbc80 msgr2=0x7fe7140fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:17:53.219 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 -- 172.21.15.191:0/2113073174 shutdown_connections 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 -- 172.21.15.191:0/2113073174 wait complete. 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 Processor -- start 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.216+0000 7fe71a83d700 1 -- start start 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7141000c0 0x7fe714197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe714100a60 0x7fe7141984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 0x7fe71419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71419fb20 con 0x7fe714100a60 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71419fc60 con 0x7fe7141000c0 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71419fdf0 con 0x7fe714103640 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 0x7fe71419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 0x7fe71419f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52194/0 (socket says 172.21.15.191:52194) 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 -- 172.21.15.191:0/2566534047 learned_addr learned my addr 172.21.15.191:0/2566534047 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:17:53.220 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe7137fe700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe714100a60 0x7fe7141984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe713fff700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7141000c0 0x7fe714197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 -- 172.21.15.191:0/2566534047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7141000c0 msgr2=0x7fe714197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7141000c0 0x7fe714197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 -- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe714100a60 msgr2=0x7fe7141984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe714100a60 0x7fe7141984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 -- 172.21.15.191:0/2566534047 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080095f0 con 0x7fe714103640 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe718dda700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 0x7fe71419f550 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00add0 tx=0x7fe70c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00f070 con 0x7fe714103640 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7141a00d0 con 0x7fe714103640 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe70c007b30 con 0x7fe714103640 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.217+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0053e0 con 0x7fe714103640 2024-02-19T16:17:53.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.218+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7141a05d0 con 0x7fe714103640 2024-02-19T16:17:53.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.218+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7141095a0 con 0x7fe714103640 2024-02-19T16:17:53.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.219+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe70c00c940 con 0x7fe714103640 2024-02-19T16:17:53.222 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.220+0000 7fe7117fa700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6fc0a2150 0x7fe6fc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:17:53.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.220+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fe70c0ccda0 con 0x7fe714103640 2024-02-19T16:17:53.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.220+0000 7fe713fff700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6fc0a2150 0x7fe6fc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:17:53.223 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.221+0000 7fe713fff700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6fc0a2150 0x7fe6fc0a4610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe704005950 tx=0x7fe7040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:17:53.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.224+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe70c096940 con 0x7fe714103640 2024-02-19T16:17:53.431 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:17:53.430+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe71402d9c0 con 0x7fe6fc0a2150 2024-02-19T16:17:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:53 smithi008 ceph-mon[32573]: osdmap e54: 8 total, 8 up, 8 in 2024-02-19T16:17:53.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:53 smithi156 ceph-mon[36502]: osdmap e54: 8 total, 8 up, 8 in 2024-02-19T16:17:53.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:53 smithi191 ceph-mon[37203]: osdmap e54: 8 total, 8 up, 8 in 2024-02-19T16:17:54.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:54 smithi156 ceph-mon[36502]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:54.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:54 smithi156 ceph-mon[36502]: osdmap e55: 8 total, 8 up, 8 in 2024-02-19T16:17:54.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:54 smithi156 ceph-mon[36502]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:54.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:54.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:54.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:54 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:54.671 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:54 smithi191 ceph-mon[37203]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:54.671 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:54 smithi191 ceph-mon[37203]: osdmap e55: 8 total, 8 up, 8 in 2024-02-19T16:17:54.671 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:54 smithi191 ceph-mon[37203]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:54.671 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:54.671 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:54.671 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:54 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:54 smithi008 ceph-mon[32573]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:54 smithi008 ceph-mon[32573]: osdmap e55: 8 total, 8 up, 8 in 2024-02-19T16:17:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:54 smithi008 ceph-mon[32573]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:17:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:17:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:17:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:54 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:17:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:55 smithi156 ceph-mon[36502]: osdmap e56: 8 total, 8 up, 8 in 2024-02-19T16:17:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:55 smithi008 ceph-mon[32573]: osdmap e56: 8 total, 8 up, 8 in 2024-02-19T16:17:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:55 smithi191 ceph-mon[37203]: osdmap e56: 8 total, 8 up, 8 in 2024-02-19T16:17:56.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:56 smithi156 ceph-mon[36502]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:56.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:56 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/3095011072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]: dispatch 2024-02-19T16:17:56.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:56 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]: dispatch 2024-02-19T16:17:56.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:56 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]': finished 2024-02-19T16:17:56.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:56 smithi156 ceph-mon[36502]: osdmap e57: 9 total, 8 up, 9 in 2024-02-19T16:17:56.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:56 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:17:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:56 smithi008 ceph-mon[32573]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:56 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/3095011072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]: dispatch 2024-02-19T16:17:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:56 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]: dispatch 2024-02-19T16:17:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:56 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]': finished 2024-02-19T16:17:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:56 smithi008 ceph-mon[32573]: osdmap e57: 9 total, 8 up, 9 in 2024-02-19T16:17:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:17:56.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:56 smithi191 ceph-mon[37203]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:56.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:56 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/3095011072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]: dispatch 2024-02-19T16:17:56.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:56 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]: dispatch 2024-02-19T16:17:56.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:56 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6e14c6-7bb9-46da-9e3a-1581f86e1764"}]': finished 2024-02-19T16:17:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:56 smithi191 ceph-mon[37203]: osdmap e57: 9 total, 8 up, 9 in 2024-02-19T16:17:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:56 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:17:57.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:57 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/564939713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:57.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:57 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/564939713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:57.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:57 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/564939713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:17:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:17:58 smithi156 ceph-mon[36502]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:58.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:17:58 smithi008 ceph-mon[32573]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:17:58.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:17:58 smithi191 ceph-mon[37203]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:00.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:00 smithi191 ceph-mon[37203]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:00.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:00 smithi156 ceph-mon[36502]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:00 smithi008 ceph-mon[32573]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:02.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:02 smithi191 ceph-mon[37203]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:02 smithi156 ceph-mon[36502]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:02.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:02 smithi008 ceph-mon[32573]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:04 smithi191 ceph-mon[37203]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-19T16:18:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:04 smithi191 ceph-mon[37203]: Deploying daemon osd.8 on smithi191 2024-02-19T16:18:04.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:04 smithi156 ceph-mon[36502]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:04.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-19T16:18:04.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:04.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:04 smithi156 ceph-mon[36502]: Deploying daemon osd.8 on smithi191 2024-02-19T16:18:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:04 smithi008 ceph-mon[32573]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-19T16:18:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:04 smithi008 ceph-mon[32573]: Deploying daemon osd.8 on smithi191 2024-02-19T16:18:06.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:06 smithi156 ceph-mon[36502]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:06.674 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:06 smithi191 ceph-mon[37203]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:06.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:06 smithi008 ceph-mon[32573]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:08.473 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:08 smithi191 ceph-mon[37203]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:08.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:08 smithi156 ceph-mon[36502]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:08.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:08 smithi008 ceph-mon[32573]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:10 smithi008 ceph-mon[32573]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:10.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:10 smithi191 ceph-mon[37203]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:10.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:10 smithi156 ceph-mon[36502]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:10.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:10.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:10.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:12.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:12 smithi191 ceph-mon[37203]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:12.348 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:12 smithi008 ceph-mon[32573]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:12 smithi156 ceph-mon[36502]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:12.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.952+0000 7fe7117fa700 1 -- 172.21.15.191:0/2566534047 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe71402d9c0 con 0x7fe6fc0a2150 2024-02-19T16:18:12.954 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 8 on host 'smithi191' 2024-02-19T16:18:12.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6fc0a2150 msgr2=0x7fe6fc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:12.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6fc0a2150 0x7fe6fc0a4610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe704005950 tx=0x7fe7040058e0 comp rx=0 tx=0).stop 2024-02-19T16:18:12.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 msgr2=0x7fe71419f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:12.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 0x7fe71419f550 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00add0 tx=0x7fe70c00c330 comp rx=0 tx=0).stop 2024-02-19T16:18:12.957 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 shutdown_connections 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6fc0a2150 0x7fe6fc0a4610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe714103640 0x7fe71419f550 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7141000c0 0x7fe714197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 --2- 172.21.15.191:0/2566534047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe714100a60 0x7fe7141984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 >> 172.21.15.191:0/2566534047 conn(0x7fe7140fbc80 msgr2=0x7fe7141066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 shutdown_connections 2024-02-19T16:18:12.958 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:18:12.956+0000 7fe71a83d700 1 -- 172.21.15.191:0/2566534047 wait complete. 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-19T16:18:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:13.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:13.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:13.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-19T16:18:13.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-19T16:18:13.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:13.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-19T16:18:13.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:13.648 DEBUG:teuthology.orchestra.run.smithi191:osd.8> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.8.service 2024-02-19T16:18:13.650 INFO:tasks.cephadm:Deploying osd.9 on smithi008 with /dev/vg_nvme/lv_1... 2024-02-19T16:18:13.650 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-19T16:18:13.888 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:18:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:14 smithi008 ceph-mon[32573]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:14 smithi008 ceph-mon[32573]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-19T16:18:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:14 smithi008 ceph-mon[32573]: osdmap e58: 9 total, 8 up, 9 in 2024-02-19T16:18:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:14 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:14 smithi008 ceph-mon[32573]: from='osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:18:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:14 smithi008 ceph-mon[32573]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:18:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-mon[37203]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-mon[37203]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-19T16:18:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-mon[37203]: osdmap e58: 9 total, 8 up, 9 in 2024-02-19T16:18:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-mon[37203]: from='osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:18:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-mon[37203]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:18:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:14 smithi156 ceph-mon[36502]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:14 smithi156 ceph-mon[36502]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-19T16:18:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:14 smithi156 ceph-mon[36502]: osdmap e58: 9 total, 8 up, 9 in 2024-02-19T16:18:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:14 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:14 smithi156 ceph-mon[36502]: from='osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:18:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:14 smithi156 ceph-mon[36502]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:18:15.110 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:18:14.948+0000 7fbf936ca700 -1 osd.8 0 waiting for initial osdmap 2024-02-19T16:18:15.110 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:18:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:18:14.957+0000 7fbf8f4bd700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:18:15.971 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-19T16:18:15.971 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-19T16:18:15.971 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-02-19T16:18:15.971 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-02-19T16:18:15.971 INFO:teuthology.orchestra.run.smithi008.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00843547 s, 1.2 GB/s 2024-02-19T16:18:15.971 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-02-19T16:18:16.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:15 smithi008 ceph-mon[32573]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:16.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:15 smithi008 ceph-mon[32573]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:18:16.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:15 smithi008 ceph-mon[32573]: osdmap e59: 9 total, 8 up, 9 in 2024-02-19T16:18:16.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:16.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:16.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:15 smithi191 ceph-mon[37203]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:16.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:15 smithi191 ceph-mon[37203]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:18:16.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:15 smithi191 ceph-mon[37203]: osdmap e59: 9 total, 8 up, 9 in 2024-02-19T16:18:16.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:16.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:15 smithi156 ceph-mon[36502]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-02-19T16:18:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:15 smithi156 ceph-mon[36502]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:18:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:15 smithi156 ceph-mon[36502]: osdmap e59: 9 total, 8 up, 9 in 2024-02-19T16:18:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:16.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:16.611 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_1 2024-02-19T16:18:16.856 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:18:17.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:18:17.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:18:17.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] boot 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: osdmap e60: 9 total, 9 up, 9 in 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: osdmap e61: 9 total, 9 up, 9 in 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi191 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi191 to 5693M 2024-02-19T16:18:17.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:17.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:17.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:16 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] boot 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: osdmap e60: 9 total, 9 up, 9 in 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: osdmap e61: 9 total, 9 up, 9 in 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi191 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi191 to 5693M 2024-02-19T16:18:17.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:17.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:17.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:16 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: osd.8 [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] boot 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: osdmap e60: 9 total, 9 up, 9 in 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: osdmap e61: 9 total, 9 up, 9 in 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi191 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:17.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi191 to 5693M 2024-02-19T16:18:17.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:17.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:17.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:17.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:16 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:18.388 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:18 smithi008 ceph-mon[32573]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:18.388 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:18 smithi008 ceph-mon[32573]: osdmap e62: 9 total, 9 up, 9 in 2024-02-19T16:18:18.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:18 smithi156 ceph-mon[36502]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:18.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:18 smithi156 ceph-mon[36502]: osdmap e62: 9 total, 9 up, 9 in 2024-02-19T16:18:18.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 -- 172.21.15.8:0/3067445750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 msgr2=0x7f637010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:18.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 --2- 172.21.15.8:0/3067445750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637010a550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f636800b0a0 tx=0x7f636800b3b0 comp rx=0 tx=0).stop 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 -- 172.21.15.8:0/3067445750 shutdown_connections 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 --2- 172.21.15.8:0/3067445750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637010a550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 --2- 172.21.15.8:0/3067445750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6370100320 0x7f6370100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 --2- 172.21.15.8:0/3067445750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 0x7f6370103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 -- 172.21.15.8:0/3067445750 >> 172.21.15.8:0/3067445750 conn(0x7f63700fbc80 msgr2=0x7f63700fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 -- 172.21.15.8:0/3067445750 shutdown_connections 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 -- 172.21.15.8:0/3067445750 wait complete. 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.632+0000 7f63755b4700 1 Processor -- start 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 -- start start 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6370100320 0x7f6370198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 0x7f6370198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:18.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637019fdd0 con 0x7f6370103ca0 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637019ff10 con 0x7f6370103300 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f63755b4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63701a00a0 con 0x7f6370100320 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37386/0 (socket says 172.21.15.8:37386) 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 -- 172.21.15.8:0/1430354761 learned_addr learned my addr 172.21.15.8:0/1430354761 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636effd700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6370100320 0x7f6370198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636e7fc700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 0x7f6370198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 -- 172.21.15.8:0/1430354761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6370100320 msgr2=0x7f6370198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6370100320 0x7f6370198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 -- 172.21.15.8:0/1430354761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 msgr2=0x7f6370198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 0x7f6370198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 -- 172.21.15.8:0/1430354761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009d00 con 0x7f6370103ca0 2024-02-19T16:18:18.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636e7fc700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 0x7f6370198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:18:18.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.633+0000 7f636f7fe700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637019f800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63680094f0 tx=0x7f6368009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:18:18.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.634+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800e040 con 0x7f6370103ca0 2024-02-19T16:18:18.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.634+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6368004e10 con 0x7f6370103ca0 2024-02-19T16:18:18.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.634+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63701a0320 con 0x7f6370103ca0 2024-02-19T16:18:18.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.634+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636801b980 con 0x7f6370103ca0 2024-02-19T16:18:18.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.634+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63701a0810 con 0x7f6370103ca0 2024-02-19T16:18:18.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.635+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63701094c0 con 0x7f6370103ca0 2024-02-19T16:18:18.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.636+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f636801dc10 con 0x7f6370103ca0 2024-02-19T16:18:18.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.637+0000 7f6357fff700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f63580a2150 0x7f63580a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:18.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.637+0000 7f636effd700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f63580a2150 0x7f63580a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:18.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.637+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f63680cf810 con 0x7f6370103ca0 2024-02-19T16:18:18.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.637+0000 7f636effd700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f63580a2150 0x7f63580a4610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f637006a4c0 tx=0x7f6360009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:18:18.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.641+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f6368099200 con 0x7f6370103ca0 2024-02-19T16:18:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:18 smithi191 ceph-mon[37203]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:18 smithi191 ceph-mon[37203]: osdmap e62: 9 total, 9 up, 9 in 2024-02-19T16:18:18.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:18.858+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f637002da30 con 0x7f63580a2150 2024-02-19T16:18:19.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:19 smithi156 ceph-mon[36502]: osdmap e63: 9 total, 9 up, 9 in 2024-02-19T16:18:19.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:18:19.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:18:19.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:19.581 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:19 smithi008 ceph-mon[32573]: osdmap e63: 9 total, 9 up, 9 in 2024-02-19T16:18:19.581 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:18:19.582 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:18:19.582 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:19.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:19 smithi191 ceph-mon[37203]: osdmap e63: 9 total, 9 up, 9 in 2024-02-19T16:18:19.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:18:19.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:18:19.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:20.437 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:20 smithi008 ceph-mon[32573]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:20.437 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:20 smithi008 ceph-mon[32573]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:18:20.437 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:20 smithi008 ceph-mon[32573]: osdmap e64: 9 total, 9 up, 9 in 2024-02-19T16:18:20.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:20 smithi156 ceph-mon[36502]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:20.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:20 smithi156 ceph-mon[36502]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:18:20.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:20 smithi156 ceph-mon[36502]: osdmap e64: 9 total, 9 up, 9 in 2024-02-19T16:18:20.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:20 smithi191 ceph-mon[37203]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:20 smithi191 ceph-mon[37203]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:18:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:20 smithi191 ceph-mon[37203]: osdmap e64: 9 total, 9 up, 9 in 2024-02-19T16:18:22.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:22 smithi156 ceph-mon[36502]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:22.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:22 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2634686153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00fb3fac-7fc8-4d1f-8bae-dae54e114041"}]: dispatch 2024-02-19T16:18:22.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:22 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2634686153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00fb3fac-7fc8-4d1f-8bae-dae54e114041"}]': finished 2024-02-19T16:18:22.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:22 smithi156 ceph-mon[36502]: osdmap e65: 10 total, 9 up, 10 in 2024-02-19T16:18:22.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:22.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:22 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2574598557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:18:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:22 smithi008 ceph-mon[32573]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:22 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2634686153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00fb3fac-7fc8-4d1f-8bae-dae54e114041"}]: dispatch 2024-02-19T16:18:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:22 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2634686153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00fb3fac-7fc8-4d1f-8bae-dae54e114041"}]': finished 2024-02-19T16:18:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:22 smithi008 ceph-mon[32573]: osdmap e65: 10 total, 9 up, 10 in 2024-02-19T16:18:22.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:22.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:22 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2574598557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:18:22.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:22 smithi191 ceph-mon[37203]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:22 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2634686153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00fb3fac-7fc8-4d1f-8bae-dae54e114041"}]: dispatch 2024-02-19T16:18:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:22 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2634686153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00fb3fac-7fc8-4d1f-8bae-dae54e114041"}]': finished 2024-02-19T16:18:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:22 smithi191 ceph-mon[37203]: osdmap e65: 10 total, 9 up, 10 in 2024-02-19T16:18:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:22 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2574598557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:18:24.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:24 smithi156 ceph-mon[36502]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-02-19T16:18:24.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:24 smithi008 ceph-mon[32573]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-02-19T16:18:24.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:24 smithi191 ceph-mon[37203]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-02-19T16:18:26.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:26 smithi156 ceph-mon[36502]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-02-19T16:18:26.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:26 smithi008 ceph-mon[32573]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-02-19T16:18:26.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:26 smithi191 ceph-mon[37203]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-02-19T16:18:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:28 smithi156 ceph-mon[36502]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-19T16:18:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:28 smithi008 ceph-mon[32573]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-19T16:18:28.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:28 smithi191 ceph-mon[37203]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-19T16:18:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-19T16:18:29.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:29 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:29.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-19T16:18:29.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:29 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:29.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-19T16:18:29.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:29 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:30 smithi008 ceph-mon[32573]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-19T16:18:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:30 smithi008 ceph-mon[32573]: Deploying daemon osd.9 on smithi008 2024-02-19T16:18:30.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:30 smithi156 ceph-mon[36502]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-19T16:18:30.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:30 smithi156 ceph-mon[36502]: Deploying daemon osd.9 on smithi008 2024-02-19T16:18:30.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:30 smithi191 ceph-mon[37203]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-19T16:18:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:30 smithi191 ceph-mon[37203]: Deploying daemon osd.9 on smithi008 2024-02-19T16:18:32.460 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:32 smithi008 ceph-mon[32573]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:18:32.579 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:32 smithi191 ceph-mon[37203]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:18:32.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:32 smithi156 ceph-mon[36502]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-19T16:18:34.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:34 smithi008 ceph-mon[32573]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-19T16:18:34.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:34 smithi156 ceph-mon[36502]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-19T16:18:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:34 smithi191 ceph-mon[37203]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-19T16:18:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:36 smithi008 ceph-mon[32573]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:36 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:36.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:36 smithi191 ceph-mon[37203]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:36.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:36.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:36.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:36 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:36.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:36 smithi156 ceph-mon[36502]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:36.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:36.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:36.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:36 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:38 smithi008 ceph-mon[32573]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:38.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:38 smithi191 ceph-mon[37203]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:38.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:38 smithi156 ceph-mon[36502]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:39.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.170+0000 7f6357fff700 1 -- 172.21.15.8:0/1430354761 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f637002da30 con 0x7f63580a2150 2024-02-19T16:18:39.172 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 9 on host 'smithi008' 2024-02-19T16:18:39.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f63580a2150 msgr2=0x7f63580a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f63580a2150 0x7f63580a4610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f637006a4c0 tx=0x7f6360009190 comp rx=0 tx=0).stop 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 msgr2=0x7f637019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637019f800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63680094f0 tx=0x7f6368009520 comp rx=0 tx=0).stop 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 shutdown_connections 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f63580a2150 0x7f63580a4610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6370103ca0 0x7f637019f800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6370100320 0x7f6370198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 --2- 172.21.15.8:0/1430354761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6370103300 0x7f6370198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 >> 172.21.15.8:0/1430354761 conn(0x7f63700fbc80 msgr2=0x7f6370106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 shutdown_connections 2024-02-19T16:18:39.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:18:39.173+0000 7f63755b4700 1 -- 172.21.15.8:0/1430354761 wait complete. 2024-02-19T16:18:39.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:39 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:39.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:39.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:39.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-19T16:18:39.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:39 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:39 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:18:39.879 DEBUG:teuthology.orchestra.run.smithi008:osd.9> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.9.service 2024-02-19T16:18:39.881 INFO:tasks.cephadm:Deploying osd.10 on smithi156 with /dev/vg_nvme/lv_1... 2024-02-19T16:18:39.882 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-19T16:18:40.131 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:18:40.450 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-mon[32573]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:40.450 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-mon[32573]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-19T16:18:40.450 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-mon[32573]: osdmap e66: 10 total, 9 up, 10 in 2024-02-19T16:18:40.450 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-mon[32573]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:18:40.450 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:40.450 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:18:40.318+0000 7f0d9ed1d700 -1 osd.9 0 waiting for initial osdmap 2024-02-19T16:18:40.450 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:18:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:18:40.332+0000 7f0d9ab10700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:18:40.515 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:40 smithi156 ceph-mon[36502]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:40.515 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:40 smithi156 ceph-mon[36502]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-19T16:18:40.516 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:40 smithi156 ceph-mon[36502]: osdmap e66: 10 total, 9 up, 10 in 2024-02-19T16:18:40.516 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:40 smithi156 ceph-mon[36502]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:18:40.516 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:40 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:40.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:40 smithi191 ceph-mon[37203]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:40 smithi191 ceph-mon[37203]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-19T16:18:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:40 smithi191 ceph-mon[37203]: osdmap e66: 10 total, 9 up, 10 in 2024-02-19T16:18:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:40 smithi191 ceph-mon[37203]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-02-19T16:18:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:40 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:41.497 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:41 smithi008 ceph-mon[32573]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:18:41.497 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:41 smithi008 ceph-mon[32573]: osdmap e67: 10 total, 9 up, 10 in 2024-02-19T16:18:41.497 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:41.497 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:41 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:41.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:41 smithi156 ceph-mon[36502]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:18:41.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:41 smithi156 ceph-mon[36502]: osdmap e67: 10 total, 9 up, 10 in 2024-02-19T16:18:41.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:41.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:41 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:41.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:41 smithi191 ceph-mon[37203]: from='osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-02-19T16:18:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:41 smithi191 ceph-mon[37203]: osdmap e67: 10 total, 9 up, 10 in 2024-02-19T16:18:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:41 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.197 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-19T16:18:42.198 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-19T16:18:42.198 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-19T16:18:42.198 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-19T16:18:42.198 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00905644 s, 1.2 GB/s 2024-02-19T16:18:42.198 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-19T16:18:42.365 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:18:42.366 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:18:42.366 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:42.366 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.366 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] boot 2024-02-19T16:18:42.366 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: osdmap e68: 10 total, 10 up, 10 in 2024-02-19T16:18:42.366 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:42 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] boot 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: osdmap e68: 10 total, 10 up, 10 in 2024-02-19T16:18:42.671 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:42 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:18:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:18:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-19T16:18:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: osd.9 [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] boot 2024-02-19T16:18:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: osdmap e68: 10 total, 10 up, 10 in 2024-02-19T16:18:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:42 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:18:42.922 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2024-02-19T16:18:43.177 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:18:43.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: osdmap e69: 10 total, 10 up, 10 in 2024-02-19T16:18:43.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:43.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.565 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: osdmap e69: 10 total, 10 up, 10 in 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:43.566 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:43.567 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: osdmap e69: 10 total, 10 up, 10 in 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:18:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:18:44.632 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:44 smithi156 ceph-mon[36502]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:44.632 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:44 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi008 2024-02-19T16:18:44.632 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:44 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi008 to 4273M 2024-02-19T16:18:44.634 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:44 smithi008 ceph-mon[32573]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:44.634 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:44 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi008 2024-02-19T16:18:44.634 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:44 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi008 to 4273M 2024-02-19T16:18:44.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:44 smithi191 ceph-mon[37203]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:44.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:44 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi008 2024-02-19T16:18:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:44 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi008 to 4273M 2024-02-19T16:18:44.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 -- 172.21.15.156:0/2812226586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c102b00 msgr2=0x7fe28c104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:44.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 --2- 172.21.15.156:0/2812226586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c102b00 0x7fe28c104ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009990 tx=0x7fe27c009ca0 comp rx=0 tx=0).stop 2024-02-19T16:18:44.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 -- 172.21.15.156:0/2812226586 shutdown_connections 2024-02-19T16:18:44.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 --2- 172.21.15.156:0/2812226586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c107d60 0x7fe28c10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:44.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 --2- 172.21.15.156:0/2812226586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c102b00 0x7fe28c104ef0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:44.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 --2- 172.21.15.156:0/2812226586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe28c105430 0x7fe28c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:44.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 -- 172.21.15.156:0/2812226586 >> 172.21.15.156:0/2812226586 conn(0x7fe28c0fc4d0 msgr2=0x7fe28c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:18:44.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 -- 172.21.15.156:0/2812226586 shutdown_connections 2024-02-19T16:18:44.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 -- 172.21.15.156:0/2812226586 wait complete. 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 Processor -- start 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.875+0000 7fe290a36700 1 -- start start 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c102b00 0x7fe28c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe28c105430 0x7fe28c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 0x7fe28c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19fb30 con 0x7fe28c102b00 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19fc70 con 0x7fe28c107d60 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19fe00 con 0x7fe28c105430 2024-02-19T16:18:44.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 0x7fe28c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 0x7fe28c19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52774/0 (socket says 172.21.15.156:52774) 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c102b00 0x7fe28c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 -- 172.21.15.156:0/2338089316 learned_addr learned my addr 172.21.15.156:0/2338089316 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe289d9b700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe28c105430 0x7fe28c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 -- 172.21.15.156:0/2338089316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe28c105430 msgr2=0x7fe28c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe28c105430 0x7fe28c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 -- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c102b00 msgr2=0x7fe28c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c102b00 0x7fe28c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 -- 172.21.15.156:0/2338089316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0095f0 con 0x7fe28c107d60 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28a59c700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c102b00 0x7fe28c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe28ad9d700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 0x7fe28c19f560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe28400ef90 tx=0x7fe28400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284018070 con 0x7fe28c107d60 2024-02-19T16:18:44.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe28400f040 con 0x7fe28c107d60 2024-02-19T16:18:44.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c1a0080 con 0x7fe28c107d60 2024-02-19T16:18:44.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.876+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400cea0 con 0x7fe28c107d60 2024-02-19T16:18:44.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.877+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c1a0580 con 0x7fe28c107d60 2024-02-19T16:18:44.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.877+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c0fe050 con 0x7fe28c107d60 2024-02-19T16:18:44.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.879+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe284004ad0 con 0x7fe28c107d60 2024-02-19T16:18:44.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.879+0000 7fe27b7fe700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2740a2150 0x7fe2740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:18:44.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.879+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fe2840cd8f0 con 0x7fe28c107d60 2024-02-19T16:18:44.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.879+0000 7fe28a59c700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2740a2150 0x7fe2740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:18:44.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.880+0000 7fe28a59c700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2740a2150 0x7fe2740a4610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b2e0 tx=0x7fe27c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:18:44.884 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:44.883+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe284097150 con 0x7fe28c107d60 2024-02-19T16:18:45.086 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:18:45.085+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe28c02da30 con 0x7fe2740a2150 2024-02-19T16:18:45.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:18:45.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:18:45.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:45 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:45.686 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:18:45.687 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:18:45.687 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:45 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:18:45.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:18:45.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:45 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:46 smithi008 ceph-mon[32573]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:46.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:46 smithi008 ceph-mon[32573]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:18:46.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:46 smithi191 ceph-mon[37203]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:46.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:46 smithi191 ceph-mon[37203]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:18:46.807 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:46 smithi156 ceph-mon[36502]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:46.807 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:46 smithi156 ceph-mon[36502]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:18:48.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:48 smithi156 ceph-mon[36502]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:48.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:48 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/1345925420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]: dispatch 2024-02-19T16:18:48.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:48 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]: dispatch 2024-02-19T16:18:48.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:48 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]': finished 2024-02-19T16:18:48.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:48 smithi156 ceph-mon[36502]: osdmap e70: 11 total, 10 up, 11 in 2024-02-19T16:18:48.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:48 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:18:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:48 smithi008 ceph-mon[32573]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:48 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/1345925420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]: dispatch 2024-02-19T16:18:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:48 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]: dispatch 2024-02-19T16:18:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:48 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]': finished 2024-02-19T16:18:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:48 smithi008 ceph-mon[32573]: osdmap e70: 11 total, 10 up, 11 in 2024-02-19T16:18:48.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:48 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:18:48.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:48 smithi191 ceph-mon[37203]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:48 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/1345925420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]: dispatch 2024-02-19T16:18:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:48 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]: dispatch 2024-02-19T16:18:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:48 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "816f1015-93ed-443a-9ad8-e0291b9970cf"}]': finished 2024-02-19T16:18:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:48 smithi191 ceph-mon[37203]: osdmap e70: 11 total, 10 up, 11 in 2024-02-19T16:18:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:48 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:18:49.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:49 smithi008 ceph-mon[32573]: from='client.? 172.21.15.156:0/2400426784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:18:49.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:49 smithi191 ceph-mon[37203]: from='client.? 172.21.15.156:0/2400426784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:18:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:49 smithi156 ceph-mon[36502]: from='client.? 172.21.15.156:0/2400426784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:18:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:50 smithi008 ceph-mon[32573]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:50.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:50 smithi191 ceph-mon[37203]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:50 smithi156 ceph-mon[36502]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:52.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:52 smithi008 ceph-mon[32573]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:52.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:52 smithi191 ceph-mon[37203]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:52.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:52 smithi156 ceph-mon[36502]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:54 smithi008 ceph-mon[32573]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:54.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:54 smithi191 ceph-mon[37203]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:54.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:54 smithi156 ceph-mon[36502]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:56.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:56 smithi156 ceph-mon[36502]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:56.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:56 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-19T16:18:56.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:56 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:56.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:56 smithi156 ceph-mon[36502]: Deploying daemon osd.10 on smithi156 2024-02-19T16:18:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:56 smithi008 ceph-mon[32573]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-19T16:18:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:56 smithi008 ceph-mon[32573]: Deploying daemon osd.10 on smithi156 2024-02-19T16:18:56.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:56 smithi191 ceph-mon[37203]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:56 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-19T16:18:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:56 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:18:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:56 smithi191 ceph-mon[37203]: Deploying daemon osd.10 on smithi156 2024-02-19T16:18:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:18:58 smithi156 ceph-mon[36502]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:58.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:18:58 smithi008 ceph-mon[32573]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:18:58.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:18:58 smithi191 ceph-mon[37203]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:00 smithi008 ceph-mon[32573]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:00.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:00 smithi191 ceph-mon[37203]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:00.811 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:00 smithi156 ceph-mon[36502]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:02.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:02 smithi008 ceph-mon[32573]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:02.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:02 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:02.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:02 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:02.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:02 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:02.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:02 smithi191 ceph-mon[37203]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:02.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:02 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:02.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:02 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:02.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:02 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:02 smithi156 ceph-mon[36502]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:02 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:02 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:02 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:04.517 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:04 smithi156 ceph-mon[36502]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:04.517 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.517 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.517 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:04.517 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:04.517 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:04 smithi008 ceph-mon[32573]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:04 smithi191 ceph-mon[37203]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:04.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:04.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:04.954 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 10 on host 'smithi156' 2024-02-19T16:19:04.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.953+0000 7fe27b7fe700 1 -- 172.21.15.156:0/2338089316 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe28c02da30 con 0x7fe2740a2150 2024-02-19T16:19:04.956 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2740a2150 msgr2=0x7fe2740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:04.956 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2740a2150 0x7fe2740a4610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b2e0 tx=0x7fe27c005fb0 comp rx=0 tx=0).stop 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 msgr2=0x7fe28c19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 0x7fe28c19f560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe28400ef90 tx=0x7fe28400c3a0 comp rx=0 tx=0).stop 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 shutdown_connections 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2740a2150 0x7fe2740a4610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe28c107d60 0x7fe28c19f560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe28c102b00 0x7fe28c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 --2- 172.21.15.156:0/2338089316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe28c105430 0x7fe28c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 >> 172.21.15.156:0/2338089316 conn(0x7fe28c0fc4d0 msgr2=0x7fe28c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 shutdown_connections 2024-02-19T16:19:04.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-19T16:19:04.956+0000 7fe290a36700 1 -- 172.21.15.156:0/2338089316 wait complete. 2024-02-19T16:19:05.600 DEBUG:teuthology.orchestra.run.smithi156:osd.10> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.10.service 2024-02-19T16:19:05.602 INFO:tasks.cephadm:Deploying osd.11 on smithi191 with /dev/vg_nvme/lv_1... 2024-02-19T16:19:05.602 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-19T16:19:05.849 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:19:06.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:05 smithi191 ceph-mon[37203]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:06.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:06.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:06.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:05 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:06.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:05 smithi191 ceph-mon[37203]: from='osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-19T16:19:06.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:05 smithi191 ceph-mon[37203]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-19T16:19:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:05 smithi008 ceph-mon[32573]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:05 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:05 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:05 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:05 smithi008 ceph-mon[32573]: from='osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-19T16:19:06.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:05 smithi008 ceph-mon[32573]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-19T16:19:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:05 smithi156 ceph-mon[36502]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:05 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:05 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:05 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:05 smithi156 ceph-mon[36502]: from='osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-19T16:19:06.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:05 smithi156 ceph-mon[36502]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-19T16:19:07.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-mon[36502]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-19T16:19:07.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-mon[36502]: osdmap e71: 11 total, 10 up, 11 in 2024-02-19T16:19:07.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:07.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-mon[36502]: from='osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:19:07.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-mon[36502]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:19:07.081 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:19:06.953+0000 7fcef32c3700 -1 osd.10 0 waiting for initial osdmap 2024-02-19T16:19:07.081 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:19:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:19:06.965+0000 7fceef0b6700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:19:07.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:06 smithi191 ceph-mon[37203]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-19T16:19:07.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:06 smithi191 ceph-mon[37203]: osdmap e71: 11 total, 10 up, 11 in 2024-02-19T16:19:07.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:07.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:06 smithi191 ceph-mon[37203]: from='osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:19:07.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:06 smithi191 ceph-mon[37203]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:19:07.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:06 smithi008 ceph-mon[32573]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-19T16:19:07.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:06 smithi008 ceph-mon[32573]: osdmap e71: 11 total, 10 up, 11 in 2024-02-19T16:19:07.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:07.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:06 smithi008 ceph-mon[32573]: from='osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:19:07.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:06 smithi008 ceph-mon[32573]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-19T16:19:08.089 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-19T16:19:08.089 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-19T16:19:08.090 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-19T16:19:08.090 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-19T16:19:08.090 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111614 s, 939 MB/s 2024-02-19T16:19:08.090 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-19T16:19:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:07 smithi008 ceph-mon[32573]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:07 smithi008 ceph-mon[32573]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:19:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:07 smithi008 ceph-mon[32573]: osdmap e72: 11 total, 10 up, 11 in 2024-02-19T16:19:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:07 smithi191 ceph-mon[37203]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:07 smithi191 ceph-mon[37203]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:19:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:07 smithi191 ceph-mon[37203]: osdmap e72: 11 total, 10 up, 11 in 2024-02-19T16:19:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:08.299 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:07 smithi156 ceph-mon[36502]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:08.299 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:07 smithi156 ceph-mon[36502]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-19T16:19:08.299 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:07 smithi156 ceph-mon[36502]: osdmap e72: 11 total, 10 up, 11 in 2024-02-19T16:19:08.299 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:08.299 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:08.791 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_1 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] boot 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: osdmap e73: 11 total, 11 up, 11 in 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:08 smithi191 ceph-mon[37203]: osdmap e74: 11 total, 11 up, 11 in 2024-02-19T16:19:09.038 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:19:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:19:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:19:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] boot 2024-02-19T16:19:09.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: osdmap e73: 11 total, 11 up, 11 in 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:08 smithi008 ceph-mon[32573]: osdmap e74: 11 total, 11 up, 11 in 2024-02-19T16:19:09.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: osd.10 [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] boot 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: osdmap e73: 11 total, 11 up, 11 in 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:09.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:09.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:08 smithi156 ceph-mon[36502]: osdmap e74: 11 total, 11 up, 11 in 2024-02-19T16:19:10.103 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:09 smithi191 ceph-mon[37203]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:10.103 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:09 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi156 2024-02-19T16:19:10.104 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:09 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi156 to 4270M 2024-02-19T16:19:10.104 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:09 smithi191 ceph-mon[37203]: osdmap e75: 11 total, 11 up, 11 in 2024-02-19T16:19:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:09 smithi008 ceph-mon[32573]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:09 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi156 2024-02-19T16:19:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:09 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi156 to 4270M 2024-02-19T16:19:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:09 smithi008 ceph-mon[32573]: osdmap e75: 11 total, 11 up, 11 in 2024-02-19T16:19:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:09 smithi156 ceph-mon[36502]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-02-19T16:19:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:09 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi156 2024-02-19T16:19:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:09 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi156 to 4270M 2024-02-19T16:19:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:09 smithi156 ceph-mon[36502]: osdmap e75: 11 total, 11 up, 11 in 2024-02-19T16:19:10.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 -- 172.21.15.191:0/38398001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8100f80 msgr2=0x7f32c81013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:10.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 --2- 172.21.15.191:0/38398001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8100f80 0x7f32c81013a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009990 tx=0x7f32bc009ca0 comp rx=0 tx=0).stop 2024-02-19T16:19:10.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 -- 172.21.15.191:0/38398001 shutdown_connections 2024-02-19T16:19:10.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 --2- 172.21.15.191:0/38398001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c8102b20 0x7f32c81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:10.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 --2- 172.21.15.191:0/38398001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8100f80 0x7f32c81013a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:10.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 --2- 172.21.15.191:0/38398001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32c8102180 0x7f32c81025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 -- 172.21.15.191:0/38398001 >> 172.21.15.191:0/38398001 conn(0x7f32c80fc540 msgr2=0x7f32c80fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 -- 172.21.15.191:0/38398001 shutdown_connections 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.625+0000 7f32cec8a700 1 -- 172.21.15.191:0/38398001 wait complete. 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 Processor -- start 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 -- start start 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32c8100f80 0x7f32c8104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c8102180 0x7f32c8103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 0x7f32c8103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8104240 con 0x7f32c8100f80 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81043b0 con 0x7f32c8102180 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cec8a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81a2260 con 0x7f32c8102b20 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 0x7f32c8103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 0x7f32c8103aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44960/0 (socket says 172.21.15.191:44960) 2024-02-19T16:19:10.629 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 -- 172.21.15.191:0/2446464048 learned_addr learned my addr 172.21.15.191:0/2446464048 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cca26700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32c8100f80 0x7f32c8104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32c7fff700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c8102180 0x7f32c8103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 -- 172.21.15.191:0/2446464048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c8102180 msgr2=0x7f32c8103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c8102180 0x7f32c8103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 -- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32c8100f80 msgr2=0x7f32c8104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32c8100f80 0x7f32c8104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.626+0000 7f32cd227700 1 -- 172.21.15.191:0/2446464048 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0095f0 con 0x7f32c8102b20 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.627+0000 7f32cd227700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 0x7f32c8103aa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b970 tx=0x7f32c000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.627+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000c930 con 0x7f32c8102b20 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.627+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c81a24f0 con 0x7f32c8102b20 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.627+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c0014440 con 0x7f32c8102b20 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.627+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00075d0 con 0x7f32c8102b20 2024-02-19T16:19:10.630 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.627+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81a2a70 con 0x7f32c8102b20 2024-02-19T16:19:10.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.628+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8068ca0 con 0x7f32c8102b20 2024-02-19T16:19:10.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.629+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f32c00145b0 con 0x7f32c8102b20 2024-02-19T16:19:10.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.629+0000 7f32c5ffb700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f32b80a2150 0x7f32b80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:10.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.629+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f32c0006e40 con 0x7f32c8102b20 2024-02-19T16:19:10.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.630+0000 7f32cca26700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f32b80a2150 0x7f32b80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:10.632 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.630+0000 7f32cca26700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f32b80a2150 0x7f32b80a4610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f32bc003800 tx=0x7f32bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:10.634 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.634+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f32c00962b0 con 0x7f32c8102b20 2024-02-19T16:19:10.838 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:10.837+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f32c806ae40 con 0x7f32b80a2150 2024-02-19T16:19:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:19:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:19:11.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:11.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:19:11.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:19:11.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:11.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T16:19:11.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T16:19:11.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:12.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:11 smithi191 ceph-mon[37203]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:12.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:11 smithi191 ceph-mon[37203]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:19:12.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:11 smithi191 ceph-mon[37203]: osdmap e76: 11 total, 11 up, 11 in 2024-02-19T16:19:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:11 smithi008 ceph-mon[32573]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:11 smithi008 ceph-mon[32573]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:19:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:11 smithi008 ceph-mon[32573]: osdmap e76: 11 total, 11 up, 11 in 2024-02-19T16:19:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:11 smithi156 ceph-mon[36502]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:11 smithi156 ceph-mon[36502]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:19:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:11 smithi156 ceph-mon[36502]: osdmap e76: 11 total, 11 up, 11 in 2024-02-19T16:19:13.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:12 smithi008 ceph-mon[32573]: osdmap e77: 11 total, 11 up, 11 in 2024-02-19T16:19:13.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:12 smithi191 ceph-mon[37203]: osdmap e77: 11 total, 11 up, 11 in 2024-02-19T16:19:13.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:12 smithi156 ceph-mon[36502]: osdmap e77: 11 total, 11 up, 11 in 2024-02-19T16:19:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:13 smithi008 ceph-mon[32573]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-19T16:19:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:13 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/755193497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]: dispatch 2024-02-19T16:19:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:13 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]: dispatch 2024-02-19T16:19:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:13 smithi008 ceph-mon[32573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]': finished 2024-02-19T16:19:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:13 smithi008 ceph-mon[32573]: osdmap e78: 12 total, 11 up, 12 in 2024-02-19T16:19:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:13 smithi191 ceph-mon[37203]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-19T16:19:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:13 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/755193497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]: dispatch 2024-02-19T16:19:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:13 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]: dispatch 2024-02-19T16:19:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:13 smithi191 ceph-mon[37203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]': finished 2024-02-19T16:19:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:13 smithi191 ceph-mon[37203]: osdmap e78: 12 total, 11 up, 12 in 2024-02-19T16:19:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:13 smithi156 ceph-mon[36502]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-19T16:19:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:13 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/755193497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]: dispatch 2024-02-19T16:19:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:13 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]: dispatch 2024-02-19T16:19:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:13 smithi156 ceph-mon[36502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b3c59f-d69e-4a0a-a285-6dd5b20b738a"}]': finished 2024-02-19T16:19:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:13 smithi156 ceph-mon[36502]: osdmap e78: 12 total, 11 up, 12 in 2024-02-19T16:19:14.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:15.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:14 smithi156 ceph-mon[36502]: from='client.? 172.21.15.191:0/1815987151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:19:15.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:14 smithi008 ceph-mon[32573]: from='client.? 172.21.15.191:0/1815987151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:19:15.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:14 smithi191 ceph-mon[37203]: from='client.? 172.21.15.191:0/1815987151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T16:19:16.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:15 smithi008 ceph-mon[32573]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-19T16:19:16.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:15 smithi191 ceph-mon[37203]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-19T16:19:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:15 smithi156 ceph-mon[36502]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-19T16:19:18.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:17 smithi008 ceph-mon[32573]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:19:18.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:17 smithi191 ceph-mon[37203]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:19:18.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:17 smithi156 ceph-mon[36502]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-19T16:19:20.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:19 smithi191 ceph-mon[37203]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-19T16:19:20.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:19 smithi008 ceph-mon[32573]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-19T16:19:20.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:19 smithi156 ceph-mon[36502]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-19T16:19:22.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:21 smithi191 ceph-mon[37203]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:22.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-19T16:19:22.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:22.056 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:21 smithi191 ceph-mon[37203]: Deploying daemon osd.11 on smithi191 2024-02-19T16:19:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:21 smithi008 ceph-mon[32573]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-19T16:19:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:21 smithi008 ceph-mon[32573]: Deploying daemon osd.11 on smithi191 2024-02-19T16:19:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:21 smithi156 ceph-mon[36502]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-19T16:19:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:21 smithi156 ceph-mon[36502]: Deploying daemon osd.11 on smithi191 2024-02-19T16:19:24.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:23 smithi008 ceph-mon[32573]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:24.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:23 smithi191 ceph-mon[37203]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:24.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:23 smithi156 ceph-mon[36502]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:26 smithi008 ceph-mon[32573]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:26.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:26 smithi191 ceph-mon[37203]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:26.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:26 smithi156 ceph-mon[36502]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:28.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:28 smithi191 ceph-mon[37203]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:28.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:28.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:28.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:28 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:28.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:28 smithi008 ceph-mon[32573]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:28.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:28.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:28.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:28 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:28 smithi156 ceph-mon[36502]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:28.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:28 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:30.445 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:30 smithi191 ceph-mon[37203]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:30 smithi008 ceph-mon[32573]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:30.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:30 smithi156 ceph-mon[36502]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:30.906 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 11 on host 'smithi191' 2024-02-19T16:19:30.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.905+0000 7f32c5ffb700 1 -- 172.21.15.191:0/2446464048 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f32c806ae40 con 0x7f32b80a2150 2024-02-19T16:19:30.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.907+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f32b80a2150 msgr2=0x7f32b80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.907+0000 7f32cec8a700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f32b80a2150 0x7f32b80a4610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f32bc003800 tx=0x7f32bc005fd0 comp rx=0 tx=0).stop 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.907+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 msgr2=0x7f32c8103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.907+0000 7f32cec8a700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 0x7f32c8103aa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b970 tx=0x7f32c000bc80 comp rx=0 tx=0).stop 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.907+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 shutdown_connections 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.907+0000 7f32cec8a700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f32b80a2150 0x7f32b80a4610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.908+0000 7f32cec8a700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c8102b20 0x7f32c8103aa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.908+0000 7f32cec8a700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32c8100f80 0x7f32c8104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.908+0000 7f32cec8a700 1 --2- 172.21.15.191:0/2446464048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c8102180 0x7f32c8103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.908+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 >> 172.21.15.191:0/2446464048 conn(0x7f32c80fc540 msgr2=0x7f32c806a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.908+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 shutdown_connections 2024-02-19T16:19:30.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:30.908+0000 7f32cec8a700 1 -- 172.21.15.191:0/2446464048 wait complete. 2024-02-19T16:19:31.539 DEBUG:teuthology.orchestra.run.smithi191:osd.11> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.11.service 2024-02-19T16:19:31.542 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-19T16:19:31.542 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd stat -f json 2024-02-19T16:19:31.795 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:19:32.216 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:32.216 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.216 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.217 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: from='osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-19T16:19:32.217 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-19T16:19:32.217 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.217 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:31 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: from='osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:31 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:32.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: from='osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-19T16:19:32.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-19T16:19:32.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:32.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:31 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:33.184 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:32 smithi008 ceph-mon[32573]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-19T16:19:33.184 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:32 smithi008 ceph-mon[32573]: osdmap e79: 12 total, 11 up, 12 in 2024-02-19T16:19:33.184 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:32 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:33.184 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:32 smithi008 ceph-mon[32573]: from='osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:19:33.184 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:32 smithi008 ceph-mon[32573]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:19:33.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-mon[37203]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-19T16:19:33.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-mon[37203]: osdmap e79: 12 total, 11 up, 12 in 2024-02-19T16:19:33.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:33.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-mon[37203]: from='osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:19:33.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-mon[37203]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:19:33.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:19:32.876+0000 7fdc53ecd700 -1 osd.11 0 waiting for initial osdmap 2024-02-19T16:19:33.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:19:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:19:32.890+0000 7fdc4fcc0700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T16:19:33.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:32 smithi156 ceph-mon[36502]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-19T16:19:33.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:32 smithi156 ceph-mon[36502]: osdmap e79: 12 total, 11 up, 12 in 2024-02-19T16:19:33.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:32 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:33.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:32 smithi156 ceph-mon[36502]: from='osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:19:33.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:32 smithi156 ceph-mon[36502]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-19T16:19:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 -- 172.21.15.8:0/2566190605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102070 msgr2=0x7fd5841024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2566190605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102070 0x7fd5841024f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd578009990 tx=0x7fd578009ca0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 -- 172.21.15.8:0/2566190605 shutdown_connections 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2566190605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584102a30 0x7fd584109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2566190605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584100f10 0x7fd584101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2566190605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102070 0x7fd5841024f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 -- 172.21.15.8:0/2566190605 >> 172.21.15.8:0/2566190605 conn(0x7fd5840fc4b0 msgr2=0x7fd5840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 -- 172.21.15.8:0/2566190605 shutdown_connections 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.719+0000 7fd58baa3700 1 -- 172.21.15.8:0/2566190605 wait complete. 2024-02-19T16:19:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 Processor -- start 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 -- start start 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584100f10 0x7fd584197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584102070 0x7fd584198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 0x7fd58419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419fb60 con 0x7fd584102a30 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419fca0 con 0x7fd584102070 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58baa3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419fe30 con 0x7fd584100f10 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 0x7fd58419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58983f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584100f10 0x7fd584197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 0x7fd58419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40594/0 (socket says 172.21.15.8:40594) 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 -- 172.21.15.8:0/2648660578 learned_addr learned my addr 172.21.15.8:0/2648660578 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58983f700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584100f10 0x7fd584197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:50792/0 (socket says 172.21.15.8:50792) 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58903e700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584102070 0x7fd584198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 -- 172.21.15.8:0/2648660578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584100f10 msgr2=0x7fd584197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584100f10 0x7fd584197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 -- 172.21.15.8:0/2648660578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584102070 msgr2=0x7fd584198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584102070 0x7fd584198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.720+0000 7fd58a040700 1 -- 172.21.15.8:0/2648660578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780095f0 con 0x7fd584102a30 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.721+0000 7fd58a040700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 0x7fd58419f590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd57400d9b0 tx=0x7fd57400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.721+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574004a30 con 0x7fd584102a30 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.721+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5841a0110 con 0x7fd584102a30 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.721+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57400b5c0 con 0x7fd584102a30 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.721+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574009ab0 con 0x7fd584102a30 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.721+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841a0660 con 0x7fd584102a30 2024-02-19T16:19:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.722+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5841082c0 con 0x7fd584102a30 2024-02-19T16:19:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.723+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd574009c30 con 0x7fd584102a30 2024-02-19T16:19:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.723+0000 7fd582ffd700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd56c0a21d0 0x7fd56c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.724+0000 7fd58983f700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd56c0a21d0 0x7fd56c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.724+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fd5740cd950 con 0x7fd584102a30 2024-02-19T16:19:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.724+0000 7fd58983f700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd56c0a21d0 0x7fd56c0a4690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009550 tx=0x7fd57c0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.727+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd574096e40 con 0x7fd584102a30 2024-02-19T16:19:33.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.935+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd58410db20 con 0x7fd584102a30 2024-02-19T16:19:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.936+0000 7fd582ffd700 1 -- 172.21.15.8:0/2648660578 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7fd5740170a0 con 0x7fd584102a30 2024-02-19T16:19:33.937 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd56c0a21d0 msgr2=0x7fd56c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd56c0a21d0 0x7fd56c0a4690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009550 tx=0x7fd57c0091b0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 msgr2=0x7fd58419f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 0x7fd58419f590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd57400d9b0 tx=0x7fd57400dcc0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 shutdown_connections 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd56c0a21d0 0x7fd56c0a4690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd584102a30 0x7fd58419f590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd584100f10 0x7fd584197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 --2- 172.21.15.8:0/2648660578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584102070 0x7fd584198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 >> 172.21.15.8:0/2648660578 conn(0x7fd5840fc4b0 msgr2=0x7fd584105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 shutdown_connections 2024-02-19T16:19:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:33.938+0000 7fd58baa3700 1 -- 172.21.15.8:0/2648660578 wait complete. 2024-02-19T16:19:34.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:33 smithi008 ceph-mon[32573]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:34.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:33 smithi008 ceph-mon[32573]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:19:34.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:33 smithi008 ceph-mon[32573]: osdmap e80: 12 total, 11 up, 12 in 2024-02-19T16:19:34.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:33 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:34.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:33 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:34.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:33 smithi191 ceph-mon[37203]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:34.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:33 smithi191 ceph-mon[37203]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:19:34.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:33 smithi191 ceph-mon[37203]: osdmap e80: 12 total, 11 up, 12 in 2024-02-19T16:19:34.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:33 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:34.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:33 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:34.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:33 smithi156 ceph-mon[36502]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:34.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:33 smithi156 ceph-mon[36502]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-19T16:19:34.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:33 smithi156 ceph-mon[36502]: osdmap e80: 12 total, 11 up, 12 in 2024-02-19T16:19:34.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:33 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:34.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:33 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:34.645 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1708359573,"num_in_osds":12,"osd_in_since":1708359553,"num_remapped_pgs":0} 2024-02-19T16:19:34.646 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd dump --format=json 2024-02-19T16:19:34.898 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:19:35.151 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:34 smithi008 ceph-mon[32573]: purged_snaps scrub starts 2024-02-19T16:19:35.151 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:34 smithi008 ceph-mon[32573]: purged_snaps scrub ok 2024-02-19T16:19:35.151 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:34 smithi008 ceph-mon[32573]: osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] boot 2024-02-19T16:19:35.151 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:34 smithi008 ceph-mon[32573]: osdmap e81: 12 total, 12 up, 12 in 2024-02-19T16:19:35.151 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:34 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:35.151 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:34 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2648660578' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T16:19:35.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:34 smithi191 ceph-mon[37203]: purged_snaps scrub starts 2024-02-19T16:19:35.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:34 smithi191 ceph-mon[37203]: purged_snaps scrub ok 2024-02-19T16:19:35.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:34 smithi191 ceph-mon[37203]: osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] boot 2024-02-19T16:19:35.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:34 smithi191 ceph-mon[37203]: osdmap e81: 12 total, 12 up, 12 in 2024-02-19T16:19:35.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:34 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:35.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:34 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2648660578' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T16:19:35.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:34 smithi156 ceph-mon[36502]: purged_snaps scrub starts 2024-02-19T16:19:35.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:34 smithi156 ceph-mon[36502]: purged_snaps scrub ok 2024-02-19T16:19:35.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:34 smithi156 ceph-mon[36502]: osd.11 [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] boot 2024-02-19T16:19:35.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:34 smithi156 ceph-mon[36502]: osdmap e81: 12 total, 12 up, 12 in 2024-02-19T16:19:35.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:34 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:19:35.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:34 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2648660578' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T16:19:36.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: osdmap e82: 12 total, 12 up, 12 in 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: Detected new or changed devices on smithi191 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi191 to 4270M 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:36.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:35 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:36.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: osdmap e82: 12 total, 12 up, 12 in 2024-02-19T16:19:36.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: Detected new or changed devices on smithi191 2024-02-19T16:19:36.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi191 to 4270M 2024-02-19T16:19:36.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:36.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:35 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-02-19T16:19:36.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: osdmap e82: 12 total, 12 up, 12 in 2024-02-19T16:19:36.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: Detected new or changed devices on smithi191 2024-02-19T16:19:36.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi191 to 4270M 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:36.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:36.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:36.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:36.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:36.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:35 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:36.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.730+0000 7fe1064ce700 1 -- 172.21.15.8:0/1535350531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006b000 msgr2=0x7fe10010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.730+0000 7fe1064ce700 1 --2- 172.21.15.8:0/1535350531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006b000 0x7fe10010e580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009990 tx=0x7fe0f4009ca0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 -- 172.21.15.8:0/1535350531 shutdown_connections 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 --2- 172.21.15.8:0/1535350531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10010eac0 0x7fe100110eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 --2- 172.21.15.8:0/1535350531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10006a6e0 0x7fe10006aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 --2- 172.21.15.8:0/1535350531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006b000 0x7fe10010e580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 -- 172.21.15.8:0/1535350531 >> 172.21.15.8:0/1535350531 conn(0x7fe100076f00 msgr2=0x7fe100077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 -- 172.21.15.8:0/1535350531 shutdown_connections 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 -- 172.21.15.8:0/1535350531 wait complete. 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 Processor -- start 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.731+0000 7fe1064ce700 1 -- start start 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 0x7fe1001a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10006b000 0x7fe1001a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:36.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10010eac0 0x7fe10019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10019ac70 con 0x7fe10006a6e0 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10019adb0 con 0x7fe10010eac0 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10019af70 con 0x7fe10006b000 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 0x7fe1001a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 0x7fe1001a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54718/0 (socket says 172.21.15.8:54718) 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 -- 172.21.15.8:0/3853965648 learned_addr learned my addr 172.21.15.8:0/3853965648 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe104a6b700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10010eac0 0x7fe10019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0ff7fe700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10006b000 0x7fe1001a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 -- 172.21.15.8:0/3853965648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10006b000 msgr2=0x7fe1001a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10006b000 0x7fe1001a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 -- 172.21.15.8:0/3853965648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10010eac0 msgr2=0x7fe10019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:36.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10010eac0 0x7fe10019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 -- 172.21.15.8:0/3853965648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40095f0 con 0x7fe10006a6e0 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe104a6b700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10010eac0 0x7fe10019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0ff7fe700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10006b000 0x7fe1001a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fffff700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 0x7fe1001a0450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000b530 tx=0x7fe0f000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0011830 con 0x7fe10006a6e0 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10019b250 con 0x7fe10006a6e0 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0f0011e70 con 0x7fe10006a6e0 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000f2e0 con 0x7fe10006a6e0 2024-02-19T16:19:36.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.732+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1001a74f0 con 0x7fe10006a6e0 2024-02-19T16:19:36.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.733+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe100109ce0 con 0x7fe10006a6e0 2024-02-19T16:19:36.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.735+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe0f00105f0 con 0x7fe10006a6e0 2024-02-19T16:19:36.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.736+0000 7fe0fd7fa700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe0e80a2150 0x7fe0e80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:36.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.736+0000 7fe0ff7fe700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe0e80a2150 0x7fe0e80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:36.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.736+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0f00cd540 con 0x7fe10006a6e0 2024-02-19T16:19:36.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.736+0000 7fe0ff7fe700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe0e80a2150 0x7fe0e80a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b5c0 tx=0x7fe0f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:36.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.739+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe0f0096910 con 0x7fe10006a6e0 2024-02-19T16:19:36.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.942+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1001014f0 con 0x7fe10006a6e0 2024-02-19T16:19:36.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.943+0000 7fe0fd7fa700 1 -- 172.21.15.8:0/3853965648 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18373 (secure 0 0 0) 0x7fe0f0010380 con 0x7fe10006a6e0 2024-02-19T16:19:36.944 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:36.944 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":82,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:19:34.880156+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:19:36.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe0e80a2150 msgr2=0x7fe0e80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe0e80a2150 0x7fe0e80a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b5c0 tx=0x7fe0f4005fb0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 msgr2=0x7fe1001a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 0x7fe1001a0450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000b530 tx=0x7fe0f000b8f0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 shutdown_connections 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe0e80a2150 0x7fe0e80a4610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10010eac0 0x7fe10019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe10006a6e0 0x7fe1001a0450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 --2- 172.21.15.8:0/3853965648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe10006b000 0x7fe1001a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 >> 172.21.15.8:0/3853965648 conn(0x7fe100076f00 msgr2=0x7fe10010ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 shutdown_connections 2024-02-19T16:19:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:36.945+0000 7fe1064ce700 1 -- 172.21.15.8:0/3853965648 wait complete. 2024-02-19T16:19:37.646 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T16:19:37.646 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-19T16:19:37.889 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:19:38.193 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:37 smithi008 ceph-mon[32573]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:38.193 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3853965648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:19:38.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:37 smithi191 ceph-mon[37203]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:38.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3853965648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:19:38.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:37 smithi156 ceph-mon[36502]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:38.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3853965648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:19:39.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 -- 172.21.15.8:0/1696447440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 msgr2=0x7f5f8c108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:39.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 --2- 172.21.15.8:0/1696447440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c108440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0099c0 tx=0x7f5f7c009cd0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 -- 172.21.15.8:0/1696447440 shutdown_connections 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 --2- 172.21.15.8:0/1696447440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f8c06b080 0x7f5f8c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 --2- 172.21.15.8:0/1696447440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c108440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 --2- 172.21.15.8:0/1696447440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06a6e0 0x7f5f8c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 -- 172.21.15.8:0/1696447440 >> 172.21.15.8:0/1696447440 conn(0x7f5f8c076f00 msgr2=0x7f5f8c077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 -- 172.21.15.8:0/1696447440 shutdown_connections 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.645+0000 7f5f92491700 1 -- 172.21.15.8:0/1696447440 wait complete. 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 Processor -- start 2024-02-19T16:19:39.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 -- start start 2024-02-19T16:19:39.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f8c06a6e0 0x7f5f8c100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06b080 0x7f5f8c101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1a6970 con 0x7f5f8c108060 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1a6ae0 con 0x7f5f8c06a6e0 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f92491700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1a6ca0 con 0x7f5f8c06b080 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f90a2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f8b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06b080 0x7f5f8c101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f8bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f8c06a6e0 0x7f5f8c100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f8b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06b080 0x7f5f8c101520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:43558/0 (socket says 172.21.15.8:43558) 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f8b7fe700 1 -- 172.21.15.8:0/2057745089 learned_addr learned my addr 172.21.15.8:0/2057745089 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.646+0000 7f5f90a2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54732/0 (socket says 172.21.15.8:54732) 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f90a2e700 1 -- 172.21.15.8:0/2057745089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06b080 msgr2=0x7f5f8c101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f90a2e700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06b080 0x7f5f8c101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f90a2e700 1 -- 172.21.15.8:0/2057745089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f8c06a6e0 msgr2=0x7f5f8c100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f90a2e700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f8c06a6e0 0x7f5f8c100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f90a2e700 1 -- 172.21.15.8:0/2057745089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0095f0 con 0x7f5f8c108060 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f90a2e700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c101a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007f10 tx=0x7f5f8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84004db0 con 0x7f5f8c108060 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f8400f040 con 0x7f5f8c108060 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c1a6f80 con 0x7f5f8c108060 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84005a90 con 0x7f5f8c108060 2024-02-19T16:19:39.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.647+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c1a7400 con 0x7f5f8c108060 2024-02-19T16:19:39.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.648+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c10a580 con 0x7f5f8c108060 2024-02-19T16:19:39.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.649+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5f840050b0 con 0x7f5f8c108060 2024-02-19T16:19:39.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.650+0000 7f5f897fa700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5f780a2150 0x7f5f780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:39.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.650+0000 7f5f8bfff700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5f780a2150 0x7f5f780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:39.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.650+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5f840cd020 con 0x7f5f8c108060 2024-02-19T16:19:39.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.650+0000 7f5f8bfff700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5f780a2150 0x7f5f780a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009990 tx=0x7f5f7c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:39.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.654+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5f84096490 con 0x7f5f8c108060 2024-02-19T16:19:39.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.855+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f8c102510 con 0x7f5f8c108060 2024-02-19T16:19:39.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.855+0000 7f5f897fa700 1 -- 172.21.15.8:0/2057745089 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f84099b50 con 0x7f5f8c108060 2024-02-19T16:19:39.856 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5f780a2150 msgr2=0x7f5f780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5f780a2150 0x7f5f780a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009990 tx=0x7f5f7c005a90 comp rx=0 tx=0).stop 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 msgr2=0x7f5f8c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c101a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007f10 tx=0x7f5f8400d3b0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 shutdown_connections 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5f780a2150 0x7f5f780a4610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f8c108060 0x7f5f8c101a60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f8c06a6e0 0x7f5f8c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 --2- 172.21.15.8:0/2057745089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f8c06b080 0x7f5f8c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:39.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 >> 172.21.15.8:0/2057745089 conn(0x7f5f8c076f00 msgr2=0x7f5f8c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:39.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 shutdown_connections 2024-02-19T16:19:39.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:39.857+0000 7f5f92491700 1 -- 172.21.15.8:0/2057745089 wait complete. 2024-02-19T16:19:40.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:39 smithi008 ceph-mon[32573]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:40.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2057745089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:19:40.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:39 smithi191 ceph-mon[37203]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:40.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2057745089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:19:40.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:39 smithi156 ceph-mon[36502]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:40.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2057745089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:19:40.640 INFO:tasks.cephadm:Adding mds.a on smithi008 2024-02-19T16:19:40.640 INFO:tasks.cephadm:Adding mds.d on smithi008 2024-02-19T16:19:40.640 INFO:tasks.cephadm:Adding mds.g on smithi008 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.j on smithi008 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.b on smithi156 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.e on smithi156 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.h on smithi156 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.k on smithi156 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.c on smithi191 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.f on smithi191 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.i on smithi191 2024-02-19T16:19:40.641 INFO:tasks.cephadm:Adding mds.l on smithi191 2024-02-19T16:19:40.641 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph orch apply mds all '12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l' 2024-02-19T16:19:40.891 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:19:42.268 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:41 smithi191 ceph-mon[37203]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:42.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:41 smithi008 ceph-mon[32573]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:42.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:41 smithi156 ceph-mon[36502]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 -- 172.21.15.191:0/3835264612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524100f10 msgr2=0x7f1524101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 --2- 172.21.15.191:0/3835264612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524100f10 0x7f1524101330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1518009990 tx=0x7f1518009ca0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 -- 172.21.15.191:0/3835264612 shutdown_connections 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 --2- 172.21.15.191:0/3835264612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1524102a30 0x7f1524109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 --2- 172.21.15.191:0/3835264612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524100f10 0x7f1524101330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 --2- 172.21.15.191:0/3835264612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1524102070 0x7f15241024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 -- 172.21.15.191:0/3835264612 >> 172.21.15.191:0/3835264612 conn(0x7f15240fc4b0 msgr2=0x7f15240fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:42.586 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 -- 172.21.15.191:0/3835264612 shutdown_connections 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.583+0000 7f15297ad700 1 -- 172.21.15.191:0/3835264612 wait complete. 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 Processor -- start 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 -- start start 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1524100f10 0x7f152406a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1524102070 0x7f152419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 0x7f152419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419fb20 con 0x7f1524100f10 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419fc90 con 0x7f1524102070 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15297ad700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419fe50 con 0x7f1524102a30 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15237fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 0x7f152419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15237fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 0x7f152419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52104/0 (socket says 172.21.15.191:52104) 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15237fe700 1 -- 172.21.15.191:0/1736512244 learned_addr learned my addr 172.21.15.191:0/1736512244 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f15227fc700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1524102070 0x7f152419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:42.587 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.584+0000 7f1522ffd700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1524100f10 0x7f152406a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15237fe700 1 -- 172.21.15.191:0/1736512244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1524102070 msgr2=0x7f152419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15237fe700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1524102070 0x7f152419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15237fe700 1 -- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1524100f10 msgr2=0x7f152406a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15237fe700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1524100f10 0x7f152406a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15237fe700 1 -- 172.21.15.191:0/1736512244 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180095f0 con 0x7f1524102a30 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15237fe700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 0x7f152419f380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f151c007ee0 tx=0x7f151c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c017070 con 0x7f1524102a30 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15241a0130 con 0x7f1524102a30 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f151c00f040 con 0x7f1524102a30 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c005650 con 0x7f1524102a30 2024-02-19T16:19:42.588 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.585+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15241a05e0 con 0x7f1524102a30 2024-02-19T16:19:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.586+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15241082c0 con 0x7f1524102a30 2024-02-19T16:19:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.587+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f151c004cd0 con 0x7f1524102a30 2024-02-19T16:19:42.589 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.587+0000 7f150bfff700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f150c0a2150 0x7f150c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.588+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f151c0ccf50 con 0x7f1524102a30 2024-02-19T16:19:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.588+0000 7f1522ffd700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f150c0a2150 0x7f150c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:42.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.588+0000 7f1522ffd700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f150c0a2150 0x7f150c0a4610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f151800b5c0 tx=0x7f1518003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:42.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.592+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f151c0963c0 con 0x7f1524102a30 2024-02-19T16:19:42.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.794+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f152402da90 con 0x7f150c0a2150 2024-02-19T16:19:42.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.803+0000 7f150bfff700 1 -- 172.21.15.191:0/1736512244 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f152402da90 con 0x7f150c0a2150 2024-02-19T16:19:42.804 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mds.all update... 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f150c0a2150 msgr2=0x7f150c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f150c0a2150 0x7f150c0a4610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f151800b5c0 tx=0x7f1518003860 comp rx=0 tx=0).stop 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 msgr2=0x7f152419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 0x7f152419f380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f151c007ee0 tx=0x7f151c00d3b0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 shutdown_connections 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f150c0a2150 0x7f150c0a4610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1524102a30 0x7f152419f380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.806 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1524100f10 0x7f152406a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 --2- 172.21.15.191:0/1736512244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1524102070 0x7f152419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 >> 172.21.15.191:0/1736512244 conn(0x7f15240fc4b0 msgr2=0x7f1524105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 shutdown_connections 2024-02-19T16:19:42.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-19T16:19:42.805+0000 7f15297ad700 1 -- 172.21.15.191:0/1736512244 wait complete. 2024-02-19T16:19:43.447 DEBUG:teuthology.orchestra.run.smithi008:mds.a> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.a.service 2024-02-19T16:19:43.450 DEBUG:teuthology.orchestra.run.smithi008:mds.d> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.d.service 2024-02-19T16:19:43.493 DEBUG:teuthology.orchestra.run.smithi008:mds.g> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.g.service 2024-02-19T16:19:43.536 DEBUG:teuthology.orchestra.run.smithi008:mds.j> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.j.service 2024-02-19T16:19:43.579 DEBUG:teuthology.orchestra.run.smithi156:mds.b> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.b.service 2024-02-19T16:19:43.582 DEBUG:teuthology.orchestra.run.smithi156:mds.e> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.e.service 2024-02-19T16:19:43.625 DEBUG:teuthology.orchestra.run.smithi156:mds.h> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.h.service 2024-02-19T16:19:43.668 DEBUG:teuthology.orchestra.run.smithi156:mds.k> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.k.service 2024-02-19T16:19:43.711 DEBUG:teuthology.orchestra.run.smithi191:mds.c> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.c.service 2024-02-19T16:19:43.714 DEBUG:teuthology.orchestra.run.smithi191:mds.f> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.f.service 2024-02-19T16:19:43.756 DEBUG:teuthology.orchestra.run.smithi191:mds.i> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.i.service 2024-02-19T16:19:43.800 DEBUG:teuthology.orchestra.run.smithi191:mds.l> sudo journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.l.service 2024-02-19T16:19:43.842 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-19T16:19:43.843 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:19:43.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.955+0000 7f84d1383640 1 -- 172.21.15.8:0/1274774584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc101040 msgr2=0x7f84cc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:43.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.955+0000 7f84d1383640 1 --2- 172.21.15.8:0/1274774584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc101040 0x7f84cc1088e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009ca0 tx=0x7f84bc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 -- 172.21.15.8:0/1274774584 shutdown_connections 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 --2- 172.21.15.8:0/1274774584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc101040 0x7f84cc1088e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 --2- 172.21.15.8:0/1274774584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84cc100680 0x7f84cc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 --2- 172.21.15.8:0/1274774584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84cc101a30 0x7f84cc101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 -- 172.21.15.8:0/1274774584 >> 172.21.15.8:0/1274774584 conn(0x7f84cc0fb110 msgr2=0x7f84cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 -- 172.21.15.8:0/1274774584 shutdown_connections 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 -- 172.21.15.8:0/1274774584 wait complete. 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 Processor -- start 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 -- start start 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.956+0000 7f84d1383640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 0x7f84cc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84cc101040 0x7f84cc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:43.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84cc101a30 0x7f84cc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc19f080 con 0x7f84cc100680 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc19f1f0 con 0x7f84cc101a30 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc19f3b0 con 0x7f84cc101040 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 0x7f84cc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 0x7f84cc197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54742/0 (socket says 172.21.15.8:54742) 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 -- 172.21.15.8:0/4155981336 learned_addr learned my addr 172.21.15.8:0/4155981336 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84ca7fc640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84cc101040 0x7f84cc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 -- 172.21.15.8:0/4155981336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84cc101040 msgr2=0x7f84cc197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84cc101040 0x7f84cc197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84cb7fe640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84cc101a30 0x7f84cc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 -- 172.21.15.8:0/4155981336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84cc101a30 msgr2=0x7f84cc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84cc101a30 0x7f84cc19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 -- 172.21.15.8:0/4155981336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009490 con 0x7f84cc100680 2024-02-19T16:19:43.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84caffd640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 0x7f84cc197590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9c0 tx=0x7f84c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:43.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400cde0 con 0x7f84cc100680 2024-02-19T16:19:43.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc009900 con 0x7f84cc100680 2024-02-19T16:19:43.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84c4010910 con 0x7f84cc100680 2024-02-19T16:19:43.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018960 con 0x7f84cc100680 2024-02-19T16:19:43.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.957+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc19fa60 con 0x7f84cc100680 2024-02-19T16:19:43.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.958+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc10be70 con 0x7f84cc100680 2024-02-19T16:19:43.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.959+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f84c4010a80 con 0x7f84cc100680 2024-02-19T16:19:43.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.960+0000 7f84c3fff640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f84b80a20c0 0x7f84b80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:43.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.960+0000 7f84ca7fc640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f84b80a20c0 0x7f84b80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:43.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.960+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f84c4014070 con 0x7f84cc100680 2024-02-19T16:19:43.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.960+0000 7f84ca7fc640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f84b80a20c0 0x7f84b80a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84cc198aa0 tx=0x7f84b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:43.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:43.964+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f84c40972d0 con 0x7f84cc100680 2024-02-19T16:19:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:19:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: Saving service mds.all spec with placement smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-19T16:19:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:44.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:44.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:44.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:44.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:44.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:44.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:44.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:43 smithi191 ceph-mon[37203]: Deploying daemon mds.l on smithi191 2024-02-19T16:19:44.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: Saving service mds.all spec with placement smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:44.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:44.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:44.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:44.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:44.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:43 smithi156 ceph-mon[36502]: Deploying daemon mds.l on smithi191 2024-02-19T16:19:44.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.147+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84cc10c000 con 0x7f84cc100680 2024-02-19T16:19:44.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.148+0000 7f84c3fff640 1 -- 172.21.15.8:0/4155981336 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18373 (secure 0 0 0) 0x7f84c409a990 con 0x7f84cc100680 2024-02-19T16:19:44.149 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:44.149 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":82,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:19:34.880156+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:19:44.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f84b80a20c0 msgr2=0x7f84b80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f84b80a20c0 0x7f84b80a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84cc198aa0 tx=0x7f84b4005f90 comp rx=0 tx=0).stop 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 msgr2=0x7f84cc197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 0x7f84cc197590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9c0 tx=0x7f84c400ee80 comp rx=0 tx=0).stop 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 shutdown_connections 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f84b80a20c0 0x7f84b80a4580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84cc101a30 0x7f84cc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84cc101040 0x7f84cc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 --2- 172.21.15.8:0/4155981336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84cc100680 0x7f84cc197590 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 >> 172.21.15.8:0/4155981336 conn(0x7f84cc0fb110 msgr2=0x7f84cc0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:44.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 shutdown_connections 2024-02-19T16:19:44.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.150+0000 7f84d1383640 1 -- 172.21.15.8:0/4155981336 wait complete. 2024-02-19T16:19:44.160 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: Saving service mds.all spec with placement smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:44.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:43 smithi008 ceph-mon[32573]: Deploying daemon mds.l on smithi191 2024-02-19T16:19:44.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T16:19:44.163 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:19:44.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 -- 172.21.15.8:0/3821551144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798100410 msgr2=0x7f2798100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 --2- 172.21.15.8:0/3821551144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798100410 0x7f2798100830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:44.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 -- 172.21.15.8:0/3821551144 shutdown_connections 2024-02-19T16:19:44.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 --2- 172.21.15.8:0/3821551144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2798101fc0 0x7f2798108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 --2- 172.21.15.8:0/3821551144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101600 0x7f2798101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 --2- 172.21.15.8:0/3821551144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798100410 0x7f2798100830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 -- 172.21.15.8:0/3821551144 >> 172.21.15.8:0/3821551144 conn(0x7f27980fb9a0 msgr2=0x7f27980fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 -- 172.21.15.8:0/3821551144 shutdown_connections 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.268+0000 7f279ee40640 1 -- 172.21.15.8:0/3821551144 wait complete. 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 Processor -- start 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 -- start start 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2798100410 0x7f279806a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101600 0x7f279819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 0x7f279819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819f0c0 con 0x7f2798101fc0 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819f230 con 0x7f2798101600 2024-02-19T16:19:44.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279ee40640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819f3f0 con 0x7f2798100410 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 0x7f279819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 0x7f279819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54760/0 (socket says 172.21.15.8:54760) 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 -- 172.21.15.8:0/863450012 learned_addr learned my addr 172.21.15.8:0/863450012 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279cbdc640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2798100410 0x7f279806a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 -- 172.21.15.8:0/863450012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2798100410 msgr2=0x7f279806a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2798100410 0x7f279806a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 -- 172.21.15.8:0/863450012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101600 msgr2=0x7f279819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101600 0x7f279819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 -- 172.21.15.8:0/863450012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c009560 con 0x7f2798101fc0 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f279d3dd640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 0x7f279819e930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f279000eaa0 tx=0x7f279000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.269+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000cdb0 con 0x7f2798101fc0 2024-02-19T16:19:44.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.270+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279819f6d0 con 0x7f2798101fc0 2024-02-19T16:19:44.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.270+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2790005020 con 0x7f2798101fc0 2024-02-19T16:19:44.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.270+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27900116b0 con 0x7f2798101fc0 2024-02-19T16:19:44.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.270+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279819fbe0 con 0x7f2798101fc0 2024-02-19T16:19:44.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.271+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2798107730 con 0x7f2798101fc0 2024-02-19T16:19:44.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.272+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2790005520 con 0x7f2798101fc0 2024-02-19T16:19:44.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.272+0000 7f2795ffb640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f27880a20c0 0x7f27880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.272+0000 7f279cbdc640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f27880a20c0 0x7f27880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.272+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27900cdb60 con 0x7f2798101fc0 2024-02-19T16:19:44.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.273+0000 7f279cbdc640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f27880a20c0 0x7f27880a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f278c005ab0 tx=0x7f278c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:44.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.277+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2790096f30 con 0x7f2798101fc0 2024-02-19T16:19:44.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.459+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f279802d000 con 0x7f2798101fc0 2024-02-19T16:19:44.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.459+0000 7f2795ffb640 1 -- 172.21.15.8:0/863450012 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f279009a5f0 con 0x7f2798101fc0 2024-02-19T16:19:44.460 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f27880a20c0 msgr2=0x7f27880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f27880a20c0 0x7f27880a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f278c005ab0 tx=0x7f278c005fd0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 msgr2=0x7f279819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 0x7f279819e930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f279000eaa0 tx=0x7f279000ef60 comp rx=0 tx=0).stop 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 shutdown_connections 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f27880a20c0 0x7f27880a4580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2798101fc0 0x7f279819e930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101600 0x7f279819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 --2- 172.21.15.8:0/863450012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2798100410 0x7f279806a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 >> 172.21.15.8:0/863450012 conn(0x7f27980fb9a0 msgr2=0x7f2798104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:44.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 shutdown_connections 2024-02-19T16:19:44.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.462+0000 7f279ee40640 1 -- 172.21.15.8:0/863450012 wait complete. 2024-02-19T16:19:44.474 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:44.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 -- 172.21.15.8:0/592291510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794100410 msgr2=0x7f5794100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 --2- 172.21.15.8:0/592291510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794100410 0x7f5794100830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5784009900 tx=0x7f578402bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 -- 172.21.15.8:0/592291510 shutdown_connections 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 --2- 172.21.15.8:0/592291510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5794101fc0 0x7f5794108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 --2- 172.21.15.8:0/592291510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5794101600 0x7f5794101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 --2- 172.21.15.8:0/592291510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794100410 0x7f5794100830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 -- 172.21.15.8:0/592291510 >> 172.21.15.8:0/592291510 conn(0x7f57940fb980 msgr2=0x7f57940fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 -- 172.21.15.8:0/592291510 shutdown_connections 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.575+0000 7f5799223640 1 -- 172.21.15.8:0/592291510 wait complete. 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 Processor -- start 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 -- start start 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5794100410 0x7f57941975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5794101600 0x7f5794197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 0x7f579419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579419f0e0 con 0x7f5794101fc0 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579419f250 con 0x7f5794101600 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5799223640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579419f410 con 0x7f5794100410 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f579359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 0x7f579419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f579359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 0x7f579419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54778/0 (socket says 172.21.15.8:54778) 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f5792d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5794100410 0x7f57941975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.576+0000 7f579359e640 1 -- 172.21.15.8:0/3856905416 learned_addr learned my addr 172.21.15.8:0/3856905416 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579259c640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5794101600 0x7f5794197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579359e640 1 -- 172.21.15.8:0/3856905416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5794100410 msgr2=0x7f57941975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579359e640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5794100410 0x7f57941975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579359e640 1 -- 172.21.15.8:0/3856905416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5794101600 msgr2=0x7f5794197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579359e640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5794101600 0x7f5794197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579359e640 1 -- 172.21.15.8:0/3856905416 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5784009560 con 0x7f5794101fc0 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f579359e640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 0x7f579419eba0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f578c00eaa0 tx=0x7f578c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00cdb0 con 0x7f5794101fc0 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579419f6f0 con 0x7f5794101fc0 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f578c005020 con 0x7f5794101fc0 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0116d0 con 0x7f5794101fc0 2024-02-19T16:19:44.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.577+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579419fc50 con 0x7f5794101fc0 2024-02-19T16:19:44.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.578+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579410be20 con 0x7f5794101fc0 2024-02-19T16:19:44.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.579+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f578c005490 con 0x7f5794101fc0 2024-02-19T16:19:44.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.580+0000 7f577bfff640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f577c0a20c0 0x7f577c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:44.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.580+0000 7f5792d9d640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f577c0a20c0 0x7f577c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:44.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.580+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f578c0cd930 con 0x7f5794101fc0 2024-02-19T16:19:44.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.580+0000 7f5792d9d640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f577c0a20c0 0x7f577c0a4580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5784005ab0 tx=0x7f5784005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:44.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.584+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f578c096e50 con 0x7f5794101fc0 2024-02-19T16:19:44.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.804+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57941a00f0 con 0x7f5794101fc0 2024-02-19T16:19:44.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.804+0000 7f577bfff640 1 -- 172.21.15.8:0/3856905416 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f578c01f230 con 0x7f5794101fc0 2024-02-19T16:19:44.805 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:44.805 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-19T16:19:44.806 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 1 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f577c0a20c0 msgr2=0x7f577c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f577c0a20c0 0x7f577c0a4580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5784005ab0 tx=0x7f5784005fd0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 msgr2=0x7f579419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 0x7f579419eba0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f578c00eaa0 tx=0x7f578c00ef60 comp rx=0 tx=0).stop 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 shutdown_connections 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f577c0a20c0 0x7f577c0a4580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5794101fc0 0x7f579419eba0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5794101600 0x7f5794197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 --2- 172.21.15.8:0/3856905416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5794100410 0x7f57941975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.806+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 >> 172.21.15.8:0/3856905416 conn(0x7f57940fb980 msgr2=0x7f57940fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.807+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 shutdown_connections 2024-02-19T16:19:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:44.807+0000 7f5799223640 1 -- 172.21.15.8:0/3856905416 wait complete. 2024-02-19T16:19:45.078 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4155981336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:19:45.078 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/863450012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:19:45.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4155981336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:19:45.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/863450012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:19:45.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4155981336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:19:45.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/863450012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:19:45.777 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:45 smithi191 systemd[1]: Starting Ceph mds.l for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:19:46.034 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:45 smithi191 ceph-mon[37203]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:46.034 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:45 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3856905416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:46.034 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:45 smithi191 podman[66879]: 2024-02-19 16:19:45.681873371 +0000 UTC m=+0.017635474 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:19:46.034 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:45 smithi191 podman[66879]: 2024-02-19 16:19:45.780775727 +0000 UTC m=+0.116537824 container create ba5a4dcad17016ea893ea06d267a1c9ef552f868f9d6d820c5389dd1c6395399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=) 2024-02-19T16:19:46.034 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:46 smithi191 podman[66879]: 2024-02-19 16:19:46.034043244 +0000 UTC m=+0.369805340 container init ba5a4dcad17016ea893ea06d267a1c9ef552f868f9d6d820c5389dd1c6395399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:19:46.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:45 smithi156 ceph-mon[36502]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:46.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:45 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3856905416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:46.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:45 smithi008 ceph-mon[32573]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:46.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:45 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3856905416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:46.527 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:46 smithi191 podman[66879]: 2024-02-19 16:19:46.036797805 +0000 UTC m=+0.372559901 container start ba5a4dcad17016ea893ea06d267a1c9ef552f868f9d6d820c5389dd1c6395399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l, RELEASE=wip-lusov-qdb-9f8908e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-19T16:19:46.527 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:46 smithi191 ceph-mds[66898]: -- 172.21.15.191:0/1293285665 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564e1dcba1a0 con 0x564e1dc90800 2024-02-19T16:19:46.527 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:46 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: starting mds.l at 2024-02-19T16:19:46.527 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:46 smithi191 bash[66879]: ba5a4dcad17016ea893ea06d267a1c9ef552f868f9d6d820c5389dd1c6395399 2024-02-19T16:19:46.527 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:19:46 smithi191 systemd[1]: Started Ceph mds.l for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:19:46.819 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:46.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3695210961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8101600 msgr2=0x7fdfa8101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:46.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3695210961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8101600 0x7fdfa8101a80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9802bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3695210961 shutdown_connections 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3695210961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 0x7fdfa8108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3695210961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8101600 0x7fdfa8101a80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3695210961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfa8100410 0x7fdfa8100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3695210961 >> 172.21.15.8:0/3695210961 conn(0x7fdfa80fb980 msgr2=0x7fdfa80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3695210961 shutdown_connections 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3695210961 wait complete. 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.934+0000 7fdfb03d7640 1 Processor -- start 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 -- start start 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 0x7fdfa81964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfa8101600 0x7fdfa8196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 0x7fdfa819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa819dfc0 con 0x7fdfa8100410 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa819e130 con 0x7fdfa8101600 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfb03d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa819e2f0 con 0x7fdfa8101fc0 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 0x7fdfa81964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 0x7fdfa81964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51488/0 (socket says 172.21.15.8:51488) 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 -- 172.21.15.8:0/3742209456 learned_addr learned my addr 172.21.15.8:0/3742209456 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfad972640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfa8101600 0x7fdfa8196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae974640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 0x7fdfa819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 -- 172.21.15.8:0/3742209456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 msgr2=0x7fdfa819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 0x7fdfa819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 -- 172.21.15.8:0/3742209456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfa8101600 msgr2=0x7fdfa8196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfa8101600 0x7fdfa8196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 -- 172.21.15.8:0/3742209456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98009560 con 0x7fdfa8100410 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae974640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 0x7fdfa819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdfae173640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 0x7fdfa81964d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000e9c0 tx=0x7fdfa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.935+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0009700 con 0x7fdfa8100410 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.936+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa819e5d0 con 0x7fdfa8100410 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.936+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa000cd80 con 0x7fdfa8100410 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.936+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0010430 con 0x7fdfa8100410 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.936+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa819eab0 con 0x7fdfa8100410 2024-02-19T16:19:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.937+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa8107730 con 0x7fdfa8100410 2024-02-19T16:19:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.938+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fdfa0003680 con 0x7fdfa8100410 2024-02-19T16:19:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.938+0000 7fdf9f7fe640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf940a20c0 0x7fdf940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.938+0000 7fdfad972640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf940a20c0 0x7fdf940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:46.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.938+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdfa0014070 con 0x7fdfa8100410 2024-02-19T16:19:46.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.939+0000 7fdfad972640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf940a20c0 0x7fdf940a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009230 tx=0x7fdf98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:46.943+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fdfa00970d0 con 0x7fdfa8100410 2024-02-19T16:19:47.152 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6832/2702528663,v1:172.21.15.191:6833/2702528663] up:boot 2024-02-19T16:19:47.152 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: fsmap 1 up:standby 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:47.153 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:47 smithi156 ceph-mon[36502]: Deploying daemon mds.k on smithi156 2024-02-19T16:19:47.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.167+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfa810be20 con 0x7fdfa8100410 2024-02-19T16:19:47.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.168+0000 7fdf9f7fe640 1 -- 172.21.15.8:0/3742209456 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fdfa009a790 con 0x7fdfa8100410 2024-02-19T16:19:47.169 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:47.169 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-19T16:19:47.169 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 2 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf940a20c0 msgr2=0x7fdf940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf940a20c0 0x7fdf940a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009230 tx=0x7fdf98005fd0 comp rx=0 tx=0).stop 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 msgr2=0x7fdfa81964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 0x7fdfa81964d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000e9c0 tx=0x7fdfa000ee80 comp rx=0 tx=0).stop 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 shutdown_connections 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf940a20c0 0x7fdf940a4580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:47.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa8101fc0 0x7fdfa819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:47.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfa8101600 0x7fdfa8196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:47.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 --2- 172.21.15.8:0/3742209456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfa8100410 0x7fdfa81964d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:47.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 >> 172.21.15.8:0/3742209456 conn(0x7fdfa80fb980 msgr2=0x7fdfa8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:47.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 shutdown_connections 2024-02-19T16:19:47.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:47.170+0000 7fdfb03d7640 1 -- 172.21.15.8:0/3742209456 wait complete. 2024-02-19T16:19:47.180 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6832/2702528663,v1:172.21.15.191:6833/2702528663] up:boot 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: fsmap 1 up:standby 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:47.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:47 smithi008 ceph-mon[32573]: Deploying daemon mds.k on smithi156 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6832/2702528663,v1:172.21.15.191:6833/2702528663] up:boot 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: fsmap 1 up:standby 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:47.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:47.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:47 smithi191 ceph-mon[37203]: Deploying daemon mds.k on smithi156 2024-02-19T16:19:48.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:48 smithi156 ceph-mon[36502]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:48.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:48 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3742209456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:48.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:48 smithi008 ceph-mon[32573]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:48.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:48 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3742209456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:48.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:48 smithi191 ceph-mon[37203]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:48.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:48 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3742209456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:48.736 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:48 smithi156 systemd[1]: Starting Ceph mds.k for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:19:49.013 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:48 smithi156 podman[66010]: 2024-02-19 16:19:48.638646958 +0000 UTC m=+0.017509680 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:19:49.013 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:48 smithi156 podman[66010]: 2024-02-19 16:19:48.768284395 +0000 UTC m=+0.147147112 container create 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-19T16:19:49.013 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:49 smithi156 podman[66010]: 2024-02-19 16:19:49.013080453 +0000 UTC m=+0.391943186 container init 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:19:49.183 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:49.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.298+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1115614512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 msgr2=0x7fb3d4102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:49.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1115614512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4102bb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009900 tx=0x7fb3c002bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1115614512 shutdown_connections 2024-02-19T16:19:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1115614512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3d41030f0 0x7fb3d4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1115614512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4102bb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1115614512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d4101540 0x7fb3d4101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1115614512 >> 172.21.15.8:0/1115614512 conn(0x7fb3d40fca90 msgr2=0x7fb3d40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1115614512 shutdown_connections 2024-02-19T16:19:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1115614512 wait complete. 2024-02-19T16:19:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 Processor -- start 2024-02-19T16:19:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- start start 2024-02-19T16:19:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3d4101540 0x7fb3d41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d41030f0 0x7fb3d419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:49.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d419f090 con 0x7fb3d4102730 2024-02-19T16:19:49.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d419f200 con 0x7fb3d4101540 2024-02-19T16:19:49.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.299+0000 7fb3dabe9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d419f3c0 con 0x7fb3d41030f0 2024-02-19T16:19:49.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:49.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51506/0 (socket says 172.21.15.8:51506) 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 -- 172.21.15.8:0/1065103600 learned_addr learned my addr 172.21.15.8:0/1065103600 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d8985640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3d4101540 0x7fb3d41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d9186640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d41030f0 0x7fb3d419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 -- 172.21.15.8:0/1065103600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d41030f0 msgr2=0x7fb3d419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d41030f0 0x7fb3d419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 -- 172.21.15.8:0/1065103600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3d4101540 msgr2=0x7fb3d41975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3d4101540 0x7fb3d41975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 -- 172.21.15.8:0/1065103600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0009560 con 0x7fb3d4102730 2024-02-19T16:19:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d3fff640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4197ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006010 tx=0x7fb3c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00044e0 con 0x7fb3d4102730 2024-02-19T16:19:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3c00054e0 con 0x7fb3d4102730 2024-02-19T16:19:49.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d419f640 con 0x7fb3d4102730 2024-02-19T16:19:49.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.300+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c003e450 con 0x7fb3d4102730 2024-02-19T16:19:49.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.301+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d419fb50 con 0x7fb3d4102730 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.301+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d4108860 con 0x7fb3d4102730 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.303+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb3c00035f0 con 0x7fb3d4102730 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.303+0000 7fb3d1ffb640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c40a20c0 0x7fb3c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.304+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3c00f0120 con 0x7fb3d4102730 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.304+0000 7fb3d8985640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c40a20c0 0x7fb3c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.304+0000 7fb3d8985640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c40a20c0 0x7fb3c40a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8005fd0 tx=0x7fb3c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:49.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.307+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb3c00b9590 con 0x7fb3d4102730 2024-02-19T16:19:49.329 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:49 smithi156 podman[66010]: 2024-02-19 16:19:49.016002368 +0000 UTC m=+0.394865094 container start 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-19T16:19:49.329 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:49 smithi156 ceph-mds[66029]: -- 172.21.15.156:0/3325721275 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561e6002e1a0 con 0x561e60005000 2024-02-19T16:19:49.329 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k[66025]: starting mds.k at 2024-02-19T16:19:49.329 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:49 smithi156 bash[66010]: 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c 2024-02-19T16:19:49.330 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:19:49 smithi156 systemd[1]: Started Ceph mds.k for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:19:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.529+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3d410cf50 con 0x7fb3d4102730 2024-02-19T16:19:49.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.530+0000 7fb3d1ffb640 1 -- 172.21.15.8:0/1065103600 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fb3c00b7e60 con 0x7fb3d4102730 2024-02-19T16:19:49.531 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:49.531 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-19T16:19:49.531 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 3 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c40a20c0 msgr2=0x7fb3c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c40a20c0 0x7fb3c40a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8005fd0 tx=0x7fb3c800b390 comp rx=0 tx=0).stop 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 msgr2=0x7fb3d4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4197ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006010 tx=0x7fb3c00042f0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 shutdown_connections 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c40a20c0 0x7fb3c40a4580 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d41030f0 0x7fb3d419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3d4102730 0x7fb3d4197ae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 --2- 172.21.15.8:0/1065103600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3d4101540 0x7fb3d41975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:49.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 >> 172.21.15.8:0/1065103600 conn(0x7fb3d40fca90 msgr2=0x7fb3d4105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:49.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 shutdown_connections 2024-02-19T16:19:49.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:49.532+0000 7fb3dabe9640 1 -- 172.21.15.8:0/1065103600 wait complete. 2024-02-19T16:19:50.129 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:50.129 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6832/2706765134,v1:172.21.15.156:6833/2706765134] up:boot 2024-02-19T16:19:50.129 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: fsmap 2 up:standby 2024-02-19T16:19:50.129 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-19T16:19:50.129 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: Deploying daemon mds.j on smithi008 2024-02-19T16:19:50.130 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:50 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1065103600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6832/2706765134,v1:172.21.15.156:6833/2706765134] up:boot 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: fsmap 2 up:standby 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: Deploying daemon mds.j on smithi008 2024-02-19T16:19:50.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:50 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1065103600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6832/2706765134,v1:172.21.15.156:6833/2706765134] up:boot 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: fsmap 2 up:standby 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:50.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: Deploying daemon mds.j on smithi008 2024-02-19T16:19:50.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:50 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1065103600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:51.519 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 systemd[1]: Starting Ceph mds.j for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:19:51.545 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 -- 172.21.15.8:0/4234711384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 msgr2=0x7f7b8c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 --2- 172.21.15.8:0/4234711384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c106d70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009900 tx=0x7f7b7c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 -- 172.21.15.8:0/4234711384 shutdown_connections 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 --2- 172.21.15.8:0/4234711384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c1072b0 0x7f7b8c1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 --2- 172.21.15.8:0/4234711384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c106d70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 --2- 172.21.15.8:0/4234711384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c102050 0x7f7b8c104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 -- 172.21.15.8:0/4234711384 >> 172.21.15.8:0/4234711384 conn(0x7f7b8c0fb980 msgr2=0x7f7b8c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 -- 172.21.15.8:0/4234711384 shutdown_connections 2024-02-19T16:19:51.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.652+0000 7f7b93c42640 1 -- 172.21.15.8:0/4234711384 wait complete. 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 Processor -- start 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 -- start start 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c102050 0x7f7b8c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c1072b0 0x7f7b8c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19f060 con 0x7f7b8c104980 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19f1d0 con 0x7f7b8c102050 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b93c42640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19f390 con 0x7f7b8c1072b0 2024-02-19T16:19:51.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51534/0 (socket says 172.21.15.8:51534) 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 -- 172.21.15.8:0/2955399743 learned_addr learned my addr 172.21.15.8:0/2955399743 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b921df640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c1072b0 0x7f7b8c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 -- 172.21.15.8:0/2955399743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c1072b0 msgr2=0x7f7b8c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c1072b0 0x7f7b8c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 -- 172.21.15.8:0/2955399743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c102050 msgr2=0x7f7b8c197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c102050 0x7f7b8c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 -- 172.21.15.8:0/2955399743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c009560 con 0x7f7b8c104980 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.653+0000 7f7b911dd640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c197ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c000c00 tx=0x7f7b7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:51.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.654+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c004df0 con 0x7f7b8c104980 2024-02-19T16:19:51.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.654+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c005430 con 0x7f7b8c104980 2024-02-19T16:19:51.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.654+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c19f610 con 0x7f7b8c104980 2024-02-19T16:19:51.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.654+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c03e450 con 0x7f7b8c104980 2024-02-19T16:19:51.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.654+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c19fb20 con 0x7f7b8c104980 2024-02-19T16:19:51.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.655+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8c0fd470 con 0x7f7b8c104980 2024-02-19T16:19:51.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.656+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f7b7c003680 con 0x7f7b8c104980 2024-02-19T16:19:51.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.656+0000 7f7b82ffd640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7b780a20c0 0x7f7b780a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:51.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.656+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b7c0efde0 con 0x7f7b8c104980 2024-02-19T16:19:51.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.656+0000 7f7b919de640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7b780a20c0 0x7f7b780a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:51.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.657+0000 7f7b919de640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7b780a20c0 0x7f7b780a4580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:51.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.661+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7b7c0b9250 con 0x7f7b8c104980 2024-02-19T16:19:51.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.882+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b8c02cf60 con 0x7f7b8c104980 2024-02-19T16:19:51.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.883+0000 7f7b82ffd640 1 -- 172.21.15.8:0/2955399743 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f7b7c0bc910 con 0x7f7b8c104980 2024-02-19T16:19:51.884 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:51.884 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-19T16:19:51.885 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 3 2024-02-19T16:19:51.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7b780a20c0 msgr2=0x7f7b780a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:51.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7b780a20c0 0x7f7b780a4580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88005850 comp rx=0 tx=0).stop 2024-02-19T16:19:51.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 msgr2=0x7f7b8c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:51.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c197ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c000c00 tx=0x7f7b7c004270 comp rx=0 tx=0).stop 2024-02-19T16:19:51.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 shutdown_connections 2024-02-19T16:19:51.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7b780a20c0 0x7f7b780a4580 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c1072b0 0x7f7b8c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b8c104980 0x7f7b8c197ab0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 --2- 172.21.15.8:0/2955399743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c102050 0x7f7b8c197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:51.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 >> 172.21.15.8:0/2955399743 conn(0x7f7b8c0fb980 msgr2=0x7f7b8c1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:51.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 shutdown_connections 2024-02-19T16:19:51.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:51.885+0000 7f7b93c42640 1 -- 172.21.15.8:0/2955399743 wait complete. 2024-02-19T16:19:51.896 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 podman[71179]: 2024-02-19 16:19:51.55320925 +0000 UTC m=+0.018078407 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:19:51.896 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 podman[71179]: 2024-02-19 16:19:51.671973432 +0000 UTC m=+0.136842586 container create 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:19:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:52 smithi008 ceph-mon[32573]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:52 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2955399743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:52.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:52.270 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 podman[71179]: 2024-02-19 16:19:51.916650823 +0000 UTC m=+0.381519982 container init 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:19:52.270 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 podman[71179]: 2024-02-19 16:19:51.919309443 +0000 UTC m=+0.384178600 container start 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-19T16:19:52.270 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 ceph-mds[71239]: -- 172.21.15.8:0/2170485828 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b4ff2e1a0 con 0x559b4ff05000 2024-02-19T16:19:52.270 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j[71233]: starting mds.j at 2024-02-19T16:19:52.270 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:52 smithi008 bash[71179]: 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 2024-02-19T16:19:52.270 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:19:52 smithi008 systemd[1]: Started Ceph mds.j for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:19:52.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:52 smithi191 ceph-mon[37203]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:52.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:52 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2955399743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:52.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:52.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:52.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:52 smithi156 ceph-mon[36502]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:52.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:52 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2955399743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:52.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:52.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.8:6834/3977271875,v1:172.21.15.8:6835/3977271875] up:boot 2024-02-19T16:19:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: fsmap 3 up:standby 2024-02-19T16:19:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-19T16:19:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:53.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:53 smithi008 ceph-mon[32573]: Deploying daemon mds.d on smithi008 2024-02-19T16:19:53.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.8:6834/3977271875,v1:172.21.15.8:6835/3977271875] up:boot 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: fsmap 3 up:standby 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:53.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:53 smithi191 ceph-mon[37203]: Deploying daemon mds.d on smithi008 2024-02-19T16:19:53.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.8:6834/3977271875,v1:172.21.15.8:6835/3977271875] up:boot 2024-02-19T16:19:53.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: fsmap 3 up:standby 2024-02-19T16:19:53.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-19T16:19:53.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:53.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:53.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:53.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:53.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:53 smithi156 ceph-mon[36502]: Deploying daemon mds.d on smithi008 2024-02-19T16:19:53.898 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 -- 172.21.15.8:0/924825168 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058101520 msgr2=0x7ff058101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/924825168 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058101520 0x7ff058101940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff048009900 tx=0x7ff04802bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 -- 172.21.15.8:0/924825168 shutdown_connections 2024-02-19T16:19:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/924825168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 0x7ff058109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/924825168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058102710 0x7ff058102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/924825168 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058101520 0x7ff058101940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 -- 172.21.15.8:0/924825168 >> 172.21.15.8:0/924825168 conn(0x7ff0580fca90 msgr2=0x7ff0580feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 -- 172.21.15.8:0/924825168 shutdown_connections 2024-02-19T16:19:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 -- 172.21.15.8:0/924825168 wait complete. 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 Processor -- start 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.005+0000 7ff05cd0d640 1 -- start start 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058101520 0x7ff058197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 0x7ff058197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 0x7ff05819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05819f210 con 0x7ff058102710 2024-02-19T16:19:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05819f380 con 0x7ff0581030d0 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05819f540 con 0x7ff058101520 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 0x7ff058197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 0x7ff058197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51548/0 (socket says 172.21.15.8:51548) 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 -- 172.21.15.8:0/467821901 learned_addr learned my addr 172.21.15.8:0/467821901 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05659c640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058101520 0x7ff058197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff056d9d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 0x7ff05819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 -- 172.21.15.8:0/467821901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058101520 msgr2=0x7ff058197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058101520 0x7ff058197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 -- 172.21.15.8:0/467821901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 msgr2=0x7ff05819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 0x7ff05819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 -- 172.21.15.8:0/467821901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009560 con 0x7ff058102710 2024-02-19T16:19:54.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05659c640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058101520 0x7ff058197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff056d9d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 0x7ff05819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04ffff640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 0x7ff058197c60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff04000b4a0 tx=0x7ff04000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040010610 con 0x7ff058102710 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05819f820 con 0x7ff058102710 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff040010c50 con 0x7ff058102710 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040017e20 con 0x7ff058102710 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.006+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05819fdb0 con 0x7ff058102710 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.007+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0581088b0 con 0x7ff058102710 2024-02-19T16:19:54.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.008+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff040017480 con 0x7ff058102710 2024-02-19T16:19:54.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.009+0000 7ff04f7fe640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff0440a20c0 0x7ff0440a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:54.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.009+0000 7ff05659c640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff0440a20c0 0x7ff0440a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:54.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.009+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0400ccf50 con 0x7ff058102710 2024-02-19T16:19:54.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.009+0000 7ff05659c640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff0440a20c0 0x7ff0440a4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff048006010 tx=0x7ff04802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:54.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.013+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff0400963c0 con 0x7ff058102710 2024-02-19T16:19:54.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.248+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff05802cf60 con 0x7ff058102710 2024-02-19T16:19:54.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.248+0000 7ff04f7fe640 1 -- 172.21.15.8:0/467821901 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7ff040099a80 con 0x7ff058102710 2024-02-19T16:19:54.249 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:54.249 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-19T16:19:54.249 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 4 2024-02-19T16:19:54.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.251+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff0440a20c0 msgr2=0x7ff0440a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:54.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.251+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff0440a20c0 0x7ff0440a4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff048006010 tx=0x7ff04802c800 comp rx=0 tx=0).stop 2024-02-19T16:19:54.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.251+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 msgr2=0x7ff058197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:54.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.251+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 0x7ff058197c60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff04000b4a0 tx=0x7ff04000b960 comp rx=0 tx=0).stop 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.255+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 shutdown_connections 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.255+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff0440a20c0 0x7ff0440a4580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.255+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0581030d0 0x7ff05819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.255+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff058102710 0x7ff058197c60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.255+0000 7ff05cd0d640 1 --2- 172.21.15.8:0/467821901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff058101520 0x7ff058197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.255+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 >> 172.21.15.8:0/467821901 conn(0x7ff0580fca90 msgr2=0x7ff058105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:54.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.256+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 shutdown_connections 2024-02-19T16:19:54.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:54.258+0000 7ff05cd0d640 1 -- 172.21.15.8:0/467821901 wait complete. 2024-02-19T16:19:54.393 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:54 smithi008 ceph-mon[32573]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:54.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:54 smithi191 ceph-mon[37203]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:54.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:54 smithi156 ceph-mon[36502]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/467821901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:55.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:55 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:55.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/467821901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:55.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:55.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:55.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:55 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/467821901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:55.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:55 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:56.273 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:56 smithi156 ceph-mon[36502]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:56 smithi156 ceph-mon[36502]: Deploying daemon mds.e on smithi156 2024-02-19T16:19:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:56 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.8:6836/1583103315,v1:172.21.15.8:6837/1583103315] up:boot 2024-02-19T16:19:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:56 smithi156 ceph-mon[36502]: fsmap 4 up:standby 2024-02-19T16:19:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:56 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-19T16:19:56.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 -- 172.21.15.8:0/2120414580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9378102710 msgr2=0x7f9378102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:56.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 --2- 172.21.15.8:0/2120414580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9378102710 0x7f9378102b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9360009900 tx=0x7f936002bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:56.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 -- 172.21.15.8:0/2120414580 shutdown_connections 2024-02-19T16:19:56.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 --2- 172.21.15.8:0/2120414580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781030d0 0x7f9378109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 --2- 172.21.15.8:0/2120414580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9378102710 0x7f9378102b90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 --2- 172.21.15.8:0/2120414580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9378101520 0x7f9378101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 -- 172.21.15.8:0/2120414580 >> 172.21.15.8:0/2120414580 conn(0x7f93780fca90 msgr2=0x7f93780feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:56.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 -- 172.21.15.8:0/2120414580 shutdown_connections 2024-02-19T16:19:56.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.390+0000 7f937d06b640 1 -- 172.21.15.8:0/2120414580 wait complete. 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.391+0000 7f937d06b640 1 Processor -- start 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.391+0000 7f937d06b640 1 -- start start 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9378101520 0x7f9378199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9378102710 0x7f9378199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 0x7f93781a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93781a1310 con 0x7f93781030d0 2024-02-19T16:19:56.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93781a1480 con 0x7f9378101520 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93781a1640 con 0x7f9378102710 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 0x7f93781a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 0x7f93781a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39308/0 (socket says 172.21.15.8:39308) 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 -- 172.21.15.8:0/755684121 learned_addr learned my addr 172.21.15.8:0/755684121 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f9376d9d640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9378101520 0x7f9378199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f936e59c640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9378102710 0x7f9378199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 -- 172.21.15.8:0/755684121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9378102710 msgr2=0x7f9378199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9378102710 0x7f9378199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 -- 172.21.15.8:0/755684121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9378101520 msgr2=0x7f9378199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9378101520 0x7f9378199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 -- 172.21.15.8:0/755684121 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360009560 con 0x7f93781030d0 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937759e640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 0x7f93781a0dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f937000eb10 tx=0x7f937000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937000ce20 con 0x7f93781030d0 2024-02-19T16:19:56.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9370005020 con 0x7f93781030d0 2024-02-19T16:19:56.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370010690 con 0x7f93781030d0 2024-02-19T16:19:56.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.392+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93781a1920 con 0x7f93781030d0 2024-02-19T16:19:56.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.393+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93781a1e30 con 0x7f93781030d0 2024-02-19T16:19:56.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.393+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937810cf90 con 0x7f93781030d0 2024-02-19T16:19:56.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.395+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9370005500 con 0x7f93781030d0 2024-02-19T16:19:56.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.395+0000 7f9374d99640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f93640a20c0 0x7f93640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:56.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.395+0000 7f9376d9d640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f93640a20c0 0x7f93640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:56.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.396+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9370014070 con 0x7f93781030d0 2024-02-19T16:19:56.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.396+0000 7f9376d9d640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f93640a20c0 0x7f93640a4580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9378102570 tx=0x7f9368009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:56.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.399+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f93700974d0 con 0x7f93781030d0 2024-02-19T16:19:56.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:56 smithi008 ceph-mon[32573]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:56.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:56 smithi008 ceph-mon[32573]: Deploying daemon mds.e on smithi156 2024-02-19T16:19:56.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:56 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.8:6836/1583103315,v1:172.21.15.8:6837/1583103315] up:boot 2024-02-19T16:19:56.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:56 smithi008 ceph-mon[32573]: fsmap 4 up:standby 2024-02-19T16:19:56.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:56 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-19T16:19:56.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:56 smithi191 ceph-mon[37203]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:56.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:56 smithi191 ceph-mon[37203]: Deploying daemon mds.e on smithi156 2024-02-19T16:19:56.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:56 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.8:6836/1583103315,v1:172.21.15.8:6837/1583103315] up:boot 2024-02-19T16:19:56.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:56 smithi191 ceph-mon[37203]: fsmap 4 up:standby 2024-02-19T16:19:56.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:56 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-19T16:19:56.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.624+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93781a2050 con 0x7f93781030d0 2024-02-19T16:19:56.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.624+0000 7f9374d99640 1 -- 172.21.15.8:0/755684121 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7f9370005190 con 0x7f93781030d0 2024-02-19T16:19:56.625 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:56.625 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-19T16:19:56.625 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 5 2024-02-19T16:19:56.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.626+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f93640a20c0 msgr2=0x7f93640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:56.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.626+0000 7f937d06b640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f93640a20c0 0x7f93640a4580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9378102570 tx=0x7f9368009110 comp rx=0 tx=0).stop 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.626+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 msgr2=0x7f93781a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.626+0000 7f937d06b640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 0x7f93781a0dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f937000eb10 tx=0x7f937000ef10 comp rx=0 tx=0).stop 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 shutdown_connections 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f93640a20c0 0x7f93640a4580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93781030d0 0x7f93781a0dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9378102710 0x7f9378199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 --2- 172.21.15.8:0/755684121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9378101520 0x7f9378199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 >> 172.21.15.8:0/755684121 conn(0x7f93780fca90 msgr2=0x7f9378073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 shutdown_connections 2024-02-19T16:19:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:56.627+0000 7f937d06b640 1 -- 172.21.15.8:0/755684121 wait complete. 2024-02-19T16:19:57.507 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:57 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/755684121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:57.508 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 systemd[1]: Starting Ceph mds.e for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:19:57.508 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 podman[66330]: 2024-02-19 16:19:57.410455841 +0000 UTC m=+0.017864695 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:19:57.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:57 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/755684121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:57.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:57 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/755684121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:57.777 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 podman[66330]: 2024-02-19 16:19:57.533205293 +0000 UTC m=+0.140614128 container create 7f17d82d3cf70665c95bc543f72dd71a7d88fa3fc642c6c76f47d9cd4ed7cd30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:19:57.778 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 podman[66330]: 2024-02-19 16:19:57.777847087 +0000 UTC m=+0.385255927 container init 7f17d82d3cf70665c95bc543f72dd71a7d88fa3fc642c6c76f47d9cd4ed7cd30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-19T16:19:58.079 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 podman[66330]: 2024-02-19 16:19:57.780462442 +0000 UTC m=+0.387871277 container start 7f17d82d3cf70665c95bc543f72dd71a7d88fa3fc642c6c76f47d9cd4ed7cd30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-19T16:19:58.079 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 ceph-mds[66349]: -- 172.21.15.156:0/3797250861 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556691a0c1a0 con 0x5566919e2800 2024-02-19T16:19:58.079 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e[66345]: starting mds.e at 2024-02-19T16:19:58.080 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 bash[66330]: 7f17d82d3cf70665c95bc543f72dd71a7d88fa3fc642c6c76f47d9cd4ed7cd30 2024-02-19T16:19:58.080 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:19:57 smithi156 systemd[1]: Started Ceph mds.e for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:19:58.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:58.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:58 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:58 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:19:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:19:58.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:19:58.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:19:58.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:58 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:19:58.640 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:19:58.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 -- 172.21.15.8:0/322032253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4104940 msgr2=0x7fa4e4106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:58.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 --2- 172.21.15.8:0/322032253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4104940 0x7fa4e4106d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc02bc30 comp rx=0 tx=0).stop 2024-02-19T16:19:58.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 -- 172.21.15.8:0/322032253 shutdown_connections 2024-02-19T16:19:58.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 --2- 172.21.15.8:0/322032253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e4107270 0x7fa4e41096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 --2- 172.21.15.8:0/322032253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4104940 0x7fa4e4106d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 --2- 172.21.15.8:0/322032253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 0x7fa4e4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 -- 172.21.15.8:0/322032253 >> 172.21.15.8:0/322032253 conn(0x7fa4e40fba20 msgr2=0x7fa4e40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 -- 172.21.15.8:0/322032253 shutdown_connections 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.754+0000 7fa4e92df640 1 -- 172.21.15.8:0/322032253 wait complete. 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 Processor -- start 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 -- start start 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 0x7fa4e41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e4104940 0x7fa4e4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 0x7fa4e419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e419f0b0 con 0x7fa4e4107270 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e419f220 con 0x7fa4e4102010 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e92df640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e419f3e0 con 0x7fa4e4104940 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 0x7fa4e419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:58.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 0x7fa4e419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39326/0 (socket says 172.21.15.8:39326) 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 -- 172.21.15.8:0/3665288479 learned_addr learned my addr 172.21.15.8:0/3665288479 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e27fc640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e4104940 0x7fa4e4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e2ffd640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 0x7fa4e41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 -- 172.21.15.8:0/3665288479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e4104940 msgr2=0x7fa4e4197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e4104940 0x7fa4e4197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 -- 172.21.15.8:0/3665288479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 msgr2=0x7fa4e41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 0x7fa4e41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e37fe640 1 -- 172.21.15.8:0/3665288479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc009560 con 0x7fa4e4107270 2024-02-19T16:19:58.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.755+0000 7fa4e2ffd640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 0x7fa4e41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:19:58.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.756+0000 7fa4e37fe640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 0x7fa4e419eb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400eb10 tx=0x7fa4d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:58.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.756+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4004b20 con 0x7fa4e4107270 2024-02-19T16:19:58.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.756+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e419f6c0 con 0x7fa4e4107270 2024-02-19T16:19:58.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.756+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4004c80 con 0x7fa4e4107270 2024-02-19T16:19:58.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.756+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4010650 con 0x7fa4e4107270 2024-02-19T16:19:58.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.756+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e419fba0 con 0x7fa4e4107270 2024-02-19T16:19:58.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.757+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e40fd510 con 0x7fa4e4107270 2024-02-19T16:19:58.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.759+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fa4d40107f0 con 0x7fa4e4107270 2024-02-19T16:19:58.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.759+0000 7fa4dbfff640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa4d00a20c0 0x7fa4d00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:19:58.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.759+0000 7fa4e2ffd640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa4d00a20c0 0x7fa4d00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:19:58.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.759+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa4d4014070 con 0x7fa4e4107270 2024-02-19T16:19:58.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.760+0000 7fa4e2ffd640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa4d00a20c0 0x7fa4d00a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009c20 tx=0x7fa4dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:19:58.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.763+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fa4d4097480 con 0x7fa4e4107270 2024-02-19T16:19:58.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.984+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e410cef0 con 0x7fa4e4107270 2024-02-19T16:19:58.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.984+0000 7fa4dbfff640 1 -- 172.21.15.8:0/3665288479 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fa4d409ab40 con 0x7fa4e4107270 2024-02-19T16:19:58.985 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:19:58.985 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-19T16:19:58.986 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 6 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa4d00a20c0 msgr2=0x7fa4d00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa4d00a20c0 0x7fa4d00a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009c20 tx=0x7fa4dc005f90 comp rx=0 tx=0).stop 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 msgr2=0x7fa4e419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 0x7fa4e419eb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400eb10 tx=0x7fa4d400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 shutdown_connections 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa4d00a20c0 0x7fa4d00a4580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e4107270 0x7fa4e419eb70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e4104940 0x7fa4e4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 --2- 172.21.15.8:0/3665288479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e4102010 0x7fa4e41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:19:58.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 >> 172.21.15.8:0/3665288479 conn(0x7fa4e40fba20 msgr2=0x7fa4e4102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:19:58.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 shutdown_connections 2024-02-19T16:19:58.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:19:58.986+0000 7fa4e92df640 1 -- 172.21.15.8:0/3665288479 wait complete. 2024-02-19T16:19:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:59 smithi008 ceph-mon[32573]: Deploying daemon mds.f on smithi191 2024-02-19T16:19:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:59 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6834/4229713412,v1:172.21.15.156:6835/4229713412] up:boot 2024-02-19T16:19:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:59 smithi008 ceph-mon[32573]: fsmap 5 up:standby 2024-02-19T16:19:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:59 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-19T16:19:59.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:19:59 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3665288479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:59.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:59 smithi191 ceph-mon[37203]: Deploying daemon mds.f on smithi191 2024-02-19T16:19:59.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:59 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6834/4229713412,v1:172.21.15.156:6835/4229713412] up:boot 2024-02-19T16:19:59.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:59 smithi191 ceph-mon[37203]: fsmap 5 up:standby 2024-02-19T16:19:59.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:59 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-19T16:19:59.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:19:59 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3665288479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:19:59.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:59 smithi156 ceph-mon[36502]: Deploying daemon mds.f on smithi191 2024-02-19T16:19:59.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:59 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6834/4229713412,v1:172.21.15.156:6835/4229713412] up:boot 2024-02-19T16:19:59.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:59 smithi156 ceph-mon[36502]: fsmap 5 up:standby 2024-02-19T16:19:59.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:59 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-19T16:19:59.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:19:59 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3665288479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:00.122 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 systemd[1]: Starting Ceph mds.f for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:20:00.373 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:00 smithi191 ceph-mon[37203]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:00.373 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:00 smithi191 ceph-mon[37203]: overall HEALTH_OK 2024-02-19T16:20:00.373 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 podman[67187]: 2024-02-19 16:20:00.227363962 +0000 UTC m=+0.110010207 container create 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-19T16:20:00.373 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 podman[67187]: 2024-02-19 16:20:00.136275223 +0000 UTC m=+0.018921457 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:20:00.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:00 smithi008 ceph-mon[32573]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:00.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:00 smithi008 ceph-mon[32573]: overall HEALTH_OK 2024-02-19T16:20:00.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:00 smithi156 ceph-mon[36502]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:00.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:00 smithi156 ceph-mon[36502]: overall HEALTH_OK 2024-02-19T16:20:00.776 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 podman[67187]: 2024-02-19 16:20:00.480230472 +0000 UTC m=+0.362876716 container init 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-19T16:20:00.777 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 podman[67187]: 2024-02-19 16:20:00.482797949 +0000 UTC m=+0.365444166 container start 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:20:00.777 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 ceph-mds[67206]: -- 172.21.15.191:0/2860173898 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5624b8d6e1a0 con 0x5624b8d44800 2024-02-19T16:20:00.777 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f[67202]: starting mds.f at 2024-02-19T16:20:00.777 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 bash[67187]: 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 2024-02-19T16:20:00.777 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:20:00 smithi191 systemd[1]: Started Ceph mds.f for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:00.999 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:01.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 -- 172.21.15.8:0/4097182279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101fc0 msgr2=0x7fa190108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:01.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 --2- 172.21.15.8:0/4097182279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101fc0 0x7fa190108850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa180009900 tx=0x7fa18002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 -- 172.21.15.8:0/4097182279 shutdown_connections 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 --2- 172.21.15.8:0/4097182279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101fc0 0x7fa190108850 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 --2- 172.21.15.8:0/4097182279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa190101600 0x7fa190101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 --2- 172.21.15.8:0/4097182279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190100410 0x7fa190100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 -- 172.21.15.8:0/4097182279 >> 172.21.15.8:0/4097182279 conn(0x7fa1900fb980 msgr2=0x7fa1900fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 -- 172.21.15.8:0/4097182279 shutdown_connections 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.114+0000 7fa19741c640 1 -- 172.21.15.8:0/4097182279 wait complete. 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 Processor -- start 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 -- start start 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa190100410 0x7fa1901974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 0x7fa1901979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190101fc0 0x7fa19019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19019ef90 con 0x7fa190101600 2024-02-19T16:20:01.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19019f100 con 0x7fa190101fc0 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa19741c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19019f2c0 con 0x7fa190100410 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 0x7fa1901979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 0x7fa1901979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39350/0 (socket says 172.21.15.8:39350) 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 -- 172.21.15.8:0/3707283205 learned_addr learned my addr 172.21.15.8:0/3707283205 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1951b8640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa190100410 0x7fa1901974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1959b9640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190101fc0 0x7fa19019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 -- 172.21.15.8:0/3707283205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa190100410 msgr2=0x7fa1901974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa190100410 0x7fa1901974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 -- 172.21.15.8:0/3707283205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190101fc0 msgr2=0x7fa19019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190101fc0 0x7fa19019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 -- 172.21.15.8:0/3707283205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180009560 con 0x7fa190101600 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1959b9640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190101fc0 0x7fa19019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.115+0000 7fa1949b7640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 0x7fa1901979e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa18400d950 tx=0x7fa18400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:01.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.116+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184009a30 con 0x7fa190101600 2024-02-19T16:20:01.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.116+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa184009b90 con 0x7fa190101600 2024-02-19T16:20:01.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.116+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19019f5a0 con 0x7fa190101600 2024-02-19T16:20:01.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.116+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184004a00 con 0x7fa190101600 2024-02-19T16:20:01.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.116+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19019fab0 con 0x7fa190101600 2024-02-19T16:20:01.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.117+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1901076d0 con 0x7fa190101600 2024-02-19T16:20:01.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.118+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fa184009d00 con 0x7fa190101600 2024-02-19T16:20:01.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.118+0000 7fa18e7fc640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa1780a20c0 0x7fa1780a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:01.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.118+0000 7fa1951b8640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa1780a20c0 0x7fa1780a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:01.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.118+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa1840cd860 con 0x7fa190101600 2024-02-19T16:20:01.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.119+0000 7fa1951b8640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa1780a20c0 0x7fa1780a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa190101460 tx=0x7fa188007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:01.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.122+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fa184096cd0 con 0x7fa190101600 2024-02-19T16:20:01.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.344+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa19010cf30 con 0x7fa190101600 2024-02-19T16:20:01.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.344+0000 7fa18e7fc640 1 -- 172.21.15.8:0/3707283205 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7fa18409a390 con 0x7fa190101600 2024-02-19T16:20:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:01.345 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-19T16:20:01.346 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 7 2024-02-19T16:20:01.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa1780a20c0 msgr2=0x7fa1780a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:01.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa1780a20c0 0x7fa1780a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa190101460 tx=0x7fa188007600 comp rx=0 tx=0).stop 2024-02-19T16:20:01.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 msgr2=0x7fa1901979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:01.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 0x7fa1901979e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa18400d950 tx=0x7fa18400de10 comp rx=0 tx=0).stop 2024-02-19T16:20:01.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 shutdown_connections 2024-02-19T16:20:01.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa1780a20c0 0x7fa1780a4580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa190101fc0 0x7fa19019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa190101600 0x7fa1901979e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 --2- 172.21.15.8:0/3707283205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa190100410 0x7fa1901974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:01.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 >> 172.21.15.8:0/3707283205 conn(0x7fa1900fb980 msgr2=0x7fa1901048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:01.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 shutdown_connections 2024-02-19T16:20:01.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:01.346+0000 7fa19741c640 1 -- 172.21.15.8:0/3707283205 wait complete. 2024-02-19T16:20:01.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: Deploying daemon mds.c on smithi191 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6834/562441805,v1:172.21.15.191:6835/562441805] up:boot 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: fsmap 6 up:standby 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:20:01.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:01 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3707283205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:02.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:02.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:02.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:02.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: Deploying daemon mds.c on smithi191 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6834/562441805,v1:172.21.15.191:6835/562441805] up:boot 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: fsmap 6 up:standby 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:20:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:01 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3707283205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:02.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: Deploying daemon mds.c on smithi191 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6834/562441805,v1:172.21.15.191:6835/562441805] up:boot 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: fsmap 6 up:standby 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:20:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:01 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3707283205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:03.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:02 smithi008 ceph-mon[32573]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:02 smithi191 ceph-mon[37203]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:03.027 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:02 smithi191 systemd[1]: Starting Ceph mds.c for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:20:03.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:02 smithi156 ceph-mon[36502]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:03.351 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 podman[67499]: 2024-02-19 16:20:02.997539818 +0000 UTC m=+0.017784878 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:20:03.351 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 podman[67499]: 2024-02-19 16:20:03.112527078 +0000 UTC m=+0.132772170 container create 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:20:03.351 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 podman[67499]: 2024-02-19 16:20:03.349617248 +0000 UTC m=+0.369862310 container init 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-19T16:20:03.359 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:03.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 -- 172.21.15.8:0/1556012835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101fc0 msgr2=0x7fe290108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:03.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 --2- 172.21.15.8:0/1556012835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101fc0 0x7fe290108850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe288009ca0 tx=0x7fe28802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 -- 172.21.15.8:0/1556012835 shutdown_connections 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 --2- 172.21.15.8:0/1556012835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101fc0 0x7fe290108850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 --2- 172.21.15.8:0/1556012835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe290101600 0x7fe290101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 --2- 172.21.15.8:0/1556012835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe290100410 0x7fe290100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 -- 172.21.15.8:0/1556012835 >> 172.21.15.8:0/1556012835 conn(0x7fe2900fb980 msgr2=0x7fe2900fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 -- 172.21.15.8:0/1556012835 shutdown_connections 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.476+0000 7fe295ca4640 1 -- 172.21.15.8:0/1556012835 wait complete. 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 Processor -- start 2024-02-19T16:20:03.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 -- start start 2024-02-19T16:20:03.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe290100410 0x7fe290196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 0x7fe2901969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe290101fc0 0x7fe29019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29019df60 con 0x7fe290101600 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29019e0d0 con 0x7fe290101fc0 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe295ca4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29019e290 con 0x7fe290100410 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 0x7fe2901969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 0x7fe2901969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39368/0 (socket says 172.21.15.8:39368) 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 -- 172.21.15.8:0/420267369 learned_addr learned my addr 172.21.15.8:0/420267369 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28f7fe640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe290100410 0x7fe290196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28ffff640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe290101fc0 0x7fe29019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 -- 172.21.15.8:0/420267369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe290100410 msgr2=0x7fe290196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:03.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe290100410 0x7fe290196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 -- 172.21.15.8:0/420267369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe290101fc0 msgr2=0x7fe29019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe290101fc0 0x7fe29019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.477+0000 7fe28effd640 1 -- 172.21.15.8:0/420267369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009900 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.478+0000 7fe28effd640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 0x7fe2901969b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e9f0 tx=0x7fe28400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.478+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284009720 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.478+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe28400cd80 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.478+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29019e570 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.478+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284010690 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.478+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29019ea30 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.479+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290107730 con 0x7fe290101600 2024-02-19T16:20:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.480+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe2840046c0 con 0x7fe290101600 2024-02-19T16:20:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.481+0000 7fe28cff9640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2780a20c0 0x7fe2780a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.481+0000 7fe28f7fe640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2780a20c0 0x7fe2780a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.481+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe284014070 con 0x7fe290101600 2024-02-19T16:20:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.481+0000 7fe28f7fe640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2780a20c0 0x7fe2780a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe290101460 tx=0x7fe280005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.485+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe284097090 con 0x7fe290101600 2024-02-19T16:20:03.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 podman[67499]: 2024-02-19 16:20:03.353584606 +0000 UTC m=+0.373829655 container start 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:20:03.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 ceph-mds[67521]: -- 172.21.15.191:0/3868846030 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625319f61a0 con 0x5625319ccc00 2024-02-19T16:20:03.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: starting mds.c at 2024-02-19T16:20:03.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 bash[67499]: 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f 2024-02-19T16:20:03.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:20:03 smithi191 systemd[1]: Started Ceph mds.c for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:03.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.708+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe29010be20 con 0x7fe290101600 2024-02-19T16:20:03.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.709+0000 7fe28cff9640 1 -- 172.21.15.8:0/420267369 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7fe28409a750 con 0x7fe290101600 2024-02-19T16:20:03.710 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:03.710 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-19T16:20:03.710 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 8 2024-02-19T16:20:03.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2780a20c0 msgr2=0x7fe2780a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2780a20c0 0x7fe2780a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe290101460 tx=0x7fe280005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 msgr2=0x7fe2901969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 0x7fe2901969b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e9f0 tx=0x7fe28400edf0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 shutdown_connections 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe2780a20c0 0x7fe2780a4580 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe290101fc0 0x7fe29019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe290101600 0x7fe2901969b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 --2- 172.21.15.8:0/420267369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe290100410 0x7fe290196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 >> 172.21.15.8:0/420267369 conn(0x7fe2900fb980 msgr2=0x7fe290104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 shutdown_connections 2024-02-19T16:20:03.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:03.711+0000 7fe295ca4640 1 -- 172.21.15.8:0/420267369 wait complete. 2024-02-19T16:20:04.766 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:04.766 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: Deploying daemon mds.h on smithi156 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] up:boot 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: fsmap 7 up:standby 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:20:04.767 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:04 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/420267369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: Deploying daemon mds.h on smithi156 2024-02-19T16:20:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] up:boot 2024-02-19T16:20:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: fsmap 7 up:standby 2024-02-19T16:20:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:20:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:04 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/420267369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: Deploying daemon mds.h on smithi156 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] up:boot 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: fsmap 7 up:standby 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:20:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:04 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/420267369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:05.723 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:05.829 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:05 smithi156 systemd[1]: Starting Ceph mds.h for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 -- 172.21.15.8:0/189518362 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54107270 msgr2=0x7f0c541096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 --2- 172.21.15.8:0/189518362 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54107270 0x7f0c541096c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009ca0 tx=0x7f0c4c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 -- 172.21.15.8:0/189518362 shutdown_connections 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 --2- 172.21.15.8:0/189518362 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54107270 0x7f0c541096c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 --2- 172.21.15.8:0/189518362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54104940 0x7f0c54106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 --2- 172.21.15.8:0/189518362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54102010 0x7f0c54104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 -- 172.21.15.8:0/189518362 >> 172.21.15.8:0/189518362 conn(0x7f0c540fba20 msgr2=0x7f0c540fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 -- 172.21.15.8:0/189518362 shutdown_connections 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.837+0000 7f0c58948640 1 -- 172.21.15.8:0/189518362 wait complete. 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 Processor -- start 2024-02-19T16:20:05.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 -- start start 2024-02-19T16:20:05.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54102010 0x7f0c54101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:05.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 0x7f0c540fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54107270 0x7f0c54100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54100c60 con 0x7f0c54104940 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54100dd0 con 0x7f0c54107270 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.838+0000 7f0c58948640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c541a15c0 con 0x7f0c54102010 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 0x7f0c540fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 0x7f0c540fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37004/0 (socket says 172.21.15.8:37004) 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 -- 172.21.15.8:0/387485600 learned_addr learned my addr 172.21.15.8:0/387485600 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c52d9d640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54107270 0x7f0c54100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c5259c640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54102010 0x7f0c54101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 -- 172.21.15.8:0/387485600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54102010 msgr2=0x7f0c54101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54102010 0x7f0c54101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:05.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 -- 172.21.15.8:0/387485600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54107270 msgr2=0x7f0c54100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54107270 0x7f0c54100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 -- 172.21.15.8:0/387485600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c009900 con 0x7f0c54104940 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c52d9d640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54107270 0x7f0c54100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4bfff640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 0x7f0c540fffc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00d770 tx=0x7f0c3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c009c30 con 0x7f0c54104940 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c00de80 con 0x7f0c54104940 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c541a1870 con 0x7f0c54104940 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c010460 con 0x7f0c54104940 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.839+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c541a1d30 con 0x7f0c54104940 2024-02-19T16:20:05.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.840+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c540fd510 con 0x7f0c54104940 2024-02-19T16:20:05.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.841+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0c3c004f10 con 0x7f0c54104940 2024-02-19T16:20:05.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.842+0000 7f0c4b7fe640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0c400a20c0 0x7f0c400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:05.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.842+0000 7f0c5259c640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0c400a20c0 0x7f0c400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:05.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.842+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0c3c0cd1c0 con 0x7f0c54104940 2024-02-19T16:20:05.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.843+0000 7f0c5259c640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0c400a20c0 0x7f0c400a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009bd0 tx=0x7f0c44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:05.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:05.846+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f0c3c096640 con 0x7f0c54104940 2024-02-19T16:20:06.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.067+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c5410cf30 con 0x7f0c54104940 2024-02-19T16:20:06.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.068+0000 7f0c4b7fe640 1 -- 172.21.15.8:0/387485600 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f0c3c099d00 con 0x7f0c54104940 2024-02-19T16:20:06.069 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:06.069 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-19T16:20:06.069 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 8 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0c400a20c0 msgr2=0x7f0c400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0c400a20c0 0x7f0c400a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009bd0 tx=0x7f0c44005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 msgr2=0x7f0c540fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 0x7f0c540fffc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00d770 tx=0x7f0c3c00dc30 comp rx=0 tx=0).stop 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 shutdown_connections 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0c400a20c0 0x7f0c400a4580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c54107270 0x7f0c54100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c54104940 0x7f0c540fffc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 --2- 172.21.15.8:0/387485600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54102010 0x7f0c54101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 >> 172.21.15.8:0/387485600 conn(0x7f0c540fba20 msgr2=0x7f0c54102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:06.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 shutdown_connections 2024-02-19T16:20:06.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:06.070+0000 7f0c58948640 1 -- 172.21.15.8:0/387485600 wait complete. 2024-02-19T16:20:06.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:05 smithi156 podman[66647]: 2024-02-19 16:20:05.853675701 +0000 UTC m=+0.018080666 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:20:06.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:05 smithi156 podman[66647]: 2024-02-19 16:20:05.980821404 +0000 UTC m=+0.145226352 container create 297efbd4b3c88532493afd32d56d9cf3f25ac099923aa848da92905030825ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:20:06.234 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:06 smithi156 podman[66647]: 2024-02-19 16:20:06.233640557 +0000 UTC m=+0.398045500 container init 297efbd4b3c88532493afd32d56d9cf3f25ac099923aa848da92905030825ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:06 smithi156 podman[66647]: 2024-02-19 16:20:06.236388367 +0000 UTC m=+0.400793325 container start 297efbd4b3c88532493afd32d56d9cf3f25ac099923aa848da92905030825ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mds[66666]: -- 172.21.15.156:0/3246186164 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ba4cfc1a0 con 0x564ba4cd2c00 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h[66662]: starting mds.h at 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:06 smithi156 bash[66647]: 297efbd4b3c88532493afd32d56d9cf3f25ac099923aa848da92905030825ab5 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:20:06 smithi156 systemd[1]: Started Ceph mds.h for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:06.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/387485600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:06.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:06.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:06.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:06 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:06.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/387485600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:06 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:06.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/387485600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:06 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:07.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:07 smithi008 ceph-mon[32573]: Deploying daemon mds.i on smithi191 2024-02-19T16:20:07.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:07 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6836/3993132637,v1:172.21.15.156:6837/3993132637] up:boot 2024-02-19T16:20:07.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:07 smithi008 ceph-mon[32573]: fsmap 8 up:standby 2024-02-19T16:20:07.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:07 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:20:07.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:07 smithi191 ceph-mon[37203]: Deploying daemon mds.i on smithi191 2024-02-19T16:20:07.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:07 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6836/3993132637,v1:172.21.15.156:6837/3993132637] up:boot 2024-02-19T16:20:07.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:07 smithi191 ceph-mon[37203]: fsmap 8 up:standby 2024-02-19T16:20:07.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:07 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:20:07.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:07 smithi156 ceph-mon[36502]: Deploying daemon mds.i on smithi191 2024-02-19T16:20:07.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:07 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6836/3993132637,v1:172.21.15.156:6837/3993132637] up:boot 2024-02-19T16:20:07.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:07 smithi156 ceph-mon[36502]: fsmap 8 up:standby 2024-02-19T16:20:07.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:07 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:20:08.083 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1033827192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc1030c0 msgr2=0x7f2dcc109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1033827192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc1030c0 0x7f2dcc109950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009ca0 tx=0x7f2dc402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1033827192 shutdown_connections 2024-02-19T16:20:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1033827192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc1030c0 0x7f2dcc109950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1033827192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc102700 0x7f2dcc102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1033827192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc101510 0x7f2dcc101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1033827192 >> 172.21.15.8:0/1033827192 conn(0x7f2dcc0fcaa0 msgr2=0x7f2dcc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1033827192 shutdown_connections 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.197+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1033827192 wait complete. 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 Processor -- start 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 -- start start 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc101510 0x7f2dcc197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 0x7f2dcc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc1030c0 0x7f2dcc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:08.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19f010 con 0x7f2dcc102700 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19f180 con 0x7f2dcc1030c0 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dd1ec0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19f340 con 0x7f2dcc101510 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dcaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 0x7f2dcc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.198+0000 7f2dcaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 0x7f2dcc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37028/0 (socket says 172.21.15.8:37028) 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 -- 172.21.15.8:0/1609627388 learned_addr learned my addr 172.21.15.8:0/1609627388 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcb7fe640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc101510 0x7f2dcc197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcbfff640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc1030c0 0x7f2dcc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 -- 172.21.15.8:0/1609627388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc101510 msgr2=0x7f2dcc197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc101510 0x7f2dcc197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 -- 172.21.15.8:0/1609627388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc1030c0 msgr2=0x7f2dcc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc1030c0 0x7f2dcc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 -- 172.21.15.8:0/1609627388 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4009900 con 0x7f2dcc102700 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dcaffd640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 0x7f2dcc197a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000c890 tx=0x7f2dc000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004220 con 0x7f2dcc102700 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc19f620 con 0x7f2dcc102700 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0007e90 con 0x7f2dcc102700 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc19fb80 con 0x7f2dcc102700 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.199+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004a00 con 0x7f2dcc102700 2024-02-19T16:20:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.200+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc108830 con 0x7f2dcc102700 2024-02-19T16:20:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.201+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2dc00079b0 con 0x7f2dcc102700 2024-02-19T16:20:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.202+0000 7f2dc8ff9640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2db40a20c0 0x7f2db40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.202+0000 7f2dcb7fe640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2db40a20c0 0x7f2db40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.202+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2dc00cd3c0 con 0x7f2dcc102700 2024-02-19T16:20:08.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.202+0000 7f2dcb7fe640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2db40a20c0 0x7f2db40a4580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc005fd0 tx=0x7f2dbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:08.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.206+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2dc00968e0 con 0x7f2dcc102700 2024-02-19T16:20:08.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.427+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dcc10cef0 con 0x7f2dcc102700 2024-02-19T16:20:08.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.428+0000 7f2dc8ff9640 1 -- 172.21.15.8:0/1609627388 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f2dcc10cef0 con 0x7f2dcc102700 2024-02-19T16:20:08.429 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:08.429 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-19T16:20:08.429 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 9 2024-02-19T16:20:08.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2db40a20c0 msgr2=0x7f2db40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2db40a20c0 0x7f2db40a4580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc005fd0 tx=0x7f2dbc00b410 comp rx=0 tx=0).stop 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 msgr2=0x7f2dcc197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 0x7f2dcc197a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000c890 tx=0x7f2dc000cd50 comp rx=0 tx=0).stop 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 shutdown_connections 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2db40a20c0 0x7f2db40a4580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc1030c0 0x7f2dcc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2dcc102700 0x7f2dcc197a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 --2- 172.21.15.8:0/1609627388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc101510 0x7f2dcc197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 >> 172.21.15.8:0/1609627388 conn(0x7f2dcc0fcaa0 msgr2=0x7f2dcc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 shutdown_connections 2024-02-19T16:20:08.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:08.430+0000 7f2dd1ec0640 1 -- 172.21.15.8:0/1609627388 wait complete. 2024-02-19T16:20:08.712 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:08 smithi191 ceph-mon[37203]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:08.712 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:08 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1609627388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:08.713 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 systemd[1]: Starting Ceph mds.i for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:20:08.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:08 smithi008 ceph-mon[32573]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:08.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:08 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1609627388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:08.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:08 smithi156 ceph-mon[36502]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:08.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:08 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1609627388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:08.978 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 podman[67827]: 2024-02-19 16:20:08.615218105 +0000 UTC m=+0.018002493 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:20:08.978 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 podman[67827]: 2024-02-19 16:20:08.723389228 +0000 UTC m=+0.126173616 container create b6a981c3c635febb50cb37c71a4f109507db3a6f3000c1ba4f23ed0cbb6b8763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-19T16:20:08.978 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 podman[67827]: 2024-02-19 16:20:08.952420418 +0000 UTC m=+0.355204807 container init b6a981c3c635febb50cb37c71a4f109507db3a6f3000c1ba4f23ed0cbb6b8763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-19T16:20:08.978 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 podman[67827]: 2024-02-19 16:20:08.954936988 +0000 UTC m=+0.357721382 container start b6a981c3c635febb50cb37c71a4f109507db3a6f3000c1ba4f23ed0cbb6b8763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-19T16:20:09.276 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 ceph-mds[67846]: -- 172.21.15.191:0/633703729 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c91d7bc1a0 con 0x55c91d792800 2024-02-19T16:20:09.277 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:08 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: starting mds.i at 2024-02-19T16:20:09.277 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:09 smithi191 bash[67827]: b6a981c3c635febb50cb37c71a4f109507db3a6f3000c1ba4f23ed0cbb6b8763 2024-02-19T16:20:09.277 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:09 smithi191 systemd[1]: Started Ceph mds.i for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: Deploying daemon mds.g on smithi008 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] up:boot 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: fsmap 9 up:standby 2024-02-19T16:20:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:10 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:20:10.442 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: Deploying daemon mds.g on smithi008 2024-02-19T16:20:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] up:boot 2024-02-19T16:20:10.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: fsmap 9 up:standby 2024-02-19T16:20:10.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:10 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:20:10.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 -- 172.21.15.8:0/2785282020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c102700 msgr2=0x7fb31c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:10.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 --2- 172.21.15.8:0/2785282020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c102700 0x7fb31c102b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb304009900 tx=0x7fb30402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:10.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 -- 172.21.15.8:0/2785282020 shutdown_connections 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 --2- 172.21.15.8:0/2785282020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 0x7fb31c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 --2- 172.21.15.8:0/2785282020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c102700 0x7fb31c102b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 --2- 172.21.15.8:0/2785282020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb31c101510 0x7fb31c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 -- 172.21.15.8:0/2785282020 >> 172.21.15.8:0/2785282020 conn(0x7fb31c0fcaa0 msgr2=0x7fb31c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 -- 172.21.15.8:0/2785282020 shutdown_connections 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.550+0000 7fb320cd4640 1 -- 172.21.15.8:0/2785282020 wait complete. 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 Processor -- start 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 -- start start 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 0x7fb31c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb31c102700 0x7fb31c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 0x7fb31c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c19f120 con 0x7fb31c101510 2024-02-19T16:20:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c19f290 con 0x7fb31c102700 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb320cd4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c19f450 con 0x7fb31c1030c0 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 0x7fb31c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 0x7fb31c197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37036/0 (socket says 172.21.15.8:37036) 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 -- 172.21.15.8:0/3944703543 learned_addr learned my addr 172.21.15.8:0/3944703543 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb313fff640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb31c102700 0x7fb31c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31ad9d640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 0x7fb31c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 -- 172.21.15.8:0/3944703543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 msgr2=0x7fb31c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 0x7fb31c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 -- 172.21.15.8:0/3944703543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb31c102700 msgr2=0x7fb31c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb31c102700 0x7fb31c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 -- 172.21.15.8:0/3944703543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009560 con 0x7fb31c101510 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31ad9d640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 0x7fb31c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:10.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb31a59c640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 0x7fb31c197630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00d770 tx=0x7fb30c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.551+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c009c30 con 0x7fb31c101510 2024-02-19T16:20:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.552+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb30c00de80 con 0x7fb31c101510 2024-02-19T16:20:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.552+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c19f730 con 0x7fb31c101510 2024-02-19T16:20:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.552+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c010460 con 0x7fb31c101510 2024-02-19T16:20:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.552+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c19fcc0 con 0x7fb31c101510 2024-02-19T16:20:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.552+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31c108830 con 0x7fb31c101510 2024-02-19T16:20:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.554+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb30c003f60 con 0x7fb31c101510 2024-02-19T16:20:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.554+0000 7fb3137fe640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3080a20c0 0x7fb3080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.554+0000 7fb313fff640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3080a20c0 0x7fb3080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.554+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb30c0cd620 con 0x7fb31c101510 2024-02-19T16:20:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.555+0000 7fb313fff640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3080a20c0 0x7fb3080a4580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb31c198b40 tx=0x7fb304000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:10.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.558+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb30c096a90 con 0x7fb31c101510 2024-02-19T16:20:10.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: Deploying daemon mds.g on smithi008 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] up:boot 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: fsmap 9 up:standby 2024-02-19T16:20:10.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:10 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:20:10.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.779+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31c02d000 con 0x7fb31c101510 2024-02-19T16:20:10.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.780+0000 7fb3137fe640 1 -- 172.21.15.8:0/3944703543 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7fb30c09a150 con 0x7fb31c101510 2024-02-19T16:20:10.781 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:10.781 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-19T16:20:10.781 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 10 2024-02-19T16:20:10.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3080a20c0 msgr2=0x7fb3080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3080a20c0 0x7fb3080a4580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb31c198b40 tx=0x7fb304000bc0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 msgr2=0x7fb31c197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 0x7fb31c197630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00d770 tx=0x7fb30c00dc30 comp rx=0 tx=0).stop 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 shutdown_connections 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3080a20c0 0x7fb3080a4580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31c1030c0 0x7fb31c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb31c102700 0x7fb31c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 --2- 172.21.15.8:0/3944703543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb31c101510 0x7fb31c197630 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 >> 172.21.15.8:0/3944703543 conn(0x7fb31c0fcaa0 msgr2=0x7fb31c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 shutdown_connections 2024-02-19T16:20:10.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:10.782+0000 7fb320cd4640 1 -- 172.21.15.8:0/3944703543 wait complete. 2024-02-19T16:20:11.497 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:11 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3944703543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:11.497 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 systemd[1]: Starting Ceph mds.g for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:20:11.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:11 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3944703543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:11.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:11 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3944703543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:11.767 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 podman[72160]: 2024-02-19 16:20:11.398854708 +0000 UTC m=+0.017835174 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:20:11.768 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 podman[72160]: 2024-02-19 16:20:11.515100315 +0000 UTC m=+0.134080822 container create 7169860ae5abac0f3f0fb7e073618948060282b75fd5c56cbdd3c723dac15927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T16:20:11.768 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 podman[72160]: 2024-02-19 16:20:11.767716254 +0000 UTC m=+0.386696720 container init 7169860ae5abac0f3f0fb7e073618948060282b75fd5c56cbdd3c723dac15927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-19T16:20:12.019 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 podman[72160]: 2024-02-19 16:20:11.770575406 +0000 UTC m=+0.389555873 container start 7169860ae5abac0f3f0fb7e073618948060282b75fd5c56cbdd3c723dac15927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-19T16:20:12.019 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 ceph-mds[72179]: -- 172.21.15.8:0/545961450 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646669b41a0 con 0x56466698b000 2024-02-19T16:20:12.019 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g[72175]: starting mds.g at 2024-02-19T16:20:12.019 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 bash[72160]: 7169860ae5abac0f3f0fb7e073618948060282b75fd5c56cbdd3c723dac15927 2024-02-19T16:20:12.020 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:20:11 smithi008 systemd[1]: Started Ceph mds.g for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:12.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:12.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:12.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:12.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:12 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:12.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:12 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:12 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:12.795 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:12.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 -- 172.21.15.8:0/4230201603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f80ffe20 msgr2=0x7f34f8100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:12.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 --2- 172.21.15.8:0/4230201603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f80ffe20 0x7f34f8100240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009900 tx=0x7f34f402bcd0 comp rx=0 tx=0).stop 2024-02-19T16:20:12.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 -- 172.21.15.8:0/4230201603 shutdown_connections 2024-02-19T16:20:12.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 --2- 172.21.15.8:0/4230201603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f8101120 0x7f34f81015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:12.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 --2- 172.21.15.8:0/4230201603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34f8100780 0x7f34f8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:12.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 --2- 172.21.15.8:0/4230201603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f80ffe20 0x7f34f8100240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 -- 172.21.15.8:0/4230201603 >> 172.21.15.8:0/4230201603 conn(0x7f34f80fb9a0 msgr2=0x7f34f80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 -- 172.21.15.8:0/4230201603 shutdown_connections 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.903+0000 7f3500160640 1 -- 172.21.15.8:0/4230201603 wait complete. 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 Processor -- start 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 -- start start 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34f80ffe20 0x7f34f81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f8100780 0x7f34f8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 0x7f34f81a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81a12e0 con 0x7f34f8101120 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81a1450 con 0x7f34f8100780 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f3500160640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81a1610 con 0x7f34f80ffe20 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 0x7f34f81a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 0x7f34f81a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37060/0 (socket says 172.21.15.8:37060) 2024-02-19T16:20:12.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 -- 172.21.15.8:0/1923122682 learned_addr learned my addr 172.21.15.8:0/1923122682 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fd6fb640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f8100780 0x7f34f8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fdefc640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34f80ffe20 0x7f34f81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 -- 172.21.15.8:0/1923122682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34f80ffe20 msgr2=0x7f34f81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34f80ffe20 0x7f34f81997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 -- 172.21.15.8:0/1923122682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f8100780 msgr2=0x7f34f8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f8100780 0x7f34f8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.904+0000 7f34fe6fd640 1 -- 172.21.15.8:0/1923122682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009560 con 0x7f34f8101120 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.905+0000 7f34fe6fd640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 0x7f34f81a0da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f34f000eaa0 tx=0x7f34f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.905+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000cdb0 con 0x7f34f8101120 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.905+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f81a18f0 con 0x7f34f8101120 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.905+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34f0005020 con 0x7f34f8101120 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.905+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00116b0 con 0x7f34f8101120 2024-02-19T16:20:12.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.905+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f81a1db0 con 0x7f34f8101120 2024-02-19T16:20:12.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.906+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f810acf0 con 0x7f34f8101120 2024-02-19T16:20:12.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.907+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f34f0005520 con 0x7f34f8101120 2024-02-19T16:20:12.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.907+0000 7f34eeffd640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e40a20c0 0x7f34e40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:12.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.908+0000 7f34fdefc640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e40a20c0 0x7f34e40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:12.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.908+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34f00cdaf0 con 0x7f34f8101120 2024-02-19T16:20:12.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.908+0000 7f34fdefc640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e40a20c0 0x7f34e40a4580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f34f402c740 tx=0x7f34f4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:12.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:12.912+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f34f0096ec0 con 0x7f34f8101120 2024-02-19T16:20:13.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.138+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34f810f120 con 0x7f34f8101120 2024-02-19T16:20:13.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.139+0000 7f34eeffd640 1 -- 172.21.15.8:0/1923122682 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7f34f009a580 con 0x7f34f8101120 2024-02-19T16:20:13.140 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:13.140 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-19T16:20:13.140 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 11 2024-02-19T16:20:13.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e40a20c0 msgr2=0x7f34e40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:13.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e40a20c0 0x7f34e40a4580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f34f402c740 tx=0x7f34f4000bc0 comp rx=0 tx=0).stop 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 msgr2=0x7f34f81a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 0x7f34f81a0da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f34f000eaa0 tx=0x7f34f000ef60 comp rx=0 tx=0).stop 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 shutdown_connections 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f34e40a20c0 0x7f34e40a4580 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34f8101120 0x7f34f81a0da0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f8100780 0x7f34f8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 --2- 172.21.15.8:0/1923122682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34f80ffe20 0x7f34f81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 >> 172.21.15.8:0/1923122682 conn(0x7f34f80fb9a0 msgr2=0x7f34f80fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 shutdown_connections 2024-02-19T16:20:13.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:13.141+0000 7f3500160640 1 -- 172.21.15.8:0/1923122682 wait complete. 2024-02-19T16:20:13.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:13 smithi008 ceph-mon[32573]: Deploying daemon mds.a on smithi008 2024-02-19T16:20:13.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:13 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.8:6838/1717311742,v1:172.21.15.8:6839/1717311742] up:boot 2024-02-19T16:20:13.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:13 smithi008 ceph-mon[32573]: fsmap 10 up:standby 2024-02-19T16:20:13.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:13 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-19T16:20:13.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:13 smithi191 ceph-mon[37203]: Deploying daemon mds.a on smithi008 2024-02-19T16:20:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:13 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.8:6838/1717311742,v1:172.21.15.8:6839/1717311742] up:boot 2024-02-19T16:20:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:13 smithi191 ceph-mon[37203]: fsmap 10 up:standby 2024-02-19T16:20:13.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:13 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-19T16:20:13.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:13 smithi156 ceph-mon[36502]: Deploying daemon mds.a on smithi008 2024-02-19T16:20:13.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:13 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.8:6838/1717311742,v1:172.21.15.8:6839/1717311742] up:boot 2024-02-19T16:20:13.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:13 smithi156 ceph-mon[36502]: fsmap 10 up:standby 2024-02-19T16:20:13.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:13 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-19T16:20:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:14 smithi008 ceph-mon[32573]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:14 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1923122682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:14 smithi191 ceph-mon[37203]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:14 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1923122682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:14 smithi156 ceph-mon[36502]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:14 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1923122682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:15.019 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:20:14 smithi008 systemd[1]: Started Ceph mds.a for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:15.153 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:15.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3569539681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 msgr2=0x7ff7c0109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:15.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3569539681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c0109640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009900 tx=0x7ff7b802bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:15.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3569539681 shutdown_connections 2024-02-19T16:20:15.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3569539681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c0109640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3569539681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 0x7ff7c0106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3569539681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 0x7ff7c0104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3569539681 >> 172.21.15.8:0/3569539681 conn(0x7ff7c00fb980 msgr2=0x7ff7c00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:15.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3569539681 shutdown_connections 2024-02-19T16:20:15.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.261+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3569539681 wait complete. 2024-02-19T16:20:15.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 Processor -- start 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 -- start start 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 0x7ff7c0197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 0x7ff7c0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c019f040 con 0x7ff7c01071f0 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c019f1b0 con 0x7ff7c01048c0 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7c4d8a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c019f370 con 0x7ff7c0101f90 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7bed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7bed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37082/0 (socket says 172.21.15.8:37082) 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.262+0000 7ff7bed9d640 1 -- 172.21.15.8:0/3088731287 learned_addr learned my addr 172.21.15.8:0/3088731287 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bdd9b640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 0x7ff7c0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7be59c640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 0x7ff7c0197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:15.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bed9d640 1 -- 172.21.15.8:0/3088731287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 msgr2=0x7ff7c0197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bed9d640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 0x7ff7c0197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bed9d640 1 -- 172.21.15.8:0/3088731287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 msgr2=0x7ff7c0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bed9d640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 0x7ff7c0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bed9d640 1 -- 172.21.15.8:0/3088731287 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a8009490 con 0x7ff7c01071f0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bdd9b640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 0x7ff7c0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7be59c640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 0x7ff7c0197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7bed9d640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c019eb00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009230 tx=0x7ff7b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80044e0 con 0x7ff7c01071f0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8004aa0 con 0x7ff7c01071f0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b8009560 con 0x7ff7c01071f0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8003d40 con 0x7ff7c01071f0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.263+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c019f970 con 0x7ff7c01071f0 2024-02-19T16:20:15.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.264+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c00fd470 con 0x7ff7c01071f0 2024-02-19T16:20:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.266+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff7b8005000 con 0x7ff7c01071f0 2024-02-19T16:20:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.266+0000 7ff7b77fe640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff7ac0a20c0 0x7ff7ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.266+0000 7ff7be59c640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff7ac0a20c0 0x7ff7ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:15.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.266+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7b80f0350 con 0x7ff7c01071f0 2024-02-19T16:20:15.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.267+0000 7ff7be59c640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff7ac0a20c0 0x7ff7ac0a4580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0005fd0 tx=0x7ff7b0009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:15.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.270+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff7b80b97c0 con 0x7ff7c01071f0 2024-02-19T16:20:15.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.489+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7c010cf20 con 0x7ff7c01071f0 2024-02-19T16:20:15.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.490+0000 7ff7b77fe640 1 -- 172.21.15.8:0/3088731287 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9245 (secure 0 0 0) 0x7ff7b80bce80 con 0x7ff7c01071f0 2024-02-19T16:20:15.491 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:15.491 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-19T16:20:15.491 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2024-02-19T16:20:15.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff7ac0a20c0 msgr2=0x7ff7ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff7ac0a20c0 0x7ff7ac0a4580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0005fd0 tx=0x7ff7b0009510 comp rx=0 tx=0).stop 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 msgr2=0x7ff7c019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c019eb00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009230 tx=0x7ff7b80042f0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 shutdown_connections 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff7ac0a20c0 0x7ff7ac0a4580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7c01071f0 0x7ff7c019eb00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7c01048c0 0x7ff7c0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 --2- 172.21.15.8:0/3088731287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c0101f90 0x7ff7c0197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 >> 172.21.15.8:0/3088731287 conn(0x7ff7c00fb980 msgr2=0x7ff7c01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 shutdown_connections 2024-02-19T16:20:15.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:15.492+0000 7ff7c4d8a640 1 -- 172.21.15.8:0/3088731287 wait complete. 2024-02-19T16:20:15.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:15.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: Deploying daemon mds.b on smithi156 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.8:6840/1752496516,v1:172.21.15.8:6841/1752496516] up:boot 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: fsmap 11 up:standby 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-19T16:20:15.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:15 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3088731287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: Deploying daemon mds.b on smithi156 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.8:6840/1752496516,v1:172.21.15.8:6841/1752496516] up:boot 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: fsmap 11 up:standby 2024-02-19T16:20:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-19T16:20:16.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:15 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3088731287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:16.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: Deploying daemon mds.b on smithi156 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.8:6840/1752496516,v1:172.21.15.8:6841/1752496516] up:boot 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: fsmap 11 up:standby 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-19T16:20:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:15 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3088731287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:17.080 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 systemd[1]: Starting Ceph mds.b for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:20:17.499 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 podman[66958]: 2024-02-19 16:20:17.128540704 +0000 UTC m=+0.017756118 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:20:17.499 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 podman[66958]: 2024-02-19 16:20:17.245556652 +0000 UTC m=+0.134772074 container create 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-19T16:20:17.499 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 podman[66958]: 2024-02-19 16:20:17.473284353 +0000 UTC m=+0.362499772 container init 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:20:17.499 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 podman[66958]: 2024-02-19 16:20:17.476098813 +0000 UTC m=+0.365314207 container start 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-19T16:20:17.505 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 -- 172.21.15.8:0/1668886120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7050101520 msgr2=0x7f7050101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 --2- 172.21.15.8:0/1668886120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7050101520 0x7f7050101940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7040009ca0 tx=0x7f704002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 -- 172.21.15.8:0/1668886120 shutdown_connections 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 --2- 172.21.15.8:0/1668886120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70501030d0 0x7f7050109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 --2- 172.21.15.8:0/1668886120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050102710 0x7f7050102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 --2- 172.21.15.8:0/1668886120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7050101520 0x7f7050101940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 -- 172.21.15.8:0/1668886120 >> 172.21.15.8:0/1668886120 conn(0x7f70500fca90 msgr2=0x7f70500feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 -- 172.21.15.8:0/1668886120 shutdown_connections 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 -- 172.21.15.8:0/1668886120 wait complete. 2024-02-19T16:20:17.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.620+0000 7f7055647640 1 Processor -- start 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 -- start start 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050101520 0x7f70501975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7050102710 0x7f7050197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 0x7f705019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705019f0b0 con 0x7f70501030d0 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705019f220 con 0x7f7050102710 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7055647640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705019f3e0 con 0x7f7050101520 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 0x7f705019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 0x7f705019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37718/0 (socket says 172.21.15.8:37718) 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 -- 172.21.15.8:0/2382874111 learned_addr learned my addr 172.21.15.8:0/2382874111 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704e7fc640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7050102710 0x7f7050197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704effd640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050101520 0x7f70501975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 -- 172.21.15.8:0/2382874111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050101520 msgr2=0x7f70501975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050101520 0x7f70501975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 -- 172.21.15.8:0/2382874111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7050102710 msgr2=0x7f7050197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7050102710 0x7f7050197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 -- 172.21.15.8:0/2382874111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009900 con 0x7f70501030d0 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f704f7fe640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 0x7f705019eb70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f704800e970 tx=0x7f704800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.621+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048009720 con 0x7f70501030d0 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.622+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705019f6c0 con 0x7f70501030d0 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.622+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7048005500 con 0x7f70501030d0 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.622+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048010430 con 0x7f70501030d0 2024-02-19T16:20:17.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.622+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705019fb80 con 0x7f70501030d0 2024-02-19T16:20:17.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.623+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7050108840 con 0x7f70501030d0 2024-02-19T16:20:17.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.624+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f7048005000 con 0x7f70501030d0 2024-02-19T16:20:17.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.624+0000 7f7037fff640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70380a20c0 0x7f70380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.624+0000 7f704effd640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70380a20c0 0x7f70380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.625+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7048014070 con 0x7f70501030d0 2024-02-19T16:20:17.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.625+0000 7f704effd640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70380a20c0 0x7f70380a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f704002c700 tx=0x7f70400326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:17.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.628+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7048097190 con 0x7f70501030d0 2024-02-19T16:20:17.829 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 ceph-mds[66977]: -- 172.21.15.156:0/3193148211 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56093c3401a0 con 0x56093c317000 2024-02-19T16:20:17.829 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b[66973]: starting mds.b at 2024-02-19T16:20:17.830 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 bash[66958]: 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 2024-02-19T16:20:17.830 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:20:17 smithi156 systemd[1]: Started Ceph mds.b for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:20:17.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.874+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f705010cf30 con 0x7f70501030d0 2024-02-19T16:20:17.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.875+0000 7f7037fff640 1 -- 172.21.15.8:0/2382874111 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10040 (secure 0 0 0) 0x7f704809a850 con 0x7f70501030d0 2024-02-19T16:20:17.876 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:17.876 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-19T16:20:17.876 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 13 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70380a20c0 msgr2=0x7f70380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70380a20c0 0x7f70380a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f704002c700 tx=0x7f70400326b0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 msgr2=0x7f705019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 0x7f705019eb70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f704800e970 tx=0x7f704800ed70 comp rx=0 tx=0).stop 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 shutdown_connections 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70380a20c0 0x7f70380a4580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f70501030d0 0x7f705019eb70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7050102710 0x7f7050197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 --2- 172.21.15.8:0/2382874111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050101520 0x7f70501975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 >> 172.21.15.8:0/2382874111 conn(0x7f70500fca90 msgr2=0x7f7050105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:17.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 shutdown_connections 2024-02-19T16:20:17.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.877+0000 7f7055647640 1 -- 172.21.15.8:0/2382874111 wait complete. 2024-02-19T16:20:17.890 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:17.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 -- 172.21.15.8:0/625038497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c101fc0 msgr2=0x7fb00c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 --2- 172.21.15.8:0/625038497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c101fc0 0x7fb00c108850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ca0 tx=0x7fb00402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 -- 172.21.15.8:0/625038497 shutdown_connections 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 --2- 172.21.15.8:0/625038497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c101fc0 0x7fb00c108850 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 --2- 172.21.15.8:0/625038497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb00c101600 0x7fb00c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 --2- 172.21.15.8:0/625038497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c100410 0x7fb00c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 -- 172.21.15.8:0/625038497 >> 172.21.15.8:0/625038497 conn(0x7fb00c0fb980 msgr2=0x7fb00c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 -- 172.21.15.8:0/625038497 shutdown_connections 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 -- 172.21.15.8:0/625038497 wait complete. 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.993+0000 7fb012147640 1 Processor -- start 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 -- start start 2024-02-19T16:20:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 0x7fb00c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c101600 0x7fb00c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb00c101fc0 0x7fb00c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a1270 con 0x7fb00c100410 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a13e0 con 0x7fb00c101600 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a15a0 con 0x7fb00c101fc0 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 0x7fb00c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 0x7fb00c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37728/0 (socket says 172.21.15.8:37728) 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 -- 172.21.15.8:0/711911886 learned_addr learned my addr 172.21.15.8:0/711911886 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00affd640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c101600 0x7fb00c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00bfff640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb00c101fc0 0x7fb00c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 -- 172.21.15.8:0/711911886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb00c101fc0 msgr2=0x7fb00c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb00c101fc0 0x7fb00c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 -- 172.21.15.8:0/711911886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c101600 msgr2=0x7fb00c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c101600 0x7fb00c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 -- 172.21.15.8:0/711911886 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb004009900 con 0x7fb00c100410 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00affd640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c101600 0x7fb00c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb00b7fe640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 0x7fb00c199780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9c0 tx=0x7faffc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc009700 con 0x7fb00c100410 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faffc00cd80 con 0x7fb00c100410 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c1a1880 con 0x7fb00c100410 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.994+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc010430 con 0x7fb00c100410 2024-02-19T16:20:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.995+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1a1d90 con 0x7fb00c100410 2024-02-19T16:20:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.996+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c10be20 con 0x7fb00c100410 2024-02-19T16:20:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.997+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7faffc003680 con 0x7fb00c100410 2024-02-19T16:20:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.997+0000 7fb008ff9640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faff40a20c0 0x7faff40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.997+0000 7fb00affd640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faff40a20c0 0x7faff40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.997+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faffc014070 con 0x7fb00c100410 2024-02-19T16:20:18.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:17.998+0000 7fb00affd640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faff40a20c0 0x7faff40a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb00c19ac90 tx=0x7fb00000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.001+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7faffc0971b0 con 0x7fb00c100410 2024-02-19T16:20:18.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.185+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb00c1a22e0 con 0x7fb00c100410 2024-02-19T16:20:18.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.186+0000 7fb008ff9640 1 -- 172.21.15.8:0/711911886 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18373 (secure 0 0 0) 0x7faffc09a870 con 0x7fb00c100410 2024-02-19T16:20:18.187 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:18.187 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":82,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:19:34.880156+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faff40a20c0 msgr2=0x7faff40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faff40a20c0 0x7faff40a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb00c19ac90 tx=0x7fb00000b410 comp rx=0 tx=0).stop 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 msgr2=0x7fb00c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 0x7fb00c199780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9c0 tx=0x7faffc00ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 shutdown_connections 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faff40a20c0 0x7faff40a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb00c101fc0 0x7fb00c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb00c101600 0x7fb00c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 --2- 172.21.15.8:0/711911886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb00c100410 0x7fb00c199780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 >> 172.21.15.8:0/711911886 conn(0x7fb00c0fb980 msgr2=0x7fb00c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:18.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 shutdown_connections 2024-02-19T16:20:18.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.188+0000 7fb012147640 1 -- 172.21.15.8:0/711911886 wait complete. 2024-02-19T16:20:18.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T16:20:18.201 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1556874598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78104920 msgr2=0x7f7c78106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1556874598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78104920 0x7f7c78106d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1556874598 shutdown_connections 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1556874598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c78107250 0x7f7c78109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1556874598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78104920 0x7f7c78106d10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1556874598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1556874598 >> 172.21.15.8:0/1556874598 conn(0x7f7c780fb9a0 msgr2=0x7f7c780fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1556874598 shutdown_connections 2024-02-19T16:20:18.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1556874598 wait complete. 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 Processor -- start 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.307+0000 7f7c7f11e640 1 -- start start 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7f11e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7f11e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c78104920 0x7f7c78197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7f11e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78107250 0x7f7c7819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7f11e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7819f150 con 0x7f7c78107250 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7f11e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7819f2c0 con 0x7f7c78104920 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7f11e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7819f480 con 0x7f7c78101ff0 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7d6bb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78107250 0x7f7c7819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c77fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c78104920 0x7f7c78197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7d6bb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78107250 0x7f7c7819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37740/0 (socket says 172.21.15.8:37740) 2024-02-19T16:20:18.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:59158/0 (socket says 172.21.15.8:59158) 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7d6bb640 1 -- 172.21.15.8:0/1418581152 learned_addr learned my addr 172.21.15.8:0/1418581152 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 -- 172.21.15.8:0/1418581152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c78104920 msgr2=0x7f7c78197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c78104920 0x7f7c78197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 -- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78107250 msgr2=0x7f7c7819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78107250 0x7f7c7819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 -- 172.21.15.8:0/1418581152 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c64009560 con 0x7f7c78101ff0 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.308+0000 7f7c7ceba640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781976b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00e9f0 tx=0x7f7c6c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.309+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00ccc0 con 0x7f7c78101ff0 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.309+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7819f760 con 0x7f7c78101ff0 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.309+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c6c00ce20 con 0x7f7c78101ff0 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.309+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c004a00 con 0x7f7c78101ff0 2024-02-19T16:20:18.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.309+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7819fc40 con 0x7f7c78101ff0 2024-02-19T16:20:18.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.310+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c780fd490 con 0x7f7c78101ff0 2024-02-19T16:20:18.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.311+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f7c6c0050b0 con 0x7f7c78101ff0 2024-02-19T16:20:18.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.311+0000 7f7c75ffb640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c680a2140 0x7f7c680a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.311+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c6c014070 con 0x7f7c78101ff0 2024-02-19T16:20:18.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.311+0000 7f7c77fff640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c680a2140 0x7f7c680a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.312+0000 7f7c77fff640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c680a2140 0x7f7c680a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005ab0 tx=0x7f7c64005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.316+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7c6c096bb0 con 0x7f7c78101ff0 2024-02-19T16:20:18.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.507+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c7802d000 con 0x7f7c78101ff0 2024-02-19T16:20:18.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.507+0000 7f7c75ffb640 1 -- 172.21.15.8:0/1418581152 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c6c09a270 con 0x7f7c78101ff0 2024-02-19T16:20:18.508 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c680a2140 msgr2=0x7f7c680a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c680a2140 0x7f7c680a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005ab0 tx=0x7f7c64005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 msgr2=0x7f7c781976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781976b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00e9f0 tx=0x7f7c6c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 shutdown_connections 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c680a2140 0x7f7c680a4600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c78107250 0x7f7c7819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c78104920 0x7f7c78197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 --2- 172.21.15.8:0/1418581152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c78101ff0 0x7f7c781976b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 >> 172.21.15.8:0/1418581152 conn(0x7f7c780fb9a0 msgr2=0x7f7c78102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 shutdown_connections 2024-02-19T16:20:18.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.510+0000 7f7c7f11e640 1 -- 172.21.15.8:0/1418581152 wait complete. 2024-02-19T16:20:18.522 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 -- 172.21.15.8:0/1297376543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 msgr2=0x7f5340100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 --2- 172.21.15.8:0/1297376543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340100d70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5330009900 tx=0x7f533002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 -- 172.21.15.8:0/1297376543 shutdown_connections 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 --2- 172.21.15.8:0/1297376543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 0x7f53401088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 --2- 172.21.15.8:0/1297376543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5340101660 0x7f5340101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 --2- 172.21.15.8:0/1297376543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340100d70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 -- 172.21.15.8:0/1297376543 >> 172.21.15.8:0/1297376543 conn(0x7f53400fb120 msgr2=0x7f53400fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:18.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 -- 172.21.15.8:0/1297376543 shutdown_connections 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 -- 172.21.15.8:0/1297376543 wait complete. 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 Processor -- start 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.625+0000 7f534509a640 1 -- start start 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5340101660 0x7f5340196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 0x7f534019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534019dff0 con 0x7f5340100950 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534019e160 con 0x7f5340101660 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534019e320 con 0x7f5340102020 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37762/0 (socket says 172.21.15.8:37762) 2024-02-19T16:20:18.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 -- 172.21.15.8:0/1105905579 learned_addr learned my addr 172.21.15.8:0/1105905579 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f5337fff640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5340101660 0x7f5340196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533f59e640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 0x7f534019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 -- 172.21.15.8:0/1105905579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 msgr2=0x7f534019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 0x7f534019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 -- 172.21.15.8:0/1105905579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5340101660 msgr2=0x7f5340196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5340101660 0x7f5340196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 -- 172.21.15.8:0/1105905579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330009560 con 0x7f5340100950 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533f59e640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 0x7f534019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533ed9d640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340196500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5330000c00 tx=0x7f5330004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330004df0 con 0x7f5340100950 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5330005430 con 0x7f5340100950 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534019e5a0 con 0x7f5340100950 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533003e450 con 0x7f5340100950 2024-02-19T16:20:18.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.626+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534019eab0 con 0x7f5340100950 2024-02-19T16:20:18.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.627+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5340107790 con 0x7f5340100950 2024-02-19T16:20:18.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.628+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5330003680 con 0x7f5340100950 2024-02-19T16:20:18.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.629+0000 7f533cd99640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f532c0a20c0 0x7f532c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.629+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f53300efd20 con 0x7f5340100950 2024-02-19T16:20:18.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.629+0000 7f5337fff640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f532c0a20c0 0x7f532c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.629+0000 7f5337fff640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f532c0a20c0 0x7f532c0a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5340197a10 tx=0x7f532800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.633+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f53300b9190 con 0x7f5340100950 2024-02-19T16:20:18.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.819+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f534002cf90 con 0x7f5340100950 2024-02-19T16:20:18.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.819+0000 7f533cd99640 1 -- 172.21.15.8:0/1105905579 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f53300bc850 con 0x7f5340100950 2024-02-19T16:20:18.820 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:18.820 INFO:teuthology.orchestra.run.smithi008.stdout:[ 2024-02-19T16:20:18.820 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:20:18.821 INFO:teuthology.orchestra.run.smithi008.stdout: "poolnum": 1, 2024-02-19T16:20:18.821 INFO:teuthology.orchestra.run.smithi008.stdout: "poolname": ".mgr" 2024-02-19T16:20:18.821 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:20:18.821 INFO:teuthology.orchestra.run.smithi008.stdout:] 2024-02-19T16:20:18.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f532c0a20c0 msgr2=0x7f532c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f532c0a20c0 0x7f532c0a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5340197a10 tx=0x7f532800b390 comp rx=0 tx=0).stop 2024-02-19T16:20:18.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 msgr2=0x7f5340196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340196500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5330000c00 tx=0x7f5330004270 comp rx=0 tx=0).stop 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 shutdown_connections 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f532c0a20c0 0x7f532c0a4580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5340102020 0x7f534019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5340101660 0x7f5340196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 --2- 172.21.15.8:0/1105905579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5340100950 0x7f5340196500 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.821+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 >> 172.21.15.8:0/1105905579 conn(0x7f53400fb120 msgr2=0x7f5340104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.822+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 shutdown_connections 2024-02-19T16:20:18.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.822+0000 7f534509a640 1 -- 172.21.15.8:0/1105905579 wait complete. 2024-02-19T16:20:18.832 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] up:boot 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: fsmap 12 up:standby 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2382874111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/711911886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:18.833 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:18 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1418581152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:18.834 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-19T16:20:18.834 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-19T16:20:18.894 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:18.894 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.894 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.894 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.894 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] up:boot 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: fsmap 12 up:standby 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2382874111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/711911886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:18.895 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:18 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1418581152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 -- 172.21.15.8:0/3428014666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 msgr2=0x7fc90c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 --2- 172.21.15.8:0/3428014666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c108840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc904009ca0 tx=0x7fc90402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 -- 172.21.15.8:0/3428014666 shutdown_connections 2024-02-19T16:20:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 --2- 172.21.15.8:0/3428014666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c108840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 --2- 172.21.15.8:0/3428014666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc90c1015f0 0x7fc90c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 --2- 172.21.15.8:0/3428014666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c100400 0x7fc90c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 -- 172.21.15.8:0/3428014666 >> 172.21.15.8:0/3428014666 conn(0x7fc90c0fb990 msgr2=0x7fc90c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 -- 172.21.15.8:0/3428014666 shutdown_connections 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.941+0000 7fc912af4640 1 -- 172.21.15.8:0/3428014666 wait complete. 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 Processor -- start 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 -- start start 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc90c100400 0x7fc90c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c1015f0 0x7fc90c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c19df60 con 0x7fc90c101fb0 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c19e0d0 con 0x7fc90c1015f0 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc912af4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c19e290 con 0x7fc90c100400 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c19da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37788/0 (socket says 172.21.15.8:37788) 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 -- 172.21.15.8:0/1987309828 learned_addr learned my addr 172.21.15.8:0/1987309828 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc910890640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc90c100400 0x7fc90c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc90bfff640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c1015f0 0x7fc90c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 -- 172.21.15.8:0/1987309828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc90c100400 msgr2=0x7fc90c196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc90c100400 0x7fc90c196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 -- 172.21.15.8:0/1987309828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c1015f0 msgr2=0x7fc90c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c1015f0 0x7fc90c1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 -- 172.21.15.8:0/1987309828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc904009900 con 0x7fc90c101fb0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.942+0000 7fc911091640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c19da20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc904009bf0 tx=0x7fc904034ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.943+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904003970 con 0x7fc90c101fb0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.943+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc904004040 con 0x7fc90c101fb0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.943+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040406b0 con 0x7fc90c101fb0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.943+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c19e510 con 0x7fc90c101fb0 2024-02-19T16:20:18.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.943+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c19ea10 con 0x7fc90c101fb0 2024-02-19T16:20:18.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.944+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c107770 con 0x7fc90c101fb0 2024-02-19T16:20:18.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.945+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fc90400a040 con 0x7fc90c101fb0 2024-02-19T16:20:18.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.946+0000 7fc909ffb640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8fc0a2140 0x7fc8fc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:18.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.946+0000 7fc910890640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8fc0a2140 0x7fc8fc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:18.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.946+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc9040f07b0 con 0x7fc90c101fb0 2024-02-19T16:20:18.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.946+0000 7fc910890640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8fc0a2140 0x7fc8fc0a4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc90c101450 tx=0x7fc90000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:18.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:18.950+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fc9040b9c20 con 0x7fc90c101fb0 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] up:boot 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: fsmap 12 up:standby 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2382874111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/711911886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:19.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:18 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1418581152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.134+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc90c02d000 con 0x7fc90c101fb0 2024-02-19T16:20:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.616+0000 7fc909ffb640 1 -- 172.21.15.8:0/1987309828 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc9040bd2e0 con 0x7fc90c101fb0 2024-02-19T16:20:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_metadata' created 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8fc0a2140 msgr2=0x7fc8fc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8fc0a2140 0x7fc8fc0a4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc90c101450 tx=0x7fc90000a2d0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 msgr2=0x7fc90c19da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c19da20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc904009bf0 tx=0x7fc904034ca0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 shutdown_connections 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc8fc0a2140 0x7fc8fc0a4600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc90c101fb0 0x7fc90c19da20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc90c1015f0 0x7fc90c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 --2- 172.21.15.8:0/1987309828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc90c100400 0x7fc90c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 >> 172.21.15.8:0/1987309828 conn(0x7fc90c0fb990 msgr2=0x7fc90c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 shutdown_connections 2024-02-19T16:20:19.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.619+0000 7fc912af4640 1 -- 172.21.15.8:0/1987309828 wait complete. 2024-02-19T16:20:19.634 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 -- 172.21.15.8:0/1451418878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101fc0 msgr2=0x7f616c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 --2- 172.21.15.8:0/1451418878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101fc0 0x7f616c108850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6164009ca0 tx=0x7f616402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 -- 172.21.15.8:0/1451418878 shutdown_connections 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 --2- 172.21.15.8:0/1451418878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101fc0 0x7f616c108850 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 --2- 172.21.15.8:0/1451418878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616c101600 0x7f616c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 --2- 172.21.15.8:0/1451418878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f616c100410 0x7f616c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 -- 172.21.15.8:0/1451418878 >> 172.21.15.8:0/1451418878 conn(0x7f616c0fb980 msgr2=0x7f616c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 -- 172.21.15.8:0/1451418878 shutdown_connections 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.742+0000 7f6173fbc640 1 -- 172.21.15.8:0/1451418878 wait complete. 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 Processor -- start 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 -- start start 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f616c100410 0x7f616c10acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 0x7f616c109310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616c101fc0 0x7f616c109850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c109fb0 con 0x7f616c101600 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c10a120 con 0x7f616c101fc0 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c1a1530 con 0x7f616c100410 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 0x7f616c109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 0x7f616c109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37800/0 (socket says 172.21.15.8:37800) 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 -- 172.21.15.8:0/944748884 learned_addr learned my addr 172.21.15.8:0/944748884 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171d58640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f616c100410 0x7f616c10acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6172559640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616c101fc0 0x7f616c109850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 -- 172.21.15.8:0/944748884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f616c100410 msgr2=0x7f616c10acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f616c100410 0x7f616c10acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 -- 172.21.15.8:0/944748884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616c101fc0 msgr2=0x7f616c109850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616c101fc0 0x7f616c109850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 -- 172.21.15.8:0/944748884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164009900 con 0x7f616c101600 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6171557640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 0x7f616c109310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f615c00d980 tx=0x7f615c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c009a30 con 0x7f616c101600 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f615c009b90 con 0x7f616c101600 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.743+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c1a17e0 con 0x7f616c101600 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.744+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c004a00 con 0x7f616c101600 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.744+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c1a1cf0 con 0x7f616c101600 2024-02-19T16:20:19.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.744+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c1076d0 con 0x7f616c101600 2024-02-19T16:20:19.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.746+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f615c009d00 con 0x7f616c101600 2024-02-19T16:20:19.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.746+0000 7f6162ffd640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f61580a20c0 0x7f61580a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:19.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.746+0000 7f6171d58640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f61580a20c0 0x7f61580a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:19.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.746+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f615c0cd930 con 0x7f616c101600 2024-02-19T16:20:19.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.747+0000 7f6171d58640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f61580a20c0 0x7f61580a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6168005fd0 tx=0x7f616800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:19.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.750+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f615c096c30 con 0x7f616c101600 2024-02-19T16:20:19.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:19.933+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f616c10cf20 con 0x7f616c101600 2024-02-19T16:20:20.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1105905579' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-19T16:20:20.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1987309828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-19T16:20:20.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:19 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1105905579' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1987309828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:19 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1105905579' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-19T16:20:20.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1987309828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-19T16:20:20.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:19 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.618+0000 7f6162ffd640 1 -- 172.21.15.8:0/944748884 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7f615c017020 con 0x7f616c101600 2024-02-19T16:20:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data' created 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f61580a20c0 msgr2=0x7f61580a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f61580a20c0 0x7f61580a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6168005fd0 tx=0x7f616800b390 comp rx=0 tx=0).stop 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 msgr2=0x7f616c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 0x7f616c109310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f615c00d980 tx=0x7f615c00de40 comp rx=0 tx=0).stop 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 shutdown_connections 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f61580a20c0 0x7f61580a4580 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6168005fd0 tx=0x7f616800b390 comp rx=0 tx=0).stop 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616c101fc0 0x7f616c109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f616c101600 0x7f616c109310 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 --2- 172.21.15.8:0/944748884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f616c100410 0x7f616c10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.621+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 >> 172.21.15.8:0/944748884 conn(0x7f616c0fb980 msgr2=0x7f616c1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.623+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 shutdown_connections 2024-02-19T16:20:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.623+0000 7f6173fbc640 1 -- 172.21.15.8:0/944748884 wait complete. 2024-02-19T16:20:20.651 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-19T16:20:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 -- 172.21.15.8:0/3043965045 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c100670 msgr2=0x7f668c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 --2- 172.21.15.8:0/3043965045 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c100670 0x7f668c100af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6680009900 tx=0x7f668002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 -- 172.21.15.8:0/3043965045 shutdown_connections 2024-02-19T16:20:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 --2- 172.21.15.8:0/3043965045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 0x7f668c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 --2- 172.21.15.8:0/3043965045 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c100670 0x7f668c100af0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 --2- 172.21.15.8:0/3043965045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c1019d0 0x7f668c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 -- 172.21.15.8:0/3043965045 >> 172.21.15.8:0/3043965045 conn(0x7f668c0fb0e0 msgr2=0x7f668c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 -- 172.21.15.8:0/3043965045 shutdown_connections 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.761+0000 7f6690e76640 1 -- 172.21.15.8:0/3043965045 wait complete. 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 Processor -- start 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 -- start start 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c100670 0x7f668c1000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 0x7f668c19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 0x7f668c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19f150 con 0x7f668c1019d0 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19f2c0 con 0x7f668c101030 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6690e76640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19f480 con 0x7f668c100670 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 0x7f668c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 0x7f668c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37820/0 (socket says 172.21.15.8:37820) 2024-02-19T16:20:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 -- 172.21.15.8:0/4173462148 learned_addr learned my addr 172.21.15.8:0/4173462148 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f6689d9b640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 0x7f668c19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668a59c640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c100670 0x7f668c1000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 -- 172.21.15.8:0/4173462148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c100670 msgr2=0x7f668c1000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c100670 0x7f668c1000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 -- 172.21.15.8:0/4173462148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 msgr2=0x7f668c19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.762+0000 7f668ad9d640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 0x7f668c19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f668ad9d640 1 -- 172.21.15.8:0/4173462148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009560 con 0x7f668c1019d0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f6689d9b640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 0x7f668c19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f668ad9d640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 0x7f668c19e970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6684009fd0 tx=0x7f668400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400bea0 con 0x7f668c1019d0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c19f700 con 0x7f668c1019d0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c19fbe0 con 0x7f668c1019d0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6684009940 con 0x7f668c1019d0 2024-02-19T16:20:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.763+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66840106c0 con 0x7f668c1019d0 2024-02-19T16:20:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.764+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c067a50 con 0x7f668c1019d0 2024-02-19T16:20:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.766+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f6684004240 con 0x7f668c1019d0 2024-02-19T16:20:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.766+0000 7f667b7fe640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f66740a20c0 0x7f66740a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.766+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f66840cdc20 con 0x7f668c1019d0 2024-02-19T16:20:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.766+0000 7f668a59c640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f66740a20c0 0x7f66740a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.767+0000 7f668a59c640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f66740a20c0 0x7f66740a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f668c068410 tx=0x7f667c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:20.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:20.771+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f6684096e50 con 0x7f668c1019d0 2024-02-19T16:20:21.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.000+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f668c02d120 con 0x7f668c1019d0 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1987309828' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: osdmap e83: 12 total, 12 up, 12 in 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/944748884' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:20:21.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:21.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1987309828' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-19T16:20:21.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: osdmap e83: 12 total, 12 up, 12 in 2024-02-19T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/944748884' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-19T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:20:21.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:21.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1987309828' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-19T16:20:21.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: osdmap e83: 12 total, 12 up, 12 in 2024-02-19T16:20:21.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/944748884' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-19T16:20:21.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:21.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:20:21.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:21.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:20:21.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.726+0000 7f667b7fe640 1 -- 172.21.15.8:0/4173462148 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f66840170c0 con 0x7f668c1019d0 2024-02-19T16:20:21.727 INFO:teuthology.orchestra.run.smithi008.stderr:new fs with metadata pool 2 and data pool 3 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f66740a20c0 msgr2=0x7f66740a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f66740a20c0 0x7f66740a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f668c068410 tx=0x7f667c005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 msgr2=0x7f668c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 0x7f668c19e970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6684009fd0 tx=0x7f668400b260 comp rx=0 tx=0).stop 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 shutdown_connections 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f66740a20c0 0x7f66740a4580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f668c1019d0 0x7f668c19e970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f668c101030 0x7f668c19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 --2- 172.21.15.8:0/4173462148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c100670 0x7f668c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 >> 172.21.15.8:0/4173462148 conn(0x7f668c0fb0e0 msgr2=0x7f668c104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:21.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 shutdown_connections 2024-02-19T16:20:21.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.728+0000 7f6690e76640 1 -- 172.21.15.8:0/4173462148 wait complete. 2024-02-19T16:20:21.744 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-19T16:20:21.744 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 -- 172.21.15.8:0/3335176060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 msgr2=0x7ff38c102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 --2- 172.21.15.8:0/3335176060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c102b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff380009900 tx=0x7ff38002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 -- 172.21.15.8:0/3335176060 shutdown_connections 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 --2- 172.21.15.8:0/3335176060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 0x7ff38c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 --2- 172.21.15.8:0/3335176060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c102b60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 --2- 172.21.15.8:0/3335176060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff38c1014f0 0x7ff38c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 -- 172.21.15.8:0/3335176060 >> 172.21.15.8:0/3335176060 conn(0x7ff38c0fca60 msgr2=0x7ff38c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.865+0000 7ff3923b5640 1 -- 172.21.15.8:0/3335176060 shutdown_connections 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 -- 172.21.15.8:0/3335176060 wait complete. 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 Processor -- start 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 -- start start 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff38c1014f0 0x7ff38c0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c0738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 0x7ff38c073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c074580 con 0x7ff38c1026e0 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0746f0 con 0x7ff38c1030a0 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff3923b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c1a15c0 con 0x7ff38c1014f0 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.866+0000 7ff38b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c0738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c0738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37852/0 (socket says 172.21.15.8:37852) 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 -- 172.21.15.8:0/1089343818 learned_addr learned my addr 172.21.15.8:0/1089343818 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38bfff640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff38c1014f0 0x7ff38c0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff390952640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 0x7ff38c073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 -- 172.21.15.8:0/1089343818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff38c1014f0 msgr2=0x7ff38c0752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff38c1014f0 0x7ff38c0752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 -- 172.21.15.8:0/1089343818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 msgr2=0x7ff38c073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 0x7ff38c073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 -- 172.21.15.8:0/1089343818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380009560 con 0x7ff38c1026e0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff390952640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 0x7ff38c073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff38b7fe640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c0738e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff380036040 tx=0x7ff380005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3800043d0 con 0x7ff38c1026e0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3800052d0 con 0x7ff38c1026e0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c1a1810 con 0x7ff38c1026e0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380003cc0 con 0x7ff38c1026e0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.867+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c1a1d20 con 0x7ff38c1026e0 2024-02-19T16:20:21.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.868+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c108810 con 0x7ff38c1026e0 2024-02-19T16:20:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.869+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff380005440 con 0x7ff38c1026e0 2024-02-19T16:20:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.870+0000 7ff3897fa640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff3740a20c0 0x7ff3740a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.870+0000 7ff38bfff640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff3740a20c0 0x7ff3740a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.870+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3800f03a0 con 0x7ff38c1026e0 2024-02-19T16:20:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.871+0000 7ff38bfff640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff3740a20c0 0x7ff3740a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff38c102540 tx=0x7ff37c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:21.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:21.875+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff3800b94c0 con 0x7ff38c1026e0 2024-02-19T16:20:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:21 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/944748884' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-19T16:20:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:21 smithi008 ceph-mon[32573]: osdmap e84: 12 total, 12 up, 12 in 2024-02-19T16:20:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:21 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4173462148' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-19T16:20:22.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:21 smithi008 ceph-mon[32573]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-02-19T16:20:22.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:21 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/944748884' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-19T16:20:22.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:21 smithi191 ceph-mon[37203]: osdmap e84: 12 total, 12 up, 12 in 2024-02-19T16:20:22.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:22.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:21 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4173462148' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-19T16:20:22.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:21 smithi191 ceph-mon[37203]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-02-19T16:20:22.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:21 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/944748884' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-19T16:20:22.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:21 smithi156 ceph-mon[36502]: osdmap e84: 12 total, 12 up, 12 in 2024-02-19T16:20:22.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:22.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:21 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4173462148' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-19T16:20:22.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:21 smithi156 ceph-mon[36502]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-02-19T16:20:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.080+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ff38c10d0c0 con 0x7ff38c1026e0 2024-02-19T16:20:22.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.653+0000 7ff3897fa640 1 -- 172.21.15.8:0/1089343818 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) v1 ==== 157+0+0 (secure 0 0 0) 0x7ff3800bcb80 con 0x7ff38c1026e0 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff3740a20c0 msgr2=0x7ff3740a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff3740a20c0 0x7ff3740a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff38c102540 tx=0x7ff37c005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 msgr2=0x7ff38c0738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c0738e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff380036040 tx=0x7ff380005e80 comp rx=0 tx=0).stop 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 shutdown_connections 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff3740a20c0 0x7ff3740a4580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff38c1030a0 0x7ff38c073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff38c1026e0 0x7ff38c0738e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 --2- 172.21.15.8:0/1089343818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff38c1014f0 0x7ff38c0752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 >> 172.21.15.8:0/1089343818 conn(0x7ff38c0fca60 msgr2=0x7ff38c1059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:22.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 shutdown_connections 2024-02-19T16:20:22.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.656+0000 7ff3923b5640 1 -- 172.21.15.8:0/1089343818 wait complete. 2024-02-19T16:20:22.669 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-19T16:20:22.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 -- 172.21.15.8:0/3885690347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 msgr2=0x7f8058102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 --2- 172.21.15.8:0/3885690347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058102b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f804c009900 tx=0x7f804c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 -- 172.21.15.8:0/3885690347 shutdown_connections 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 --2- 172.21.15.8:0/3885690347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80581030c0 0x7f8058109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 --2- 172.21.15.8:0/3885690347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058102b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 --2- 172.21.15.8:0/3885690347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8058101510 0x7f8058101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 -- 172.21.15.8:0/3885690347 >> 172.21.15.8:0/3885690347 conn(0x7f80580fcaa0 msgr2=0x7f80580feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 -- 172.21.15.8:0/3885690347 shutdown_connections 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 -- 172.21.15.8:0/3885690347 wait complete. 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 Processor -- start 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.784+0000 7f805e839640 1 -- start start 2024-02-19T16:20:22.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8058101510 0x7f8058197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:22.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:22.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80581030c0 0x7f805819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:22.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805819f110 con 0x7f8058102700 2024-02-19T16:20:22.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805819f280 con 0x7f80581030c0 2024-02-19T16:20:22.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805819f440 con 0x7f8058101510 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37872/0 (socket says 172.21.15.8:37872) 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 -- 172.21.15.8:0/2327560512 learned_addr learned my addr 172.21.15.8:0/2327560512 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f8057fff640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8058101510 0x7f8058197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 -- 172.21.15.8:0/2327560512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8058101510 msgr2=0x7f8058197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8058101510 0x7f8058197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 -- 172.21.15.8:0/2327560512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80581030c0 msgr2=0x7f805819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805cdd6640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80581030c0 0x7f805819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80581030c0 0x7f805819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:22.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 -- 172.21.15.8:0/2327560512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c009560 con 0x7f8058102700 2024-02-19T16:20:22.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80577fe640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058197b60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f804c006010 tx=0x7f804c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:22.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c0044c0 con 0x7f8058102700 2024-02-19T16:20:22.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f804c0052d0 con 0x7f8058102700 2024-02-19T16:20:22.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805819f6c0 con 0x7f8058102700 2024-02-19T16:20:22.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c003e10 con 0x7f8058102700 2024-02-19T16:20:22.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.785+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805819fc50 con 0x7f8058102700 2024-02-19T16:20:22.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.787+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8058108830 con 0x7f8058102700 2024-02-19T16:20:22.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.788+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f804c005440 con 0x7f8058102700 2024-02-19T16:20:22.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.788+0000 7f80557fa640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f80400a20c0 0x7f80400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:22.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.788+0000 7f8057fff640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f80400a20c0 0x7f80400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:22.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.788+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f804c0efff0 con 0x7f8058102700 2024-02-19T16:20:22.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.789+0000 7f8057fff640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f80400a20c0 0x7f80400a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8058102560 tx=0x7f8048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:22.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.793+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f804c0b9030 con 0x7f8058102700 2024-02-19T16:20:22.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:22.977+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f805810d1d0 con 0x7f8058102700 2024-02-19T16:20:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: pgmap v284: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-19T16:20:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: osdmap e85: 12 total, 12 up, 12 in 2024-02-19T16:20:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4173462148' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:22 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1089343818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-19T16:20:23.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: pgmap v284: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: osdmap e85: 12 total, 12 up, 12 in 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4173462148' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:22 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1089343818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-19T16:20:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: pgmap v284: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:20:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-19T16:20:23.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: osdmap e85: 12 total, 12 up, 12 in 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4173462148' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:20:23.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:22 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1089343818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-19T16:20:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:23 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1089343818' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-19T16:20:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:23 smithi008 ceph-mon[32573]: osdmap e86: 12 total, 12 up, 12 in 2024-02-19T16:20:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:23 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] up:active 2024-02-19T16:20:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:23 smithi008 ceph-mon[32573]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:23 smithi008 ceph-mon[32573]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:24.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:23 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2327560512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-19T16:20:24.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:23 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1089343818' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-19T16:20:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:23 smithi191 ceph-mon[37203]: osdmap e86: 12 total, 12 up, 12 in 2024-02-19T16:20:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:23 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] up:active 2024-02-19T16:20:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:23 smithi191 ceph-mon[37203]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:23 smithi191 ceph-mon[37203]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:23 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2327560512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-19T16:20:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:23 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1089343818' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-19T16:20:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:23 smithi156 ceph-mon[36502]: osdmap e86: 12 total, 12 up, 12 in 2024-02-19T16:20:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:23 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] up:active 2024-02-19T16:20:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:23 smithi156 ceph-mon[36502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:23 smithi156 ceph-mon[36502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:24.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:23 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2327560512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-19T16:20:24.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.733+0000 7f80557fa640 1 -- 172.21.15.8:0/2327560512 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) v1 ==== 228+0+0 (secure 0 0 0) 0x7f804c0bc6f0 con 0x7f8058102700 2024-02-19T16:20:24.734 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data_ec' created 2024-02-19T16:20:24.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f80400a20c0 msgr2=0x7f80400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f80400a20c0 0x7f80400a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8058102560 tx=0x7f8048005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 msgr2=0x7f8058197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058197b60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f804c006010 tx=0x7f804c0040c0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 shutdown_connections 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f80400a20c0 0x7f80400a4580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80581030c0 0x7f805819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8058102700 0x7f8058197b60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 --2- 172.21.15.8:0/2327560512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8058101510 0x7f8058197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 >> 172.21.15.8:0/2327560512 conn(0x7f80580fcaa0 msgr2=0x7f8058105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:24.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.737+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 shutdown_connections 2024-02-19T16:20:24.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.738+0000 7f805e839640 1 -- 172.21.15.8:0/2327560512 wait complete. 2024-02-19T16:20:24.753 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:24 smithi008 ceph-mon[32573]: pgmap v287: 97 pgs: 28 creating+peering, 38 active+clean, 31 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-19T16:20:24.753 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:24 smithi008 ceph-mon[32573]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-19T16:20:24.753 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:24 smithi008 ceph-mon[32573]: Cluster is now healthy 2024-02-19T16:20:24.753 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:24 smithi008 ceph-mon[32573]: osdmap e87: 12 total, 12 up, 12 in 2024-02-19T16:20:24.754 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-19T16:20:24.776 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:20:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: 2024-02-19T16:20:24.489+0000 7f52c242d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-19T16:20:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:24 smithi191 ceph-mon[37203]: pgmap v287: 97 pgs: 28 creating+peering, 38 active+clean, 31 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-19T16:20:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:24 smithi191 ceph-mon[37203]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-19T16:20:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:24 smithi191 ceph-mon[37203]: Cluster is now healthy 2024-02-19T16:20:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:24 smithi191 ceph-mon[37203]: osdmap e87: 12 total, 12 up, 12 in 2024-02-19T16:20:24.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 -- 172.21.15.8:0/4088887761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0101030 msgr2=0x7f85c01088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:24.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 --2- 172.21.15.8:0/4088887761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0101030 0x7f85c01088d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009ca0 tx=0x7f85b802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 -- 172.21.15.8:0/4088887761 shutdown_connections 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 --2- 172.21.15.8:0/4088887761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0101030 0x7f85c01088d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 --2- 172.21.15.8:0/4088887761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85c0100670 0x7f85c0100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 --2- 172.21.15.8:0/4088887761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c0101a20 0x7f85c0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 -- 172.21.15.8:0/4088887761 >> 172.21.15.8:0/4088887761 conn(0x7f85c00fb120 msgr2=0x7f85c00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 -- 172.21.15.8:0/4088887761 shutdown_connections 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.860+0000 7f85c646b640 1 -- 172.21.15.8:0/4088887761 wait complete. 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 Processor -- start 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 -- start start 2024-02-19T16:20:24.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 0x7f85c0197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c0101030 0x7f85c0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85c0101a20 0x7f85c019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c019f050 con 0x7f85c0100670 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c019f1c0 con 0x7f85c0101a20 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85c646b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c019f380 con 0x7f85c0101030 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85bffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 0x7f85c0197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85bffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 0x7f85c0197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37896/0 (socket says 172.21.15.8:37896) 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.861+0000 7f85bffff640 1 -- 172.21.15.8:0/53451819 learned_addr learned my addr 172.21.15.8:0/53451819 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bf7fe640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c0101030 0x7f85c0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85c4a08640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85c0101a20 0x7f85c019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bffff640 1 -- 172.21.15.8:0/53451819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c0101030 msgr2=0x7f85c0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bffff640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c0101030 0x7f85c0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bffff640 1 -- 172.21.15.8:0/53451819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85c0101a20 msgr2=0x7f85c019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bffff640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85c0101a20 0x7f85c019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bffff640 1 -- 172.21.15.8:0/53451819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8009900 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bffff640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 0x7f85c0197560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85a800e9c0 tx=0x7f85a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a80096e0 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c019f660 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a800cd80 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8010430 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.862+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c019fb70 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.863+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c010be60 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.864+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f85a8003680 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.865+0000 7f85bd7fa640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85ac0a20c0 0x7f85ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.865+0000 7f85bf7fe640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85ac0a20c0 0x7f85ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.865+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f85a8014070 con 0x7f85c0100670 2024-02-19T16:20:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.865+0000 7f85bf7fe640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85ac0a20c0 0x7f85ac0a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f85c0198a70 tx=0x7f85b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:24.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:24.868+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f85a8097180 con 0x7f85c0100670 2024-02-19T16:20:25.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.049+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f85c010bff0 con 0x7f85c0100670 2024-02-19T16:20:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:24 smithi156 ceph-mon[36502]: pgmap v287: 97 pgs: 28 creating+peering, 38 active+clean, 31 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-19T16:20:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:24 smithi156 ceph-mon[36502]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-19T16:20:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:24 smithi156 ceph-mon[36502]: Cluster is now healthy 2024-02-19T16:20:25.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:24 smithi156 ceph-mon[36502]: osdmap e87: 12 total, 12 up, 12 in 2024-02-19T16:20:25.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.735+0000 7f85bd7fa640 1 -- 172.21.15.8:0/53451819 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7f85a809a840 con 0x7f85c0100670 2024-02-19T16:20:25.736 INFO:teuthology.orchestra.run.smithi008.stderr:set pool 4 allow_ec_overwrites to true 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85ac0a20c0 msgr2=0x7f85ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85ac0a20c0 0x7f85ac0a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f85c0198a70 tx=0x7f85b000b390 comp rx=0 tx=0).stop 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 msgr2=0x7f85c0197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 0x7f85c0197560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85a800e9c0 tx=0x7f85a800ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 shutdown_connections 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85ac0a20c0 0x7f85ac0a4580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85c0101a20 0x7f85c019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c0101030 0x7f85c0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 --2- 172.21.15.8:0/53451819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85c0100670 0x7f85c0197560 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 >> 172.21.15.8:0/53451819 conn(0x7f85c00fb120 msgr2=0x7f85c00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:25.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 shutdown_connections 2024-02-19T16:20:25.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.739+0000 7f85c646b640 1 -- 172.21.15.8:0/53451819 wait complete. 2024-02-19T16:20:25.763 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 -- 172.21.15.8:0/1554961391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 msgr2=0x7f3c7c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1554961391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c108830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009ca0 tx=0x7f3c7402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 -- 172.21.15.8:0/1554961391 shutdown_connections 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1554961391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c108830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1554961391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 0x7f3c7c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1554961391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c7c1003f0 0x7f3c7c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 -- 172.21.15.8:0/1554961391 >> 172.21.15.8:0/1554961391 conn(0x7f3c7c0fb980 msgr2=0x7f3c7c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 -- 172.21.15.8:0/1554961391 shutdown_connections 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 -- 172.21.15.8:0/1554961391 wait complete. 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 Processor -- start 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.872+0000 7f3c8298c640 1 -- start start 2024-02-19T16:20:25.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c8298c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c7c1003f0 0x7f3c7c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c8298c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 0x7f3c7c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c8298c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c8298c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19ef70 con 0x7f3c7c101fa0 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c8298c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19f0e0 con 0x7f3c7c1015e0 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c8298c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19f2a0 con 0x7f3c7c1003f0 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c7c1003f0 0x7f3c7c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55006/0 (socket says 172.21.15.8:55006) 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 -- 172.21.15.8:0/1198628274 learned_addr learned my addr 172.21.15.8:0/1198628274 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c7b7fe640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 0x7f3c7c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 -- 172.21.15.8:0/1198628274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c7c1003f0 msgr2=0x7f3c7c197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:25.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c7c1003f0 0x7f3c7c197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 -- 172.21.15.8:0/1198628274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 msgr2=0x7f3c7c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 0x7f3c7c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 -- 172.21.15.8:0/1198628274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74009900 con 0x7f3c7c101fa0 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c7b7fe640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 0x7f3c7c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c80f29640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c19ea30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c74037040 tx=0x7f3c74034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.873+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740329b0 con 0x7f3c7c101fa0 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.874+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c74003e50 con 0x7f3c7c101fa0 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.874+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c19f520 con 0x7f3c7c101fa0 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.874+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74040450 con 0x7f3c7c101fa0 2024-02-19T16:20:25.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.874+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c19fa20 con 0x7f3c7c101fa0 2024-02-19T16:20:25.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.875+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c10be00 con 0x7f3c7c101fa0 2024-02-19T16:20:25.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.876+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f3c7400a040 con 0x7f3c7c101fa0 2024-02-19T16:20:25.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.877+0000 7f3c797fa640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3c640a2140 0x7f3c640a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:25.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.877+0000 7f3c7bfff640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3c640a2140 0x7f3c640a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:25.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.877+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f3c740f1140 con 0x7f3c7c101fa0 2024-02-19T16:20:25.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.877+0000 7f3c7bfff640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3c640a2140 0x7f3c640a4600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c101440 tx=0x7f3c6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:25.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:25.880+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f3c740b9ef0 con 0x7f3c7c101fa0 2024-02-19T16:20:26.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:25 smithi008 ceph-mon[32573]: pgmap v289: 97 pgs: 28 creating+peering, 53 active+clean, 16 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 B/s wr, 2 op/s 2024-02-19T16:20:26.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:25 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2327560512' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-19T16:20:26.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:25 smithi008 ceph-mon[32573]: osdmap e88: 12 total, 12 up, 12 in 2024-02-19T16:20:26.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:25 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/53451819' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-19T16:20:26.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:25 smithi191 ceph-mon[37203]: pgmap v289: 97 pgs: 28 creating+peering, 53 active+clean, 16 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 B/s wr, 2 op/s 2024-02-19T16:20:26.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:25 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2327560512' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-19T16:20:26.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:25 smithi191 ceph-mon[37203]: osdmap e88: 12 total, 12 up, 12 in 2024-02-19T16:20:26.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:25 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/53451819' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-19T16:20:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:25 smithi156 ceph-mon[36502]: pgmap v289: 97 pgs: 28 creating+peering, 53 active+clean, 16 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 B/s wr, 2 op/s 2024-02-19T16:20:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:25 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2327560512' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-19T16:20:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:25 smithi156 ceph-mon[36502]: osdmap e88: 12 total, 12 up, 12 in 2024-02-19T16:20:26.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:25 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/53451819' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-19T16:20:26.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.103+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f3c7c10bf90 con 0x7f3c7c101fa0 2024-02-19T16:20:26.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.746+0000 7f3c797fa640 1 -- 172.21.15.8:0/1198628274 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3c740bd5b0 con 0x7f3c7c101fa0 2024-02-19T16:20:26.748 INFO:teuthology.orchestra.run.smithi008.stderr:added data pool 4 to fsmap 2024-02-19T16:20:26.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3c640a2140 msgr2=0x7f3c640a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:26.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3c640a2140 0x7f3c640a4600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c101440 tx=0x7f3c6c005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:26.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 msgr2=0x7f3c7c19ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:26.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c19ea30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c74037040 tx=0x7f3c74034a90 comp rx=0 tx=0).stop 2024-02-19T16:20:26.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 shutdown_connections 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3c640a2140 0x7f3c640a4600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c7c101fa0 0x7f3c7c19ea30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c7c1015e0 0x7f3c7c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 --2- 172.21.15.8:0/1198628274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c7c1003f0 0x7f3c7c197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.749+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 >> 172.21.15.8:0/1198628274 conn(0x7f3c7c0fb980 msgr2=0x7f3c7c0fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.750+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 shutdown_connections 2024-02-19T16:20:26.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.750+0000 7f3c8298c640 1 -- 172.21.15.8:0/1198628274 wait complete. 2024-02-19T16:20:26.765 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:26.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 -- 172.21.15.8:0/3461475260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0107270 msgr2=0x7f26e01096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/3461475260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0107270 0x7f26e01096c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009ca0 tx=0x7f26d802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 -- 172.21.15.8:0/3461475260 shutdown_connections 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/3461475260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0107270 0x7f26e01096c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/3461475260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0104940 0x7f26e0106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/3461475260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26e0102010 0x7f26e0104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 -- 172.21.15.8:0/3461475260 >> 172.21.15.8:0/3461475260 conn(0x7f26e00fba20 msgr2=0x7f26e00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 -- 172.21.15.8:0/3461475260 shutdown_connections 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.873+0000 7f26e6eeb640 1 -- 172.21.15.8:0/3461475260 wait complete. 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 Processor -- start 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 -- start start 2024-02-19T16:20:26.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26e0102010 0x7f26e0101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 0x7f26e00fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0107270 0x7f26e0100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0100c60 con 0x7f26e0104940 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0100dd0 con 0x7f26e0107270 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e6eeb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e01a15b0 con 0x7f26e0102010 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 0x7f26e00fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 0x7f26e00fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55022/0 (socket says 172.21.15.8:55022) 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 -- 172.21.15.8:0/1173348965 learned_addr learned my addr 172.21.15.8:0/1173348965 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e4c87640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26e0102010 0x7f26e0101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26e5488640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0107270 0x7f26e0100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 -- 172.21.15.8:0/1173348965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26e0102010 msgr2=0x7f26e0101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26e0102010 0x7f26e0101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 -- 172.21.15.8:0/1173348965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0107270 msgr2=0x7f26e0100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0107270 0x7f26e0100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.874+0000 7f26dffff640 1 -- 172.21.15.8:0/1173348965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009900 con 0x7f26e0104940 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26e5488640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0107270 0x7f26e0100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26dffff640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 0x7f26e00fffc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c9a0 tx=0x7f26cc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc007bb0 con 0x7f26e0104940 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26cc007d10 con 0x7f26e0104940 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e01a1860 con 0x7f26e0104940 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00f450 con 0x7f26e0104940 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.875+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e01a1d70 con 0x7f26e0104940 2024-02-19T16:20:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.876+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e00fd510 con 0x7f26e0104940 2024-02-19T16:20:26.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.877+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f26cc0046c0 con 0x7f26e0104940 2024-02-19T16:20:26.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.877+0000 7f26ddffb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f26d00a20c0 0x7f26d00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:26.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.878+0000 7f26e4c87640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f26d00a20c0 0x7f26d00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:26.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.878+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f26cc0cdff0 con 0x7f26e0104940 2024-02-19T16:20:26.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.878+0000 7f26e4c87640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f26d00a20c0 0x7f26d00a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009bd0 tx=0x7f26d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:26.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:26.882+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f26cc096d20 con 0x7f26e0104940 2024-02-19T16:20:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:26 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/53451819' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-19T16:20:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:26 smithi008 ceph-mon[32573]: osdmap e89: 12 total, 12 up, 12 in 2024-02-19T16:20:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:26 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1198628274' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-19T16:20:27.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:26 smithi008 ceph-mon[32573]: osdmap e90: 12 total, 12 up, 12 in 2024-02-19T16:20:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:26 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/53451819' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-19T16:20:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:26 smithi191 ceph-mon[37203]: osdmap e89: 12 total, 12 up, 12 in 2024-02-19T16:20:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:26 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1198628274' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-19T16:20:27.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:26 smithi191 ceph-mon[37203]: osdmap e90: 12 total, 12 up, 12 in 2024-02-19T16:20:27.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:26 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/53451819' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-19T16:20:27.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:26 smithi156 ceph-mon[36502]: osdmap e89: 12 total, 12 up, 12 in 2024-02-19T16:20:27.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:26 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1198628274' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-19T16:20:27.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:26 smithi156 ceph-mon[36502]: osdmap e90: 12 total, 12 up, 12 in 2024-02-19T16:20:27.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.106+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26e002cf90 con 0x7f26e0104940 2024-02-19T16:20:27.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.106+0000 7f26ddffb640 1 -- 172.21.15.8:0/1173348965 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f26cc01d020 con 0x7f26e0104940 2024-02-19T16:20:27.107 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:27.107 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:26.105247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T16:20:27.107 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 17 2024-02-19T16:20:27.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f26d00a20c0 msgr2=0x7f26d00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f26d00a20c0 0x7f26d00a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009bd0 tx=0x7f26d4005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 msgr2=0x7f26e00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 0x7f26e00fffc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c9a0 tx=0x7f26cc00ce60 comp rx=0 tx=0).stop 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 shutdown_connections 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f26d00a20c0 0x7f26d00a4580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26e0107270 0x7f26e0100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26e0104940 0x7f26e00fffc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 --2- 172.21.15.8:0/1173348965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26e0102010 0x7f26e0101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 >> 172.21.15.8:0/1173348965 conn(0x7f26e00fba20 msgr2=0x7f26e01028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 shutdown_connections 2024-02-19T16:20:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.109+0000 7f26e6eeb640 1 -- 172.21.15.8:0/1173348965 wait complete. 2024-02-19T16:20:27.121 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:27.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 -- 172.21.15.8:0/870474817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 msgr2=0x7f5df81096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 --2- 172.21.15.8:0/870474817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df81096c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009ca0 tx=0x7f5de802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 -- 172.21.15.8:0/870474817 shutdown_connections 2024-02-19T16:20:27.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 --2- 172.21.15.8:0/870474817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df81096c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 --2- 172.21.15.8:0/870474817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5df81049a0 0x7f5df8106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 --2- 172.21.15.8:0/870474817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df8102070 0x7f5df8104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 -- 172.21.15.8:0/870474817 >> 172.21.15.8:0/870474817 conn(0x7f5df80fba20 msgr2=0x7f5df80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 -- 172.21.15.8:0/870474817 shutdown_connections 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 -- 172.21.15.8:0/870474817 wait complete. 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 Processor -- start 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.223+0000 7f5e00223640 1 -- start start 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5df8102070 0x7f5df8197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df81049a0 0x7f5df8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df819f060 con 0x7f5df81072d0 2024-02-19T16:20:27.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df819f1d0 con 0x7f5df8102070 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df819f390 con 0x7f5df81049a0 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55038/0 (socket says 172.21.15.8:55038) 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 -- 172.21.15.8:0/2619272667 learned_addr learned my addr 172.21.15.8:0/2619272667 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfd7be640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df81049a0 0x7f5df8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 -- 172.21.15.8:0/2619272667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df81049a0 msgr2=0x7f5df8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df81049a0 0x7f5df8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 -- 172.21.15.8:0/2619272667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5df8102070 msgr2=0x7f5df8197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5df8102070 0x7f5df8197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 -- 172.21.15.8:0/2619272667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8009900 con 0x7f5df81072d0 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5dfe7c0640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df819eb20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5de8037040 tx=0x7f5de8034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80329b0 con 0x7f5df81072d0 2024-02-19T16:20:27.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5de8003e50 con 0x7f5df81072d0 2024-02-19T16:20:27.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df819f610 con 0x7f5df81072d0 2024-02-19T16:20:27.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8040450 con 0x7f5df81072d0 2024-02-19T16:20:27.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.224+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df819fac0 con 0x7f5df81072d0 2024-02-19T16:20:27.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.225+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df80fd510 con 0x7f5df81072d0 2024-02-19T16:20:27.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.227+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5de800a040 con 0x7f5df81072d0 2024-02-19T16:20:27.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.227+0000 7f5df6ffd640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5de00a20c0 0x7f5de00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.227+0000 7f5dfdfbf640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5de00a20c0 0x7f5de00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.227+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5de80f1200 con 0x7f5df81072d0 2024-02-19T16:20:27.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.228+0000 7f5dfdfbf640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5de00a20c0 0x7f5de00a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5df00058c0 tx=0x7f5df0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:27.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.230+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5de80b9f90 con 0x7f5df81072d0 2024-02-19T16:20:27.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.416+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5df80fcf80 con 0x7f5df81072d0 2024-02-19T16:20:27.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.418+0000 7f5df6ffd640 1 -- 172.21.15.8:0/2619272667 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f5de80bd650 con 0x7f5df81072d0 2024-02-19T16:20:27.419 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:27.419 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5de00a20c0 msgr2=0x7f5de00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5de00a20c0 0x7f5de00a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5df00058c0 tx=0x7f5df0005850 comp rx=0 tx=0).stop 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 msgr2=0x7f5df819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df819eb20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5de8037040 tx=0x7f5de8034a90 comp rx=0 tx=0).stop 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 shutdown_connections 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5de00a20c0 0x7f5de00a4580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5df81072d0 0x7f5df819eb20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df81049a0 0x7f5df8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 --2- 172.21.15.8:0/2619272667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5df8102070 0x7f5df8197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 >> 172.21.15.8:0/2619272667 conn(0x7f5df80fba20 msgr2=0x7f5df8102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:27.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 shutdown_connections 2024-02-19T16:20:27.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.420+0000 7f5e00223640 1 -- 172.21.15.8:0/2619272667 wait complete. 2024-02-19T16:20:27.432 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:27.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 -- 172.21.15.8:0/2444201836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 msgr2=0x7fdaac101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 --2- 172.21.15.8:0/2444201836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac101a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009900 tx=0x7fda9c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 -- 172.21.15.8:0/2444201836 shutdown_connections 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 --2- 172.21.15.8:0/2444201836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaac101fc0 0x7fdaac108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 --2- 172.21.15.8:0/2444201836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac101a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 --2- 172.21.15.8:0/2444201836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdaac100410 0x7fdaac100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 -- 172.21.15.8:0/2444201836 >> 172.21.15.8:0/2444201836 conn(0x7fdaac0fb980 msgr2=0x7fdaac0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 -- 172.21.15.8:0/2444201836 shutdown_connections 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.533+0000 7fdab1d08640 1 -- 172.21.15.8:0/2444201836 wait complete. 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 Processor -- start 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 -- start start 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdaac100410 0x7fdaac0691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaac101fc0 0x7fdaac19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac19f180 con 0x7fdaac101600 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac19f2f0 con 0x7fdaac101fc0 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdab1d08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac19f4b0 con 0x7fdaac100410 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdaaaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdaaaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac19c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55058/0 (socket says 172.21.15.8:55058) 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdaaaffd640 1 -- 172.21.15.8:0/524884828 learned_addr learned my addr 172.21.15.8:0/524884828 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.534+0000 7fdaab7fe640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdaac100410 0x7fdaac0691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaabfff640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaac101fc0 0x7fdaac19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaaaffd640 1 -- 172.21.15.8:0/524884828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdaac100410 msgr2=0x7fdaac0691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaaaffd640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdaac100410 0x7fdaac0691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaaaffd640 1 -- 172.21.15.8:0/524884828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaac101fc0 msgr2=0x7fdaac19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaaaffd640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaac101fc0 0x7fdaac19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaaaffd640 1 -- 172.21.15.8:0/524884828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c009560 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaaaffd640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac19c380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0044c0 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda9c0052d0 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac19f6d0 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c003e10 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.535+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac19fc60 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.536+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac107730 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.537+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fda9c005440 con 0x7fdaac101600 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.538+0000 7fdaa8ff9640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda980a20c0 0x7fda980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.538+0000 7fdaab7fe640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda980a20c0 0x7fda980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.538+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fda9c0f0200 con 0x7fdaac101600 2024-02-19T16:20:27.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.538+0000 7fdaab7fe640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda980a20c0 0x7fda980a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdaac101460 tx=0x7fda94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:27.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.542+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fda9c0b9030 con 0x7fdaac101600 2024-02-19T16:20:27.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.729+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaac10c0d0 con 0x7fdaac101600 2024-02-19T16:20:27.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.730+0000 7fdaa8ff9640 1 -- 172.21.15.8:0/524884828 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fda9c0bc6f0 con 0x7fdaac101600 2024-02-19T16:20:27.731 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:27.731 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda980a20c0 msgr2=0x7fda980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda980a20c0 0x7fda980a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdaac101460 tx=0x7fda94005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 msgr2=0x7fdaac19c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac19c380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c0040c0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 shutdown_connections 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda980a20c0 0x7fda980a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaac101fc0 0x7fdaac19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdaac101600 0x7fdaac19c380 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 --2- 172.21.15.8:0/524884828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdaac100410 0x7fdaac0691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 >> 172.21.15.8:0/524884828 conn(0x7fdaac0fb980 msgr2=0x7fdaac104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 shutdown_connections 2024-02-19T16:20:27.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.733+0000 7fdab1d08640 1 -- 172.21.15.8:0/524884828 wait complete. 2024-02-19T16:20:27.760 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:27.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 -- 172.21.15.8:0/419846601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101fc0 msgr2=0x7f8150108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 --2- 172.21.15.8:0/419846601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101fc0 0x7f8150108850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f814c009ca0 tx=0x7f814c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 -- 172.21.15.8:0/419846601 shutdown_connections 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 --2- 172.21.15.8:0/419846601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101fc0 0x7f8150108850 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 --2- 172.21.15.8:0/419846601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101600 0x7f8150101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 --2- 172.21.15.8:0/419846601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 0x7f8150100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 -- 172.21.15.8:0/419846601 >> 172.21.15.8:0/419846601 conn(0x7f81500fb980 msgr2=0x7f81500fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 -- 172.21.15.8:0/419846601 shutdown_connections 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.861+0000 7f8157393640 1 -- 172.21.15.8:0/419846601 wait complete. 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 Processor -- start 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 -- start start 2024-02-19T16:20:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 0x7f8150197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 0x7f8150197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101fc0 0x7f815019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019f030 con 0x7f8150101600 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019f1a0 con 0x7f8150100410 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8157393640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019f360 con 0x7f8150101fc0 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 0x7f8150197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 0x7f8150197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55078/0 (socket says 172.21.15.8:55078) 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 -- 172.21.15.8:0/3575649054 learned_addr learned my addr 172.21.15.8:0/3575649054 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8155930640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101fc0 0x7f815019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815512f640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 0x7f8150197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 -- 172.21.15.8:0/3575649054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101fc0 msgr2=0x7f815019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101fc0 0x7f815019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 -- 172.21.15.8:0/3575649054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 msgr2=0x7f8150197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 0x7f8150197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 -- 172.21.15.8:0/3575649054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c009900 con 0x7f8150101600 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f8155930640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101fc0 0x7f815019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815512f640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 0x7f8150197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.862+0000 7f815492e640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 0x7f8150197a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f814000e9f0 tx=0x7f814000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.863+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140009720 con 0x7f8150101600 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.863+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f814000cd80 con 0x7f8150101600 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.863+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815019f640 con 0x7f8150101600 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.863+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140010690 con 0x7f8150101600 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.863+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815019fb50 con 0x7f8150101600 2024-02-19T16:20:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.864+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81501076d0 con 0x7f8150101600 2024-02-19T16:20:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.865+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f81400046c0 con 0x7f8150101600 2024-02-19T16:20:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.866+0000 7f81467fc640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f813c0a2140 0x7f813c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.866+0000 7f815512f640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f813c0a2140 0x7f813c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.866+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8140014070 con 0x7f8150101600 2024-02-19T16:20:27.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.866+0000 7f815512f640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f813c0a2140 0x7f813c0a4600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81480058c0 tx=0x7f8148005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:27.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:27.870+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8140097290 con 0x7f8150101600 2024-02-19T16:20:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: pgmap v293: 129 pgs: 4 creating+activating, 11 creating+peering, 107 active+clean, 7 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-02-19T16:20:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1198628274' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-19T16:20:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1173348965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: osdmap e91: 12 total, 12 up, 12 in 2024-02-19T16:20:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2619272667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:27 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/524884828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: pgmap v293: 129 pgs: 4 creating+activating, 11 creating+peering, 107 active+clean, 7 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-02-19T16:20:28.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1198628274' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-19T16:20:28.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:28.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1173348965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: osdmap e91: 12 total, 12 up, 12 in 2024-02-19T16:20:28.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2619272667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:27 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/524884828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: pgmap v293: 129 pgs: 4 creating+activating, 11 creating+peering, 107 active+clean, 7 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-02-19T16:20:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1198628274' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-19T16:20:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1173348965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: osdmap e91: 12 total, 12 up, 12 in 2024-02-19T16:20:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2619272667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:27 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/524884828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.091+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f815010cf20 con 0x7f8150101600 2024-02-19T16:20:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.091+0000 7f81467fc640 1 -- 172.21.15.8:0/3575649054 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f814009a950 con 0x7f8150101600 2024-02-19T16:20:28.092 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:28.093 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:26.105247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T16:20:28.093 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 17 2024-02-19T16:20:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f813c0a2140 msgr2=0x7f813c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f813c0a2140 0x7f813c0a4600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81480058c0 tx=0x7f8148005850 comp rx=0 tx=0).stop 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 msgr2=0x7f8150197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 0x7f8150197a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f814000e9f0 tx=0x7f814000edf0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 shutdown_connections 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f813c0a2140 0x7f813c0a4600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150101fc0 0x7f815019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8150101600 0x7f8150197a80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 --2- 172.21.15.8:0/3575649054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150100410 0x7f8150197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 >> 172.21.15.8:0/3575649054 conn(0x7f81500fb980 msgr2=0x7f81501048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 shutdown_connections 2024-02-19T16:20:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.094+0000 7f8157393640 1 -- 172.21.15.8:0/3575649054 wait complete. 2024-02-19T16:20:28.106 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:28.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 -- 172.21.15.8:0/3637266640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598107270 msgr2=0x7fd5981096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/3637266640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598107270 0x7fd5981096c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd590009ca0 tx=0x7fd59002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 -- 172.21.15.8:0/3637266640 shutdown_connections 2024-02-19T16:20:28.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/3637266640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598107270 0x7fd5981096c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/3637266640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd598104940 0x7fd598106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/3637266640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd598102010 0x7fd598104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 -- 172.21.15.8:0/3637266640 >> 172.21.15.8:0/3637266640 conn(0x7fd5980fba20 msgr2=0x7fd5980fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 -- 172.21.15.8:0/3637266640 shutdown_connections 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.207+0000 7fd59cf3e640 1 -- 172.21.15.8:0/3637266640 wait complete. 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 Processor -- start 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 -- start start 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 0x7fd5981975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd598104940 0x7fd598197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd598107270 0x7fd59819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59819f0b0 con 0x7fd598102010 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59819f220 con 0x7fd598104940 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59cf3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59819f3e0 con 0x7fd598107270 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 0x7fd5981975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 0x7fd5981975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55088/0 (socket says 172.21.15.8:55088) 2024-02-19T16:20:28.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 -- 172.21.15.8:0/867636321 learned_addr learned my addr 172.21.15.8:0/867636321 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd596d9d640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd598107270 0x7fd59819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd58dd9b640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd598104940 0x7fd598197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 -- 172.21.15.8:0/867636321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd598107270 msgr2=0x7fd59819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd598107270 0x7fd59819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 -- 172.21.15.8:0/867636321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd598104940 msgr2=0x7fd598197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd598104940 0x7fd598197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.208+0000 7fd59659c640 1 -- 172.21.15.8:0/867636321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd590009900 con 0x7fd598102010 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.209+0000 7fd59659c640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 0x7fd5981975c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd58800e9f0 tx=0x7fd58800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.209+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588009720 con 0x7fd598102010 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.209+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59819f6c0 con 0x7fd598102010 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.209+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd58800cd80 con 0x7fd598102010 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.209+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588010690 con 0x7fd598102010 2024-02-19T16:20:28.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.209+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59819fb80 con 0x7fd598102010 2024-02-19T16:20:28.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.210+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980fd510 con 0x7fd598102010 2024-02-19T16:20:28.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.211+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd5880046c0 con 0x7fd598102010 2024-02-19T16:20:28.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.211+0000 7fd58ffff640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd5840a20c0 0x7fd5840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.212+0000 7fd58dd9b640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd5840a20c0 0x7fd5840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.212+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd588014070 con 0x7fd598102010 2024-02-19T16:20:28.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.212+0000 7fd58dd9b640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd5840a20c0 0x7fd5840a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd598198ad0 tx=0x7fd580008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.216+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd588097090 con 0x7fd598102010 2024-02-19T16:20:28.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.397+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd59810cfb0 con 0x7fd598102010 2024-02-19T16:20:28.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.399+0000 7fd58ffff640 1 -- 172.21.15.8:0/867636321 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fd58809a750 con 0x7fd598102010 2024-02-19T16:20:28.400 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:28.400 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd5840a20c0 msgr2=0x7fd5840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd5840a20c0 0x7fd5840a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd598198ad0 tx=0x7fd580008040 comp rx=0 tx=0).stop 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 msgr2=0x7fd5981975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 0x7fd5981975c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd58800e9f0 tx=0x7fd58800edf0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 shutdown_connections 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd5840a20c0 0x7fd5840a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd598107270 0x7fd59819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd598104940 0x7fd598197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 --2- 172.21.15.8:0/867636321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd598102010 0x7fd5981975c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 >> 172.21.15.8:0/867636321 conn(0x7fd5980fba20 msgr2=0x7fd598102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 shutdown_connections 2024-02-19T16:20:28.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.401+0000 7fd59cf3e640 1 -- 172.21.15.8:0/867636321 wait complete. 2024-02-19T16:20:28.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-19T16:20:21.002558+0000', 'modified': '2024-02-19T16:20:26.105247+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.191:6839/2229091821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2229091821}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2229091821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-19T16:20:28.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-19T16:20:28.414 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 -- 172.21.15.8:0/2444767840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 msgr2=0x7fab400ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 --2- 172.21.15.8:0/2444767840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab400ffbf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab30009900 tx=0x7fab3002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 -- 172.21.15.8:0/2444767840 shutdown_connections 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 --2- 172.21.15.8:0/2444767840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab40103160 0x7fab401099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 --2- 172.21.15.8:0/2444767840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab401027a0 0x7fab40102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 --2- 172.21.15.8:0/2444767840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab400ffbf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 -- 172.21.15.8:0/2444767840 >> 172.21.15.8:0/2444767840 conn(0x7fab400fb0e0 msgr2=0x7fab400fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 -- 172.21.15.8:0/2444767840 shutdown_connections 2024-02-19T16:20:28.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 -- 172.21.15.8:0/2444767840 wait complete. 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 Processor -- start 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.516+0000 7fab46348640 1 -- start start 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab401975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab401027a0 0x7fab40197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab40103160 0x7fab4019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4019f0c0 con 0x7fab400ff7d0 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4019f230 con 0x7fab401027a0 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4019f3f0 con 0x7fab40103160 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab401975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab401975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55092/0 (socket says 172.21.15.8:55092) 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 -- 172.21.15.8:0/2937739551 learned_addr learned my addr 172.21.15.8:0/2937739551 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab448e5640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab40103160 0x7fab4019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 -- 172.21.15.8:0/2937739551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab40103160 msgr2=0x7fab4019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab40103160 0x7fab4019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab401027a0 0x7fab40197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 -- 172.21.15.8:0/2937739551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab401027a0 msgr2=0x7fab40197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab401027a0 0x7fab40197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 -- 172.21.15.8:0/2937739551 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009560 con 0x7fab400ff7d0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3ffff640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab401975d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fab30000c00 tx=0x7fab30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3d7fa640 1 -- 172.21.15.8:0/2937739551 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30004df0 con 0x7fab400ff7d0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3d7fa640 1 -- 172.21.15.8:0/2937739551 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab30005430 con 0x7fab400ff7d0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4019f670 con 0x7fab400ff7d0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.517+0000 7fab3d7fa640 1 -- 172.21.15.8:0/2937739551 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3003e450 con 0x7fab400ff7d0 2024-02-19T16:20:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.518+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4019fbd0 con 0x7fab400ff7d0 2024-02-19T16:20:28.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab3d7fa640 1 -- 172.21.15.8:0/2937739551 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fab300035f0 con 0x7fab400ff7d0 2024-02-19T16:20:28.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 Processor -- start 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 -- start start 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab3d7fa640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab280a2140 0x7fab280a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 0x56387ffd33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab448e5640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab280a2140 0x7fab280a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd3970 0x56387ffd8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab3d7fa640 1 -- 172.21.15.8:0/2937739551 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fab300f0120 con 0x7fab400ff7d0 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffdaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ff8ecd0 con 0x56387ffd2f80 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ff41f20 con 0x56387ffd8ae0 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab49003740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ff3f450 con 0x56387ffd3970 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.520+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 0x56387ffd33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 0x56387ffd33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55106/0 (socket says 172.21.15.8:55106) 2024-02-19T16:20:28.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 -- 172.21.15.8:0/2758215573 learned_addr learned my addr 172.21.15.8:0/2758215573 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd3970 0x56387ffd8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab448e5640 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffdaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 -- 172.21.15.8:0/2758215573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd3970 msgr2=0x56387ffd8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd3970 0x56387ffd8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 -- 172.21.15.8:0/2758215573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 msgr2=0x56387ffdaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffdaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 -- 172.21.15.8:0/2758215573 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56387feeff20 con 0x56387ffd2f80 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab448e5640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab280a2140 0x7fab280a4600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab38005fd0 tx=0x7fab38010740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab3ffff640 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 0x56387ffd33a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab300b95e0 tx=0x7fab30003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23100e8d5a170305 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab267fc640 1 -- 172.21.15.8:0/2758215573 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300edea0 con 0x56387ffd2f80 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab267fc640 1 -- 172.21.15.8:0/2758215573 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab300bcc10 con 0x56387ffd2f80 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.521+0000 7fab267fc640 1 -- 172.21.15.8:0/2758215573 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300056f0 con 0x56387ffd2f80 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 -- 172.21.15.8:0/2758215573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 msgr2=0x56387ffd33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 0x56387ffd33a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab300b95e0 tx=0x7fab30003990 comp rx=0 tx=0).stop 2024-02-19T16:20:28.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 -- 172.21.15.8:0/2758215573 shutdown_connections 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffdaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd3970 0x56387ffd8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 --2- 172.21.15.8:0/2758215573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd2f80 0x56387ffd33a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 -- 172.21.15.8:0/2758215573 >> 172.21.15.8:0/2758215573 conn(0x56387ffce4f0 msgr2=0x56387ffce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 -- 172.21.15.8:0/2758215573 shutdown_connections 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 -- 172.21.15.8:0/2758215573 wait complete. 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 -- start start 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.522+0000 7fab49003740 1 Processor -- start 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffdce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffdd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffe4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ff3ac30 con 0x56387ffd3970 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387fedfd30 con 0x56387ffd2f80 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387fc0e310 con 0x56387ffd8ae0 2024-02-19T16:20:28.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 10 client.0 _ll_register_callbacks cb 0x56387fee7d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffdd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffdd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55118/0 (socket says 172.21.15.8:55118) 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 -- 172.21.15.8:0/3072589405 learned_addr learned my addr 172.21.15.8:0/3072589405 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab448e5640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffe4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3ffff640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffdce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 -- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 msgr2=0x56387ffe4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffe4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 -- 172.21.15.8:0/3072589405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 msgr2=0x56387ffdce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffdce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 -- 172.21.15.8:0/3072589405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4019fbd0 con 0x56387ffd3970 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab448e5640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffe4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3ffff640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffdce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:28.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffdd3d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fab34013710 tx=0x7fab34013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6995905e0ede20 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab1f7fe640 1 -- 172.21.15.8:0/3072589405 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34011d70 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab1f7fe640 1 -- 172.21.15.8:0/3072589405 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fab34004020 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56387fc3a6c0 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab1f7fe640 1 -- 172.21.15.8:0/3072589405 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab34011ed0 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 10 client.14745 fetch_fsmap learned FSMap version 17 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.523+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56387ff392b0 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab1f7fe640 1 -- 172.21.15.8:0/3072589405 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56387fc3a6c0 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab1f7fe640 1 -- 172.21.15.8:0/3072589405 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fab34004a00 con 0x56387ffd3970 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab49003740 10 client.14745 fetch_fsmap finished waiting for FSMap version 17 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab49003740 10 client.14745 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab49003740 20 client.14745 populate_metadata read hostname 'smithi008' 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab49003740 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56387ffed6f0 0x56387ffecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab49003740 4 client.14745 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:20:28.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56387fc8b270 con 0x56387ffed6f0 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab3ffff640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56387ffed6f0 0x56387ffecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab3ffff640 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56387ffed6f0 0x56387ffecfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6b0c40fe372c97 server_cookie=2241e9a0a19b5055 in_seq=0 out_seq=0 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.524+0000 7fab1f7fe640 10 client.14745 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab1f7fe640 1 -- 172.21.15.8:0/3072589405 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56387fc8b270 con 0x56387ffed6f0 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab1f7fe640 10 client.14745 handle_command_reply: tid=0 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 client.14745 shutdown 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 msgr2=0x56387ffdd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffdd3d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fab34013710 tx=0x7fab34013bd0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 shutdown_connections 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56387ffed6f0 0x56387ffecfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffe4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffdd3d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab1f7fe640 0 client.14745 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 --2- 172.21.15.8:0/3072589405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffdce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 >> 172.21.15.8:0/3072589405 conn(0x56387ffce4f0 msgr2=0x56387ffcd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.525+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 shutdown_connections 2024-02-19T16:20:28.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.526+0000 7fab49003740 1 -- 172.21.15.8:0/3072589405 wait complete. 2024-02-19T16:20:28.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.526+0000 7fab49003740 20 client.14745 trim_cache size 0 max 16384 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 Processor -- start 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 -- start start 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffd4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387fec0490 con 0x56387ffd3970 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ffdde80 con 0x56387ffd2f80 2024-02-19T16:20:28.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab49003740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ffcef40 con 0x56387ffd8ae0 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55130/0 (socket says 172.21.15.8:55130) 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab3f7fe640 1 -- 172.21.15.8:0/104868474 learned_addr learned my addr 172.21.15.8:0/104868474 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab448e5640 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.542+0000 7fab3ffff640 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffd4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3f7fe640 1 -- 172.21.15.8:0/104868474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 msgr2=0x56387ffccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3f7fe640 1 -- 172.21.15.8:0/104868474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 msgr2=0x56387ffd4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffd4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3f7fe640 1 -- 172.21.15.8:0/104868474 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56387fed5a20 con 0x56387ffd3970 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3ffff640 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffd4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe43d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fab34001eb0 tx=0x7fab340118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fd3a3b7e01a7f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab25ffb640 1 -- 172.21.15.8:0/104868474 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34011b70 con 0x56387ffd3970 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab25ffb640 1 -- 172.21.15.8:0/104868474 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab34004b40 con 0x56387ffd3970 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab25ffb640 1 -- 172.21.15.8:0/104868474 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34005670 con 0x56387ffd3970 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 -- 172.21.15.8:0/104868474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 msgr2=0x56387ffe43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe43d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fab34001eb0 tx=0x7fab340118e0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 -- 172.21.15.8:0/104868474 shutdown_connections 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd8ae0 0x56387ffccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe43d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 --2- 172.21.15.8:0/104868474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd2f80 0x56387ffd4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 -- 172.21.15.8:0/104868474 >> 172.21.15.8:0/104868474 conn(0x56387ffce4f0 msgr2=0x56387ffdb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 -- 172.21.15.8:0/104868474 shutdown_connections 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.543+0000 7fab49003740 1 -- 172.21.15.8:0/104868474 wait complete. 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 -- start start 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 Processor -- start 2024-02-19T16:20:28.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd2f80 0x56387ffe43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffe13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ffdc6e0 con 0x56387ffd3970 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387ffe5090 con 0x56387ffd8ae0 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563880048d90 con 0x56387ffd2f80 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab49003740 10 client.0 _ll_register_callbacks cb 0x7fab340096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55136/0 (socket says 172.21.15.8:55136) 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1087479188 learned_addr learned my addr 172.21.15.8:0/1087479188 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab3ffff640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd2f80 0x56387ffe43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab448e5640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffe13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.544+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd2f80 msgr2=0x56387ffe43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd2f80 0x56387ffe43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1087479188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 msgr2=0x56387ffe13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffe13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1087479188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56387ff392b0 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe0e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fab34022000 tx=0x7fab340055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa84fa435694495 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab1f7fe640 1 -- 172.21.15.8:0/1087479188 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34011d70 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab1f7fe640 1 -- 172.21.15.8:0/1087479188 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fab34000e40 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56387ffee0a0 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab1f7fe640 1 -- 172.21.15.8:0/1087479188 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab34011ed0 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 10 client.14751 fetch_fsmap learned FSMap version 17 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56387fefb780 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab1f7fe640 1 -- 172.21.15.8:0/1087479188 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3403e020 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab1f7fe640 1 -- 172.21.15.8:0/1087479188 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fab34027070 con 0x56387ffd3970 2024-02-19T16:20:28.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 10 client.14751 fetch_fsmap finished waiting for FSMap version 17 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 10 client.14751 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 20 client.14751 populate_metadata read hostname 'smithi008' 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56388004dca0 0x56388004a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 4 client.14751 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.545+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fab300bcc10 con 0x56388004dca0 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab3ffff640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56388004dca0 0x56388004a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab1f7fe640 10 client.14751 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab3ffff640 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56388004dca0 0x56388004a120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d34a70ee2bafb4 server_cookie=7f2ca3999aa0e336 in_seq=0 out_seq=0 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab1f7fe640 1 -- 172.21.15.8:0/1087479188 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7fab300bcc10 con 0x56388004dca0 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab1f7fe640 10 client.14751 handle_command_reply: tid=0 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 client.14751 shutdown 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 msgr2=0x56387ffe0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe0e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fab34022000 tx=0x7fab340055e0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 shutdown_connections 2024-02-19T16:20:28.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56387ffd8ae0 0x56387ffe13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56387ffd3970 0x56387ffe0e50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56388004dca0 0x56388004a120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 --2- 172.21.15.8:0/1087479188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56387ffd2f80 0x56387ffe43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 >> 172.21.15.8:0/1087479188 conn(0x56387ffce4f0 msgr2=0x56387ffd4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab1f7fe640 0 client.14751 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 shutdown_connections 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 1 -- 172.21.15.8:0/1087479188 wait complete. 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.546+0000 7fab49003740 20 client.14751 trim_cache size 0 max 16384 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:20:28.555 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24502, 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 6.8191804549999997, 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 17, 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 91, 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 90, 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 19.567094595 2024-02-19T16:20:28.556 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab280a2140 msgr2=0x7fab280a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab280a2140 0x7fab280a4600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab38005fd0 tx=0x7fab38010740 comp rx=0 tx=0).stop 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 msgr2=0x7fab401975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab401975d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fab30000c00 tx=0x7fab30004270 comp rx=0 tx=0).stop 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 shutdown_connections 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab280a2140 0x7fab280a4600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab40103160 0x7fab4019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab401027a0 0x7fab40197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 --2- 172.21.15.8:0/2937739551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab400ff7d0 0x7fab401975d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:28.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 >> 172.21.15.8:0/2937739551 conn(0x7fab400fb0e0 msgr2=0x7fab40105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:28.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 shutdown_connections 2024-02-19T16:20:28.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:28.547+0000 7fab46348640 1 -- 172.21.15.8:0/2937739551 wait complete. 2024-02-19T16:20:28.559 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-19T16:20:28.559 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:28.559 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph fs ls 2024-02-19T16:20:28.801 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:29.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:28 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3575649054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:29.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:28 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/867636321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:29.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:28 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3575649054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:29.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:28 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/867636321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:29.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:28 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3575649054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:29.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:28 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/867636321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:30.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:29 smithi008 ceph-mon[32573]: pgmap v295: 129 pgs: 4 creating+activating, 11 creating+peering, 114 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-02-19T16:20:30.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:29 smithi191 ceph-mon[37203]: pgmap v295: 129 pgs: 4 creating+activating, 11 creating+peering, 114 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-02-19T16:20:30.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:29 smithi156 ceph-mon[36502]: pgmap v295: 129 pgs: 4 creating+activating, 11 creating+peering, 114 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2877364261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 msgr2=0x7f94f010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2877364261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010dd90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009990 tx=0x7f94e0009ca0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2877364261 shutdown_connections 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2877364261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010dd90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2877364261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101170 0x7f94f0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2877364261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94f0101b20 0x7f94f0105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2877364261 >> 172.21.15.8:0/2877364261 conn(0x7f94f0076f30 msgr2=0x7f94f0077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2877364261 shutdown_connections 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.581+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2877364261 wait complete. 2024-02-19T16:20:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 Processor -- start 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 -- start start 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94f0101170 0x7f94f010e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101b20 0x7f94f010ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01a6960 con 0x7f94f01061c0 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01a6ad0 con 0x7f94f0101170 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f6eb6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01a6c90 con 0x7f94f0101b20 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55142/0 (socket says 172.21.15.8:55142) 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 -- 172.21.15.8:0/2038660898 learned_addr learned my addr 172.21.15.8:0/2038660898 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94effff700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101b20 0x7f94f010ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f4c52700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94f0101170 0x7f94f010e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 -- 172.21.15.8:0/2038660898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101b20 msgr2=0x7f94f010ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101b20 0x7f94f010ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 -- 172.21.15.8:0/2038660898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94f0101170 msgr2=0x7f94f010e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94f0101170 0x7f94f010e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 -- 172.21.15.8:0/2038660898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00095f0 con 0x7f94f01061c0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94effff700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101b20 0x7f94f010ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.582+0000 7f94f5453700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010f120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f94e0004730 tx=0x7f94e0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.583+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e001d070 con 0x7f94f01061c0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.583+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94e000b610 con 0x7f94f01061c0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.583+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f01a6f10 con 0x7f94f01061c0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.583+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00173f0 con 0x7f94f01061c0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.583+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f01a72c0 con 0x7f94f01061c0 2024-02-19T16:20:30.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.584+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f01094e0 con 0x7f94f01061c0 2024-02-19T16:20:30.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.585+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f94e0022470 con 0x7f94f01061c0 2024-02-19T16:20:30.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.585+0000 7f94edffb700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f94d80a2150 0x7f94d80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:30.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.585+0000 7f94f4c52700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f94d80a2150 0x7f94d80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:30.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.586+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f94e00cfa80 con 0x7f94f01061c0 2024-02-19T16:20:30.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.586+0000 7f94f4c52700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f94d80a2150 0x7f94d80a4610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94e800f540 tx=0x7f94e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:30.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.589+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f94e0098840 con 0x7f94f01061c0 2024-02-19T16:20:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.840+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94f0068ca0 con 0x7f94f01061c0 2024-02-19T16:20:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.840+0000 7f94edffb700 1 -- 172.21.15.8:0/2038660898 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f94e0027070 con 0x7f94f01061c0 2024-02-19T16:20:30.841 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-19T16:20:30.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f94d80a2150 msgr2=0x7f94d80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:30.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f94d80a2150 0x7f94d80a4610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94e800f540 tx=0x7f94e800b040 comp rx=0 tx=0).stop 2024-02-19T16:20:30.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 msgr2=0x7f94f010f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:30.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010f120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f94e0004730 tx=0x7f94e0009580 comp rx=0 tx=0).stop 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 shutdown_connections 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f94d80a2150 0x7f94d80a4610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f94f01061c0 0x7f94f010f120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94f0101170 0x7f94f010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 --2- 172.21.15.8:0/2038660898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f0101b20 0x7f94f010ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.842+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 >> 172.21.15.8:0/2038660898 conn(0x7f94f0076f30 msgr2=0x7f94f00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.843+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 shutdown_connections 2024-02-19T16:20:30.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:30.843+0000 7f94f6eb6700 1 -- 172.21.15.8:0/2038660898 wait complete. 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi008.front.sepia.ceph.com 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-19T16:20:31.518 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-19T16:20:31.519 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-19T16:20:31.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:31.519 DEBUG:teuthology.orchestra.run.smithi008:> ip addr 2024-02-19T16:20:31.535 INFO:teuthology.orchestra.run.smithi008.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-19T16:20:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-19T16:20:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: inet 127.0.0.1/8 scope host lo 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 ::1/128 scope host 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2e brd ff:ff:ff:ff:ff:ff 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f0 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2f brd ff:ff:ff:ff:ff:ff 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f1 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:fe brd ff:ff:ff:ff:ff:ff 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f0 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.21.15.8/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-19T16:20:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft 41961sec preferred_lft 41961sec 2024-02-19T16:20:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link noprefixroute 2024-02-19T16:20:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-02-19T16:20:31.537 INFO:teuthology.orchestra.run.smithi008.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T16:20:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:ff brd ff:ff:ff:ff:ff:ff 2024-02-19T16:20:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f1 2024-02-19T16:20:31.537 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-19T16:20:31.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add name ceph-brx type bridge 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr flush dev ceph-brx 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set ceph-brx up 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-19T16:20:31.538 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:20:31.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:31.698 DEBUG:teuthology.orchestra.run.smithi008:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-19T16:20:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:1 2024-02-19T16:20:31.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:31.726 DEBUG:teuthology.orchestra.run.smithi008:> route 2024-02-19T16:20:31.799 INFO:teuthology.orchestra.run.smithi008.stdout:Kernel IP routing table 2024-02-19T16:20:31.799 INFO:teuthology.orchestra.run.smithi008.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-19T16:20:31.799 INFO:teuthology.orchestra.run.smithi008.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-19T16:20:31.799 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-19T16:20:31.799 INFO:teuthology.orchestra.run.smithi008.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-19T16:20:31.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:31.800 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:20:31.800 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:20:31.800 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-19T16:20:31.800 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-19T16:20:31.800 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-19T16:20:31.800 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:20:32.011 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:31 smithi008 ceph-mon[32573]: pgmap v296: 129 pgs: 4 creating+activating, 125 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-02-19T16:20:32.012 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:31 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2038660898' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T16:20:32.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:31 smithi191 ceph-mon[37203]: pgmap v296: 129 pgs: 4 creating+activating, 125 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-02-19T16:20:32.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:31 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2038660898' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T16:20:32.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.057 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2024-02-19T16:20:32.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.073 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list-id 2024-02-19T16:20:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:31 smithi156 ceph-mon[36502]: pgmap v296: 129 pgs: 4 creating+activating, 125 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-02-19T16:20:32.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:31 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2038660898' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T16:20:32.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.128 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:20:32.128 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:20:32.128 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-19T16:20:32.128 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-19T16:20:32.128 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:20:32.210 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-19T16:20:32.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-19T16:20:32.211 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:20:32.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.351 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:20:32.351 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:20:32.351 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set brx.0 up 2024-02-19T16:20:32.351 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set dev brx.0 master ceph-brx 2024-02-19T16:20:32.351 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:20:32.392 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-19T16:20:32.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T16:20:32.393 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:32.447 INFO:teuthology.orchestra.run.smithi008.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-19T16:20:32.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T16:20:32.447 DEBUG:teuthology.orchestra.run.smithi008:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:32.501 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2024-02-19T16:20:32.568 DEBUG:teuthology.orchestra.run.smithi008:> cat /proc/self/mounts | awk '{print $2}' 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/proc 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/sys 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/dev 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/security 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/shm 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/pts 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/run 2024-02-19T16:20:32.624 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/cgroup 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/pstore 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/bpf 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/ 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/selinux 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/debug 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/mqueue 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/tracing 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/hugepages 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/fuse/connections 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/config 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/run/credentials/systemd-sysctl.service 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-19T16:20:32.625 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/nfs/rpc_pipefs 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/run/user/1000 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/ceph 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/29276f718722587954a1c3b0225fb7ef600372836f521f4677222611ea52c6f5/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/27185b7c2b9e84c5e3ead4db92f5c1964a66bc4948070543ebcdfc09a7b1b4fb/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/run/user/0 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/d009715ff55f9e96a9c0d46c8853616c9c88a10a52590c437e4267ee2094533a/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/094403f0af108bbe5b0e65669ec37cfdd221a28d7c8a8c31d5e8fbcda2a99ddf/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/ab9afb57a7ba7e1d219235531caf81fa40d2623cd0d5b995c1ec1961b9b57f49/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/a089493b9c32b6252884714b803cc851706b22bfd1d91e45266ac6aaf0dc86fa/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/446e66d6b2384d6fa9e9ed68c1cfe049027ac2c6e4f83358db38f74171c968a3/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/ca5b27fa2df2b77462d021d3dadd2d719b3f665ba1f40c17cec345d6bdc9ba83/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/c4c00e9dd3f5302787dd42dce7698ddb6546aa16a02735a1cf62a1ef0112e10a/merged 2024-02-19T16:20:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/f1f6d79a0aa0c50d3900017231bcd7e1baf2f19f1bbbe59e3dd9b64778e1fa0c/merged 2024-02-19T16:20:32.627 INFO:teuthology.orchestra.run.smithi008.stdout:/run/netns 2024-02-19T16:20:32.627 INFO:teuthology.orchestra.run.smithi008.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-19T16:20:32.627 INFO:teuthology.orchestra.run.smithi008.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-19T16:20:32.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.627 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2024-02-19T16:20:32.680 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-19T16:20:32.681 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-19T16:20:32.723 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2024-02-19T16:20:32.749 DEBUG:teuthology.orchestra.run.smithi008:> cat /proc/self/mounts | awk '{print $2}' 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/proc 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/sys 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/dev 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/security 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/shm 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/pts 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/run 2024-02-19T16:20:32.805 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/cgroup 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/pstore 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/bpf 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/ 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/selinux 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/debug 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/mqueue 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/tracing 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/dev/hugepages 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/fs/fuse/connections 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/sys/kernel/config 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/run/credentials/systemd-sysctl.service 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-19T16:20:32.806 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/nfs/rpc_pipefs 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/run/user/1000 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/ceph 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/29276f718722587954a1c3b0225fb7ef600372836f521f4677222611ea52c6f5/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/27185b7c2b9e84c5e3ead4db92f5c1964a66bc4948070543ebcdfc09a7b1b4fb/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/run/user/0 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/d009715ff55f9e96a9c0d46c8853616c9c88a10a52590c437e4267ee2094533a/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/094403f0af108bbe5b0e65669ec37cfdd221a28d7c8a8c31d5e8fbcda2a99ddf/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/ab9afb57a7ba7e1d219235531caf81fa40d2623cd0d5b995c1ec1961b9b57f49/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/a089493b9c32b6252884714b803cc851706b22bfd1d91e45266ac6aaf0dc86fa/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/446e66d6b2384d6fa9e9ed68c1cfe049027ac2c6e4f83358db38f74171c968a3/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/ca5b27fa2df2b77462d021d3dadd2d719b3f665ba1f40c17cec345d6bdc9ba83/merged 2024-02-19T16:20:32.807 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/c4c00e9dd3f5302787dd42dce7698ddb6546aa16a02735a1cf62a1ef0112e10a/merged 2024-02-19T16:20:32.808 INFO:teuthology.orchestra.run.smithi008.stdout:/var/lib/containers/storage/overlay/f1f6d79a0aa0c50d3900017231bcd7e1baf2f19f1bbbe59e3dd9b64778e1fa0c/merged 2024-02-19T16:20:32.808 INFO:teuthology.orchestra.run.smithi008.stdout:/run/netns 2024-02-19T16:20:32.808 INFO:teuthology.orchestra.run.smithi008.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-19T16:20:32.808 INFO:teuthology.orchestra.run.smithi008.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-19T16:20:32.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.808 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2024-02-19T16:20:32.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:2024-02-19T16:20:32.813+0000 7f6febe3c480 -1 init, newargv = 0x5636993b34f0 newargc=15 2024-02-19T16:20:32.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:ceph-fuse[74641]: starting ceph client 2024-02-19T16:20:32.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:ceph-fuse[74641]: starting fuse 2024-02-19T16:20:32.829 INFO:teuthology.orchestra.run.smithi008.stdout:119 2024-02-19T16:20:32.829 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-02-19T16:20:32.829 DEBUG:teuthology.orchestra.run.smithi008:> sudo stdin-killer -- python3 -c ' 2024-02-19T16:20:32.829 DEBUG:teuthology.orchestra.run.smithi008:> import glob 2024-02-19T16:20:32.829 DEBUG:teuthology.orchestra.run.smithi008:> import re 2024-02-19T16:20:32.829 DEBUG:teuthology.orchestra.run.smithi008:> import os 2024-02-19T16:20:32.829 DEBUG:teuthology.orchestra.run.smithi008:> import subprocess 2024-02-19T16:20:32.829 DEBUG:teuthology.orchestra.run.smithi008:> 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> def _find_admin_socket(client_name): 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> files = glob.glob(asok_path) 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> # Given a non-glob path, it better be there 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> if "*" not in asok_path: 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> assert(len(files) == 1) 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> return files[0] 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> for f in files: 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> contents = proc_f.read() 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> if mountpoint in contents: 2024-02-19T16:20:32.830 DEBUG:teuthology.orchestra.run.smithi008:> return f 2024-02-19T16:20:32.831 DEBUG:teuthology.orchestra.run.smithi008:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-19T16:20:32.831 DEBUG:teuthology.orchestra.run.smithi008:> 2024-02-19T16:20:32.831 DEBUG:teuthology.orchestra.run.smithi008:> print(_find_admin_socket("client.admin")) 2024-02-19T16:20:32.831 DEBUG:teuthology.orchestra.run.smithi008:> ' 2024-02-19T16:20:32.952 INFO:teuthology.orchestra.run.smithi008.stdout:/var/run/ceph/ceph-client.admin.74641.asok 2024-02-19T16:20:32.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-19T16:20:32.962 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74641.asok 2024-02-19T16:20:32.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:32.963 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74641.asok status 2024-02-19T16:20:33.100 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:20:33.100 INFO:teuthology.orchestra.run.smithi008.stdout: "metadata": { 2024-02-19T16:20:33.100 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_sha1": "9f8908eed2b2d7e8c316bd5be7ed6e49024931ee", 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_version": "ceph version 19.0.0-1451-g9f8908ee (9f8908eed2b2d7e8c316bd5be7ed6e49024931ee) squid (dev)", 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "entity_id": "admin", 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": "smithi008", 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "pid": "74641", 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "root": "/" 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "dentry_count": 0, 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "dentry_pinned_count": 0, 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 14763, 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "inst": { 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "name": { 2024-02-19T16:20:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "client", 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "num": 14763 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": { 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "192.168.0.1:0", 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 1443128171 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": { 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "192.168.0.1:0", 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 1443128171 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "inst_str": "client.14763 192.168.0.1:0/1443128171", 2024-02-19T16:20:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: "addr_str": "192.168.0.1:0/1443128171", 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout: "inode_count": 1, 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_epoch": 17, 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_epoch": 91, 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_epoch_barrier": 0, 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout: "blocklisted": false, 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs" 2024-02-19T16:20:33.103 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:20:33.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:33.109 DEBUG:teuthology.orchestra.run.smithi008:> stat --file-system '--printf=%T 2024-02-19T16:20:33.109 DEBUG:teuthology.orchestra.run.smithi008:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:33.123 INFO:teuthology.orchestra.run.smithi008.stdout:fuseblk 2024-02-19T16:20:33.124 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:33.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:33.124 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:33.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:33.192 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-19T16:20:33.277 INFO:teuthology.orchestra.run.smithi008.stdout:# file: . 2024-02-19T16:20:33.277 INFO:teuthology.orchestra.run.smithi008.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-19T16:20:33.277 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:33.277 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:20:33.277 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T16:20:33.294 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 -- 172.21.15.8:0/2842099010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc101fc0 msgr2=0x7fbccc108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 --2- 172.21.15.8:0/2842099010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc101fc0 0x7fbccc108850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009ca0 tx=0x7fbcc402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 -- 172.21.15.8:0/2842099010 shutdown_connections 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 --2- 172.21.15.8:0/2842099010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc101fc0 0x7fbccc108850 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 --2- 172.21.15.8:0/2842099010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 0x7fbccc101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 --2- 172.21.15.8:0/2842099010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbccc100410 0x7fbccc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 -- 172.21.15.8:0/2842099010 >> 172.21.15.8:0/2842099010 conn(0x7fbccc0fb980 msgr2=0x7fbccc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 -- 172.21.15.8:0/2842099010 shutdown_connections 2024-02-19T16:20:33.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 -- 172.21.15.8:0/2842099010 wait complete. 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 Processor -- start 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.436+0000 7fbcd2446640 1 -- start start 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 0x7fbccc10afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 0x7fbccc109630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbccc101fc0 0x7fbccc109b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc10a300 con 0x7fbccc100410 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc10a470 con 0x7fbccc101600 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1a15d0 con 0x7fbccc101fc0 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 0x7fbccc10afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 0x7fbccc10afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55164/0 (socket says 172.21.15.8:55164) 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 -- 172.21.15.8:0/201801404 learned_addr learned my addr 172.21.15.8:0/201801404 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd09e3640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbccc101fc0 0x7fbccc109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccb7fe640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 0x7fbccc109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 -- 172.21.15.8:0/201801404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbccc101fc0 msgr2=0x7fbccc109b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbccc101fc0 0x7fbccc109b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 -- 172.21.15.8:0/201801404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 msgr2=0x7fbccc109630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 0x7fbccc109630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 -- 172.21.15.8:0/201801404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4009900 con 0x7fbccc100410 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccb7fe640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 0x7fbccc109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbccbfff640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 0x7fbccc10afa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400e9f0 tx=0x7fbcb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40096e0 con 0x7fbccc100410 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc1a1830 con 0x7fbccc100410 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb400cd80 con 0x7fbccc100410 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4010430 con 0x7fbccc100410 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.437+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc1a1d40 con 0x7fbccc100410 2024-02-19T16:20:33.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.438+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc107660 con 0x7fbccc100410 2024-02-19T16:20:33.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.439+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fbcb4003680 con 0x7fbccc100410 2024-02-19T16:20:33.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.440+0000 7fbcc97fa640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbcb80a20c0 0x7fbcb80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.440+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbcb4014070 con 0x7fbccc100410 2024-02-19T16:20:33.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.440+0000 7fbccb7fe640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbcb80a20c0 0x7fbcb80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.441+0000 7fbccb7fe640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbcb80a20c0 0x7fbcb80a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbccc10ad10 tx=0x7fbcbc005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:33.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.444+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbcb4097130 con 0x7fbccc100410 2024-02-19T16:20:33.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.627+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbccc10cf00 con 0x7fbccc100410 2024-02-19T16:20:33.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.629+0000 7fbcc97fa640 1 -- 172.21.15.8:0/201801404 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fbcb409a7f0 con 0x7fbccc100410 2024-02-19T16:20:33.630 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:33.630 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbcb80a20c0 msgr2=0x7fbcb80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbcb80a20c0 0x7fbcb80a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbccc10ad10 tx=0x7fbcbc005d90 comp rx=0 tx=0).stop 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 msgr2=0x7fbccc10afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 0x7fbccc10afa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400e9f0 tx=0x7fbcb400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 shutdown_connections 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbcb80a20c0 0x7fbcb80a4580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbccc101fc0 0x7fbccc109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc101600 0x7fbccc109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 --2- 172.21.15.8:0/201801404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbccc100410 0x7fbccc10afa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 >> 172.21.15.8:0/201801404 conn(0x7fbccc0fb980 msgr2=0x7fbccc104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:33.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 shutdown_connections 2024-02-19T16:20:33.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.631+0000 7fbcd2446640 1 -- 172.21.15.8:0/201801404 wait complete. 2024-02-19T16:20:33.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:20:33.643 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:20:33.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 -- 172.21.15.8:0/50581314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91401015f0 msgr2=0x7f9140101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 --2- 172.21.15.8:0/50581314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91401015f0 0x7f9140101a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f912c009860 tx=0x7f912c02bb70 comp rx=0 tx=0).stop 2024-02-19T16:20:33.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 -- 172.21.15.8:0/50581314 shutdown_connections 2024-02-19T16:20:33.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 --2- 172.21.15.8:0/50581314 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9140101fb0 0x7f9140108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 --2- 172.21.15.8:0/50581314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91401015f0 0x7f9140101a70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 --2- 172.21.15.8:0/50581314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9140100400 0x7f9140100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 -- 172.21.15.8:0/50581314 >> 172.21.15.8:0/50581314 conn(0x7f91400fb990 msgr2=0x7f91400fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 -- 172.21.15.8:0/50581314 shutdown_connections 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.748+0000 7f91471f6640 1 -- 172.21.15.8:0/50581314 wait complete. 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 Processor -- start 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 -- start start 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140100400 0x7f9140104390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 0x7f91401029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9140101fb0 0x7f9140102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9140103670 con 0x7f91401015f0 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401037e0 con 0x7f9140101fb0 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f91471f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401a1660 con 0x7f9140100400 2024-02-19T16:20:33.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 0x7f91401029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 0x7f91401029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55186/0 (socket says 172.21.15.8:55186) 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 -- 172.21.15.8:0/3282184399 learned_addr learned my addr 172.21.15.8:0/3282184399 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f9144f92640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140100400 0x7f9140104390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 -- 172.21.15.8:0/3282184399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140100400 msgr2=0x7f9140104390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140100400 0x7f9140104390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 -- 172.21.15.8:0/3282184399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9140101fb0 msgr2=0x7f9140102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f9145793640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9140101fb0 0x7f9140102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9140101fb0 0x7f9140102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.749+0000 7f913ffff640 1 -- 172.21.15.8:0/3282184399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009900 con 0x7f91401015f0 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.750+0000 7f913ffff640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 0x7f91401029d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f912c034b10 tx=0x7f912c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.750+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c02c510 con 0x7f91401015f0 2024-02-19T16:20:33.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.750+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c009490 con 0x7f91401015f0 2024-02-19T16:20:33.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.750+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f912c02cb50 con 0x7f91401015f0 2024-02-19T16:20:33.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.750+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c003850 con 0x7f91401015f0 2024-02-19T16:20:33.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.750+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91401a1a90 con 0x7f91401015f0 2024-02-19T16:20:33.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.751+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914010be10 con 0x7f91401015f0 2024-02-19T16:20:33.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.752+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f912c03b040 con 0x7f91401015f0 2024-02-19T16:20:33.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.753+0000 7f913dffb640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f91300a20c0 0x7f91300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:33.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.753+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f912c0efc00 con 0x7f91401015f0 2024-02-19T16:20:33.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.753+0000 7f9144f92640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f91300a20c0 0x7f91300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:33.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.753+0000 7f9144f92640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f91300a20c0 0x7f91300a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9140101450 tx=0x7f9134005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:33.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.758+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f912c0f6080 con 0x7f91401015f0 2024-02-19T16:20:33.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.942+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91401a1f20 con 0x7f91401015f0 2024-02-19T16:20:33.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.942+0000 7f913dffb640 1 -- 172.21.15.8:0/3282184399 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f912c0dae60 con 0x7f91401015f0 2024-02-19T16:20:33.943 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:20:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f91300a20c0 msgr2=0x7f91300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f91300a20c0 0x7f91300a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9140101450 tx=0x7f9134005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 msgr2=0x7f91401029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 0x7f91401029d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f912c034b10 tx=0x7f912c0045e0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 shutdown_connections 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f91300a20c0 0x7f91300a4580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9140101fb0 0x7f9140102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91401015f0 0x7f91401029d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 --2- 172.21.15.8:0/3282184399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140100400 0x7f9140104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 >> 172.21.15.8:0/3282184399 conn(0x7f91400fb990 msgr2=0x7f91400fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 shutdown_connections 2024-02-19T16:20:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:33.944+0000 7f91471f6640 1 -- 172.21.15.8:0/3282184399 wait complete. 2024-02-19T16:20:33.955 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:33 smithi008 ceph-mon[32573]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 146 B/s wr, 3 op/s 2024-02-19T16:20:33.955 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:33 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/201801404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:33.956 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:20:34.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:33 smithi191 ceph-mon[37203]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 146 B/s wr, 3 op/s 2024-02-19T16:20:34.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:33 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/201801404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 -- 172.21.15.8:0/2423920737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff901048b0 msgr2=0x7eff90106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 --2- 172.21.15.8:0/2423920737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff901048b0 0x7eff90106ca0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7eff78009900 tx=0x7eff7802bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 -- 172.21.15.8:0/2423920737 shutdown_connections 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 --2- 172.21.15.8:0/2423920737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff901071e0 0x7eff90109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 --2- 172.21.15.8:0/2423920737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff901048b0 0x7eff90106ca0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 --2- 172.21.15.8:0/2423920737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff90101f80 0x7eff90104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 -- 172.21.15.8:0/2423920737 >> 172.21.15.8:0/2423920737 conn(0x7eff900fb990 msgr2=0x7eff900fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 -- 172.21.15.8:0/2423920737 shutdown_connections 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 -- 172.21.15.8:0/2423920737 wait complete. 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 Processor -- start 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.060+0000 7eff96607640 1 -- start start 2024-02-19T16:20:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 0x7eff901975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff901048b0 0x7eff90197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff901071e0 0x7eff9019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9019f0e0 con 0x7eff90101f80 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9019f250 con 0x7eff901071e0 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9019f410 con 0x7eff901048b0 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 0x7eff901975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 0x7eff901975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55216/0 (socket says 172.21.15.8:55216) 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 -- 172.21.15.8:0/2553983915 learned_addr learned my addr 172.21.15.8:0/2553983915 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8f7fe640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff901048b0 0x7eff90197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff94ba4640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff901071e0 0x7eff9019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 -- 172.21.15.8:0/2553983915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff901048b0 msgr2=0x7eff90197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff901048b0 0x7eff90197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 -- 172.21.15.8:0/2553983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff901071e0 msgr2=0x7eff9019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff901071e0 0x7eff9019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 -- 172.21.15.8:0/2553983915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78009560 con 0x7eff90101f80 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8ffff640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 0x7eff901975f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7eff8000e9f0 tx=0x7eff8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80009720 con 0x7eff90101f80 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff8000cd80 con 0x7eff90101f80 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80010690 con 0x7eff90101f80 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.061+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9019f6f0 con 0x7eff90101f80 2024-02-19T16:20:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.062+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9019fbd0 con 0x7eff90101f80 2024-02-19T16:20:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.062+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff900fd480 con 0x7eff90101f80 2024-02-19T16:20:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.063+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7eff800046c0 con 0x7eff90101f80 2024-02-19T16:20:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.064+0000 7eff8d7fa640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7eff7c0a2140 0x7eff7c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.064+0000 7eff8f7fe640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7eff7c0a2140 0x7eff7c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.064+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7eff80014070 con 0x7eff90101f80 2024-02-19T16:20:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.065+0000 7eff8f7fe640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7eff7c0a2140 0x7eff7c0a4600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7eff78005ab0 tx=0x7eff78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.068+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7eff80096f70 con 0x7eff90101f80 2024-02-19T16:20:34.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:33 smithi156 ceph-mon[36502]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 146 B/s wr, 3 op/s 2024-02-19T16:20:34.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:33 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/201801404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:34.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.249+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff9002d000 con 0x7eff90101f80 2024-02-19T16:20:34.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.249+0000 7eff8d7fa640 1 -- 172.21.15.8:0/2553983915 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff8009a630 con 0x7eff90101f80 2024-02-19T16:20:34.250 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:20:34.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7eff7c0a2140 msgr2=0x7eff7c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7eff7c0a2140 0x7eff7c0a4600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7eff78005ab0 tx=0x7eff78005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 msgr2=0x7eff901975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 0x7eff901975f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7eff8000e9f0 tx=0x7eff8000edf0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 shutdown_connections 2024-02-19T16:20:34.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7eff7c0a2140 0x7eff7c0a4600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff901071e0 0x7eff9019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff901048b0 0x7eff90197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 --2- 172.21.15.8:0/2553983915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff90101f80 0x7eff901975f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 >> 172.21.15.8:0/2553983915 conn(0x7eff900fb990 msgr2=0x7eff90103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 shutdown_connections 2024-02-19T16:20:34.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.251+0000 7eff96607640 1 -- 172.21.15.8:0/2553983915 wait complete. 2024-02-19T16:20:34.263 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:20:34.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 -- 172.21.15.8:0/2257732341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c41026f0 msgr2=0x7f90c4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 --2- 172.21.15.8:0/2257732341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c41026f0 0x7f90c4102b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009900 tx=0x7f90ac02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 -- 172.21.15.8:0/2257732341 shutdown_connections 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 --2- 172.21.15.8:0/2257732341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90c41030b0 0x7f90c4109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 --2- 172.21.15.8:0/2257732341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c41026f0 0x7f90c4102b70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 --2- 172.21.15.8:0/2257732341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c4101500 0x7f90c4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 -- 172.21.15.8:0/2257732341 >> 172.21.15.8:0/2257732341 conn(0x7f90c40fca90 msgr2=0x7f90c40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 -- 172.21.15.8:0/2257732341 shutdown_connections 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 -- 172.21.15.8:0/2257732341 wait complete. 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.369+0000 7f90c985c640 1 Processor -- start 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 -- start start 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 0x7f90c4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90c41026f0 0x7f90c4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c41030b0 0x7f90c419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c419f080 con 0x7f90c4101500 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c419f1f0 con 0x7f90c41030b0 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c985c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c419f3b0 con 0x7f90c41026f0 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 0x7f90c4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 0x7f90c4197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55234/0 (socket says 172.21.15.8:55234) 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 -- 172.21.15.8:0/800962267 learned_addr learned my addr 172.21.15.8:0/800962267 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c27fc640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90c41026f0 0x7f90c4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c37fe640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c41030b0 0x7f90c419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 -- 172.21.15.8:0/800962267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90c41026f0 msgr2=0x7f90c4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90c41026f0 0x7f90c4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 -- 172.21.15.8:0/800962267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c41030b0 msgr2=0x7f90c419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c41030b0 0x7f90c419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 -- 172.21.15.8:0/800962267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac009560 con 0x7f90c4101500 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.370+0000 7f90c2ffd640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 0x7f90c4197590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00e9c0 tx=0x7f90bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.371+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc009700 con 0x7f90c4101500 2024-02-19T16:20:34.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.371+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90bc00cd80 con 0x7f90c4101500 2024-02-19T16:20:34.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.371+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c419f690 con 0x7f90c4101500 2024-02-19T16:20:34.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.371+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc010430 con 0x7f90c4101500 2024-02-19T16:20:34.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.371+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c419fb70 con 0x7f90c4101500 2024-02-19T16:20:34.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.372+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c41088a0 con 0x7f90c4101500 2024-02-19T16:20:34.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.373+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f90bc003680 con 0x7f90c4101500 2024-02-19T16:20:34.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.373+0000 7f90c885a640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90b00a20c0 0x7f90b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.373+0000 7f90c27fc640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90b00a20c0 0x7f90b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.373+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f90bc014070 con 0x7f90c4101500 2024-02-19T16:20:34.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.374+0000 7f90c27fc640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90b00a20c0 0x7f90b00a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009230 tx=0x7f90ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.378+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f90bc097020 con 0x7f90c4101500 2024-02-19T16:20:34.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.559+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90c402d000 con 0x7f90c4101500 2024-02-19T16:20:34.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.559+0000 7f90c885a640 1 -- 172.21.15.8:0/800962267 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90bc09a6e0 con 0x7f90c4101500 2024-02-19T16:20:34.560 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90b00a20c0 msgr2=0x7f90b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90b00a20c0 0x7f90b00a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009230 tx=0x7f90ac005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 msgr2=0x7f90c4197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 0x7f90c4197590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00e9c0 tx=0x7f90bc00ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 shutdown_connections 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90b00a20c0 0x7f90b00a4580 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c41030b0 0x7f90c419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90c41026f0 0x7f90c4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 --2- 172.21.15.8:0/800962267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90c4101500 0x7f90c4197590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 >> 172.21.15.8:0/800962267 conn(0x7f90c40fca90 msgr2=0x7f90c4105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 shutdown_connections 2024-02-19T16:20:34.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.561+0000 7f90c985c640 1 -- 172.21.15.8:0/800962267 wait complete. 2024-02-19T16:20:34.572 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:20:34.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 -- 172.21.15.8:0/993942063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe980101500 msgr2=0x7fe980101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 --2- 172.21.15.8:0/993942063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe980101500 0x7fe980101920 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe978009900 tx=0x7fe97802bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:34.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 -- 172.21.15.8:0/993942063 shutdown_connections 2024-02-19T16:20:34.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 --2- 172.21.15.8:0/993942063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801030b0 0x7fe980109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 --2- 172.21.15.8:0/993942063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9801026f0 0x7fe980102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 --2- 172.21.15.8:0/993942063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe980101500 0x7fe980101920 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 -- 172.21.15.8:0/993942063 >> 172.21.15.8:0/993942063 conn(0x7fe9800fca90 msgr2=0x7fe9800feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.677+0000 7fe98765c640 1 -- 172.21.15.8:0/993942063 shutdown_connections 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 -- 172.21.15.8:0/993942063 wait complete. 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 Processor -- start 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 -- start start 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980101500 0x7fe9801975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 0x7fe980197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801030b0 0x7fe98019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019f0d0 con 0x7fe9801026f0 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019f240 con 0x7fe980101500 2024-02-19T16:20:34.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe98765c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019f400 con 0x7fe9801030b0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe984bf7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 0x7fe980197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe984bf7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 0x7fe980197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55252/0 (socket says 172.21.15.8:55252) 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe984bf7640 1 -- 172.21.15.8:0/430050343 learned_addr learned my addr 172.21.15.8:0/430050343 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe9853f8640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980101500 0x7fe9801975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.678+0000 7fe985bf9640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801030b0 0x7fe98019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe984bf7640 1 -- 172.21.15.8:0/430050343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801030b0 msgr2=0x7fe98019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe984bf7640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801030b0 0x7fe98019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe984bf7640 1 -- 172.21.15.8:0/430050343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980101500 msgr2=0x7fe9801975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe984bf7640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980101500 0x7fe9801975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe984bf7640 1 -- 172.21.15.8:0/430050343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009560 con 0x7fe9801026f0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe984bf7640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 0x7fe980197b20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe97000d950 tx=0x7fe97000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970009a30 con 0x7fe9801026f0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe970009b90 con 0x7fe9801026f0 2024-02-19T16:20:34.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98019f6e0 con 0x7fe9801026f0 2024-02-19T16:20:34.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970004a00 con 0x7fe9801026f0 2024-02-19T16:20:34.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.679+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98019fc40 con 0x7fe9801026f0 2024-02-19T16:20:34.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.680+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9801088f0 con 0x7fe9801026f0 2024-02-19T16:20:34.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.681+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe970009d00 con 0x7fe9801026f0 2024-02-19T16:20:34.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.682+0000 7fe9767fc640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe96c0a2140 0x7fe96c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.682+0000 7fe9853f8640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe96c0a2140 0x7fe96c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.682+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe9700cdf20 con 0x7fe9801026f0 2024-02-19T16:20:34.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.682+0000 7fe9853f8640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe96c0a2140 0x7fe96c0a4600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe978005ab0 tx=0x7fe978005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.685+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe970096cd0 con 0x7fe9801026f0 2024-02-19T16:20:34.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.866+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe98010d160 con 0x7fe9801026f0 2024-02-19T16:20:34.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.867+0000 7fe9767fc640 1 -- 172.21.15.8:0/430050343 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe970017070 con 0x7fe9801026f0 2024-02-19T16:20:34.868 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.868+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe96c0a2140 msgr2=0x7fe96c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.868+0000 7fe98765c640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe96c0a2140 0x7fe96c0a4600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe978005ab0 tx=0x7fe978005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.868+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 msgr2=0x7fe980197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.868+0000 7fe98765c640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 0x7fe980197b20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe97000d950 tx=0x7fe97000de10 comp rx=0 tx=0).stop 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 shutdown_connections 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe96c0a2140 0x7fe96c0a4600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801030b0 0x7fe98019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9801026f0 0x7fe980197b20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 --2- 172.21.15.8:0/430050343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980101500 0x7fe9801975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 >> 172.21.15.8:0/430050343 conn(0x7fe9800fca90 msgr2=0x7fe980105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 shutdown_connections 2024-02-19T16:20:34.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.869+0000 7fe98765c640 1 -- 172.21.15.8:0/430050343 wait complete. 2024-02-19T16:20:34.879 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:34 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3282184399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:34.879 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:34 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2553983915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:20:34.879 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:34 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/800962267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:20:34.880 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 -- 172.21.15.8:0/3471179687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030100410 msgr2=0x7fb030100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 --2- 172.21.15.8:0/3471179687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030100410 0x7fb030100830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009900 tx=0x7fb02c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 -- 172.21.15.8:0/3471179687 shutdown_connections 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 --2- 172.21.15.8:0/3471179687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb030101fc0 0x7fb030108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 --2- 172.21.15.8:0/3471179687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb030101600 0x7fb030101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 --2- 172.21.15.8:0/3471179687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030100410 0x7fb030100830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 -- 172.21.15.8:0/3471179687 >> 172.21.15.8:0/3471179687 conn(0x7fb0300fb9a0 msgr2=0x7fb0300fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 -- 172.21.15.8:0/3471179687 shutdown_connections 2024-02-19T16:20:34.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.983+0000 7fb0379d0640 1 -- 172.21.15.8:0/3471179687 wait complete. 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 Processor -- start 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 -- start start 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb030100410 0x7fb0301964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb030101600 0x7fb030196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 0x7fb03019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019dfd0 con 0x7fb030101fc0 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019e140 con 0x7fb030100410 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb0379d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019e300 con 0x7fb030101600 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 0x7fb03019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb03576c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb030100410 0x7fb0301964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 0x7fb03019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55260/0 (socket says 172.21.15.8:55260) 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 -- 172.21.15.8:0/1509261039 learned_addr learned my addr 172.21.15.8:0/1509261039 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb034f6b640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb030101600 0x7fb030196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 -- 172.21.15.8:0/1509261039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb030101600 msgr2=0x7fb030196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb030101600 0x7fb030196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 -- 172.21.15.8:0/1509261039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb030100410 msgr2=0x7fb0301964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb030100410 0x7fb0301964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.984+0000 7fb035f6d640 1 -- 172.21.15.8:0/1509261039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009560 con 0x7fb030101fc0 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.985+0000 7fb035f6d640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 0x7fb03019da90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb02800eb10 tx=0x7fb02800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.985+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028004b20 con 0x7fb030101fc0 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.985+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb028004c80 con 0x7fb030101fc0 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.985+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03019e5e0 con 0x7fb030101fc0 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.985+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028010650 con 0x7fb030101fc0 2024-02-19T16:20:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.985+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03019eaf0 con 0x7fb030101fc0 2024-02-19T16:20:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.986+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0301077b0 con 0x7fb030101fc0 2024-02-19T16:20:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.987+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb028005020 con 0x7fb030101fc0 2024-02-19T16:20:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.987+0000 7fb0267fc640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb01c0a2140 0x7fb01c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.988+0000 7fb03576c640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb01c0a2140 0x7fb01c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.988+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb028014070 con 0x7fb030101fc0 2024-02-19T16:20:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.988+0000 7fb03576c640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb01c0a2140 0x7fb01c0a4600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005ab0 tx=0x7fb02c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:34.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:34.992+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb0280971b0 con 0x7fb030101fc0 2024-02-19T16:20:35.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:34 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3282184399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:34 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2553983915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:34 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/800962267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:34 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3282184399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:34 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2553983915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:34 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/800962267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.187+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb03002d000 con 0x7fb030101fc0 2024-02-19T16:20:35.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.187+0000 7fb0267fc640 1 -- 172.21.15.8:0/1509261039 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+579 (secure 0 0 0) 0x7fb02809a870 con 0x7fb030101fc0 2024-02-19T16:20:35.188 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3227434625 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:35.188 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/886492473 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:35.188 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/886492473 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3125167978 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1092270301 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2946360420 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3125167978 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/414318988 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2746045673 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/4121837085 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:35.189 INFO:teuthology.orchestra.run.smithi008.stderr:listed 10 entries 2024-02-19T16:20:35.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.189+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb01c0a2140 msgr2=0x7fb01c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.189+0000 7fb0379d0640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb01c0a2140 0x7fb01c0a4600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005ab0 tx=0x7fb02c005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.189+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 msgr2=0x7fb03019da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.189+0000 7fb0379d0640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 0x7fb03019da90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb02800eb10 tx=0x7fb02800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 shutdown_connections 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb01c0a2140 0x7fb01c0a4600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb030101fc0 0x7fb03019da90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb030101600 0x7fb030196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 --2- 172.21.15.8:0/1509261039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb030100410 0x7fb0301964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 >> 172.21.15.8:0/1509261039 conn(0x7fb0300fb9a0 msgr2=0x7fb030104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 shutdown_connections 2024-02-19T16:20:35.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.190+0000 7fb0379d0640 1 -- 172.21.15.8:0/1509261039 wait complete. 2024-02-19T16:20:35.201 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:20:35.201 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T16:20:35.217 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:35.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 -- 172.21.15.8:0/2095773605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 msgr2=0x7f18b4101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/2095773605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 0x7f18b4101a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009900 tx=0x7f18a402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:35.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 -- 172.21.15.8:0/2095773605 shutdown_connections 2024-02-19T16:20:35.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/2095773605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b4101fc0 0x7f18b4108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/2095773605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 0x7f18b4101a80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/2095773605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 -- 172.21.15.8:0/2095773605 >> 172.21.15.8:0/2095773605 conn(0x7f18b40fb980 msgr2=0x7f18b40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 -- 172.21.15.8:0/2095773605 shutdown_connections 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.359+0000 7f18bb8c0640 1 -- 172.21.15.8:0/2095773605 wait complete. 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 Processor -- start 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 -- start start 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 0x7f18b41029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b4101fc0 0x7f18b4102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4103610 con 0x7f18b4100410 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4103780 con 0x7f18b4101fc0 2024-02-19T16:20:35.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18bb8c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41a1720 con 0x7f18b4101600 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4104330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55278/0 (socket says 172.21.15.8:55278) 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 -- 172.21.15.8:0/3834727615 learned_addr learned my addr 172.21.15.8:0/3834727615 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b8e5b640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 0x7f18b41029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b9e5d640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b4101fc0 0x7f18b4102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 -- 172.21.15.8:0/3834727615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 msgr2=0x7f18b41029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 0x7f18b41029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 -- 172.21.15.8:0/3834727615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b4101fc0 msgr2=0x7f18b4102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b4101fc0 0x7f18b4102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.360+0000 7f18b965c640 1 -- 172.21.15.8:0/3834727615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a4009560 con 0x7f18b4100410 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.361+0000 7f18b965c640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4104330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f18b000e9c0 tx=0x7f18b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.361+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0009700 con 0x7f18b4100410 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.361+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b41a1980 con 0x7f18b4100410 2024-02-19T16:20:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.361+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18b000cd80 con 0x7f18b4100410 2024-02-19T16:20:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.361+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0010430 con 0x7f18b4100410 2024-02-19T16:20:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.361+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b41a1eb0 con 0x7f18b4100410 2024-02-19T16:20:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.362+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b410be20 con 0x7f18b4100410 2024-02-19T16:20:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.363+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f18b0003680 con 0x7f18b4100410 2024-02-19T16:20:35.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.363+0000 7f18aa7fc640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18a00a20c0 0x7f18a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.364+0000 7f18b8e5b640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18a00a20c0 0x7f18a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.364+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f18b0014070 con 0x7f18b4100410 2024-02-19T16:20:35.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.364+0000 7f18b8e5b640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18a00a20c0 0x7f18a00a4580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18a4006010 tx=0x7f18a402c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:35.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.367+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f18b00971a0 con 0x7f18b4100410 2024-02-19T16:20:35.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.550+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18b41a23d0 con 0x7f18b4100410 2024-02-19T16:20:35.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.552+0000 7f18aa7fc640 1 -- 172.21.15.8:0/3834727615 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f18b009a860 con 0x7f18b4100410 2024-02-19T16:20:35.553 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:35.553 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18a00a20c0 msgr2=0x7f18a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18a00a20c0 0x7f18a00a4580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18a4006010 tx=0x7f18a402c740 comp rx=0 tx=0).stop 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 msgr2=0x7f18b4104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4104330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f18b000e9c0 tx=0x7f18b000ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 shutdown_connections 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18a00a20c0 0x7f18a00a4580 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18b4101fc0 0x7f18b4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b4101600 0x7f18b41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 --2- 172.21.15.8:0/3834727615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18b4100410 0x7f18b4104330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 >> 172.21.15.8:0/3834727615 conn(0x7f18b40fb980 msgr2=0x7f18b40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:35.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 shutdown_connections 2024-02-19T16:20:35.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.554+0000 7f18bb8c0640 1 -- 172.21.15.8:0/3834727615 wait complete. 2024-02-19T16:20:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:20:35.566 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:20:35.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 -- 172.21.15.8:0/2278413813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe0101500 msgr2=0x7fabe0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 --2- 172.21.15.8:0/2278413813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe0101500 0x7fabe0101920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 -- 172.21.15.8:0/2278413813 shutdown_connections 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 --2- 172.21.15.8:0/2278413813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01030b0 0x7fabe0109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 --2- 172.21.15.8:0/2278413813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01026f0 0x7fabe0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 --2- 172.21.15.8:0/2278413813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe0101500 0x7fabe0101920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 -- 172.21.15.8:0/2278413813 >> 172.21.15.8:0/2278413813 conn(0x7fabe00fca90 msgr2=0x7fabe00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 -- 172.21.15.8:0/2278413813 shutdown_connections 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.670+0000 7fabe61f7640 1 -- 172.21.15.8:0/2278413813 wait complete. 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 Processor -- start 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 -- start start 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe0101500 0x7fabe01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 0x7fabe0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01030b0 0x7fabe019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe019f0d0 con 0x7fabe01026f0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe019f240 con 0x7fabe0101500 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabe61f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe019f400 con 0x7fabe01030b0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabdeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 0x7fabe0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabdeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 0x7fabe0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32810/0 (socket says 172.21.15.8:32810) 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabdeffd640 1 -- 172.21.15.8:0/4292096580 learned_addr learned my addr 172.21.15.8:0/4292096580 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.671+0000 7fabdffff640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01030b0 0x7fabe019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdf7fe640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe0101500 0x7fabe01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdeffd640 1 -- 172.21.15.8:0/4292096580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01030b0 msgr2=0x7fabe019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdeffd640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01030b0 0x7fabe019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdeffd640 1 -- 172.21.15.8:0/4292096580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe0101500 msgr2=0x7fabe01975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdeffd640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe0101500 0x7fabe01975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdeffd640 1 -- 172.21.15.8:0/4292096580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009560 con 0x7fabe01026f0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdf7fe640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe0101500 0x7fabe01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdeffd640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 0x7fabe0197b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d950 tx=0x7fabd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:35.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4009c30 con 0x7fabe01026f0 2024-02-19T16:20:35.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabd4009d90 con 0x7fabe01026f0 2024-02-19T16:20:35.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe019f6e0 con 0x7fabe01026f0 2024-02-19T16:20:35.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400f7e0 con 0x7fabe01026f0 2024-02-19T16:20:35.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.672+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe019fbf0 con 0x7fabe01026f0 2024-02-19T16:20:35.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.673+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe0108890 con 0x7fabe01026f0 2024-02-19T16:20:35.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.674+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fabd4010ad0 con 0x7fabe01026f0 2024-02-19T16:20:35.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.674+0000 7fabdcff9640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fabc80a20c0 0x7fabc80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.675+0000 7fabdf7fe640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fabc80a20c0 0x7fabc80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.675+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fabd40cdbd0 con 0x7fabe01026f0 2024-02-19T16:20:35.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.675+0000 7fabdf7fe640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fabc80a20c0 0x7fabc80a4580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fabd0006010 tx=0x7fabd0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:35.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.679+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fabd4096900 con 0x7fabe01026f0 2024-02-19T16:20:35.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.860+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabe002cf60 con 0x7fabe01026f0 2024-02-19T16:20:35.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.861+0000 7fabdcff9640 1 -- 172.21.15.8:0/4292096580 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabd4099fc0 con 0x7fabe01026f0 2024-02-19T16:20:35.862 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:20:35.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.862+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fabc80a20c0 msgr2=0x7fabc80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.862+0000 7fabe61f7640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fabc80a20c0 0x7fabc80a4580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fabd0006010 tx=0x7fabd0005be0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 msgr2=0x7fabe0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 0x7fabe0197b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d950 tx=0x7fabd400de10 comp rx=0 tx=0).stop 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 shutdown_connections 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fabc80a20c0 0x7fabc80a4580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01030b0 0x7fabe019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01026f0 0x7fabe0197b20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 --2- 172.21.15.8:0/4292096580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe0101500 0x7fabe01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 >> 172.21.15.8:0/4292096580 conn(0x7fabe00fca90 msgr2=0x7fabe0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 shutdown_connections 2024-02-19T16:20:35.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.863+0000 7fabe61f7640 1 -- 172.21.15.8:0/4292096580 wait complete. 2024-02-19T16:20:35.873 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:35 smithi008 ceph-mon[32573]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 365 B/s wr, 0 op/s 2024-02-19T16:20:35.873 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:35 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/430050343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:20:35.873 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:35 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1509261039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:20:35.873 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:35 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3834727615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:35.874 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:20:35.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.978+0000 7f85f85f7640 1 -- 172.21.15.8:0/1466560111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 msgr2=0x7f85f0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.978+0000 7f85f85f7640 1 --2- 172.21.15.8:0/1466560111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f0108830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009ca0 tx=0x7f85e802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- 172.21.15.8:0/1466560111 shutdown_connections 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 --2- 172.21.15.8:0/1466560111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f0108830 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 --2- 172.21.15.8:0/1466560111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f01015e0 0x7f85f0101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 --2- 172.21.15.8:0/1466560111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85f01003f0 0x7f85f0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- 172.21.15.8:0/1466560111 >> 172.21.15.8:0/1466560111 conn(0x7f85f00fb980 msgr2=0x7f85f00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- 172.21.15.8:0/1466560111 shutdown_connections 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- 172.21.15.8:0/1466560111 wait complete. 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 Processor -- start 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- start start 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85f01003f0 0x7f85f0197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f01015e0 0x7f85f0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f019f0f0 con 0x7f85f0101fa0 2024-02-19T16:20:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f019f260 con 0x7f85f01015e0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f85f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f019f420 con 0x7f85f01003f0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f6b94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f6b94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32832/0 (socket says 172.21.15.8:32832) 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f6b94640 1 -- 172.21.15.8:0/789564259 learned_addr learned my addr 172.21.15.8:0/789564259 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.979+0000 7f85f6393640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85f01003f0 0x7f85f0197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f5b92640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f01015e0 0x7f85f0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f6b94640 1 -- 172.21.15.8:0/789564259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85f01003f0 msgr2=0x7f85f0197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f6b94640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85f01003f0 0x7f85f0197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f6b94640 1 -- 172.21.15.8:0/789564259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f01015e0 msgr2=0x7f85f0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f6b94640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f01015e0 0x7f85f0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f6b94640 1 -- 172.21.15.8:0/789564259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8009900 con 0x7f85f0101fa0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f6b94640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f019ebb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85e8037040 tx=0x7f85e8034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80329b0 con 0x7f85f0101fa0 2024-02-19T16:20:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f019f6a0 con 0x7f85f0101fa0 2024-02-19T16:20:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e8003e50 con 0x7f85f0101fa0 2024-02-19T16:20:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8040450 con 0x7f85f0101fa0 2024-02-19T16:20:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.980+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f019fba0 con 0x7f85f0101fa0 2024-02-19T16:20:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.981+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f010be60 con 0x7f85f0101fa0 2024-02-19T16:20:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.982+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f85e800a040 con 0x7f85f0101fa0 2024-02-19T16:20:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.983+0000 7f85e77fe640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85dc0a20c0 0x7f85dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.983+0000 7f85f6393640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85dc0a20c0 0x7f85dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.983+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f85e80f0de0 con 0x7f85f0101fa0 2024-02-19T16:20:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.983+0000 7f85f6393640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85dc0a20c0 0x7f85dc0a4580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85f0101440 tx=0x7f85ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:35.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:35.987+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f85e80b9b70 con 0x7f85f0101fa0 2024-02-19T16:20:36.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:35 smithi191 ceph-mon[37203]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 365 B/s wr, 0 op/s 2024-02-19T16:20:36.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:35 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/430050343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:20:36.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:35 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1509261039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:20:36.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:35 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3834727615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:35 smithi156 ceph-mon[36502]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 365 B/s wr, 0 op/s 2024-02-19T16:20:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:35 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/430050343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:20:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:35 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1509261039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:20:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:35 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3834727615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:36.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.171+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85f019ffd0 con 0x7f85f0101fa0 2024-02-19T16:20:36.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.172+0000 7f85e77fe640 1 -- 172.21.15.8:0/789564259 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85e80a6e60 con 0x7f85f0101fa0 2024-02-19T16:20:36.173 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:20:36.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85dc0a20c0 msgr2=0x7f85dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85dc0a20c0 0x7f85dc0a4580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85f0101440 tx=0x7f85ec005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 msgr2=0x7f85f019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f019ebb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85e8037040 tx=0x7f85e8034a90 comp rx=0 tx=0).stop 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 shutdown_connections 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f85dc0a20c0 0x7f85dc0a4580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85f0101fa0 0x7f85f019ebb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85f01015e0 0x7f85f0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 --2- 172.21.15.8:0/789564259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85f01003f0 0x7f85f0197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 >> 172.21.15.8:0/789564259 conn(0x7f85f00fb980 msgr2=0x7f85f00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 shutdown_connections 2024-02-19T16:20:36.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.174+0000 7f85f85f7640 1 -- 172.21.15.8:0/789564259 wait complete. 2024-02-19T16:20:36.186 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:20:36.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/1959804951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 msgr2=0x7f5e68100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/1959804951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e68100810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/1959804951 shutdown_connections 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/1959804951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e68101fa0 0x7f5e68108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/1959804951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e681015e0 0x7f5e68101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/1959804951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e68100810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/1959804951 >> 172.21.15.8:0/1959804951 conn(0x7f5e680fb980 msgr2=0x7f5e680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/1959804951 shutdown_connections 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.290+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/1959804951 wait complete. 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 Processor -- start 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 -- start start 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e680691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e681015e0 0x7f5e6819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e68101fa0 0x7f5e681a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681a1330 con 0x7f5e681003f0 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681a14a0 con 0x7f5e681015e0 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6cd1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681a1660 con 0x7f5e68101fa0 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e680691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e680691c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32844/0 (socket says 172.21.15.8:32844) 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.291+0000 7f5e6659c640 1 -- 172.21.15.8:0/226981450 learned_addr learned my addr 172.21.15.8:0/226981450 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e66d9d640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e68101fa0 0x7f5e681a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e5ffff640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e681015e0 0x7f5e6819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6659c640 1 -- 172.21.15.8:0/226981450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e68101fa0 msgr2=0x7f5e681a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6659c640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e68101fa0 0x7f5e681a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6659c640 1 -- 172.21.15.8:0/226981450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e681015e0 msgr2=0x7f5e6819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6659c640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e681015e0 0x7f5e6819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6659c640 1 -- 172.21.15.8:0/226981450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58009560 con 0x7f5e681003f0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e66d9d640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e68101fa0 0x7f5e681a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e5ffff640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e681015e0 0x7f5e6819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6659c640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e680691c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e58005ab0 tx=0x7f5e58005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580043d0 con 0x7f5e681003f0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e681a1880 con 0x7f5e681003f0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e580052d0 con 0x7f5e681003f0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58003d40 con 0x7f5e681003f0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.292+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e681a1d90 con 0x7f5e681003f0 2024-02-19T16:20:36.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.293+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e681077e0 con 0x7f5e681003f0 2024-02-19T16:20:36.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.294+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5e58005440 con 0x7f5e681003f0 2024-02-19T16:20:36.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.295+0000 7f5e5f7fe640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5e540a20f0 0x7f5e540a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.295+0000 7f5e5ffff640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5e540a20f0 0x7f5e540a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.295+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5e580f0710 con 0x7f5e681003f0 2024-02-19T16:20:36.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.295+0000 7f5e5ffff640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5e540a20f0 0x7f5e540a45b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e68068f30 tx=0x7f5e50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:36.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.299+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5e580b9440 con 0x7f5e681003f0 2024-02-19T16:20:36.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.481+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e6802cf60 con 0x7f5e681003f0 2024-02-19T16:20:36.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.481+0000 7f5e5f7fe640 1 -- 172.21.15.8:0/226981450 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e580430a0 con 0x7f5e681003f0 2024-02-19T16:20:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:20:36.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5e540a20f0 msgr2=0x7f5e540a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5e540a20f0 0x7f5e540a45b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e68068f30 tx=0x7f5e50005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:36.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 msgr2=0x7f5e680691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e680691c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e58005ab0 tx=0x7f5e58005e50 comp rx=0 tx=0).stop 2024-02-19T16:20:36.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 shutdown_connections 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5e540a20f0 0x7f5e540a45b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e68101fa0 0x7f5e681a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e681015e0 0x7f5e6819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 --2- 172.21.15.8:0/226981450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e681003f0 0x7f5e680691c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 >> 172.21.15.8:0/226981450 conn(0x7f5e680fb980 msgr2=0x7f5e681049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 shutdown_connections 2024-02-19T16:20:36.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.483+0000 7f5e6cd1e640 1 -- 172.21.15.8:0/226981450 wait complete. 2024-02-19T16:20:36.494 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:20:36.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.598+0000 7f1eea73e640 1 -- 172.21.15.8:0/2450058354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee41071d0 msgr2=0x7f1ee4109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.598+0000 7f1eea73e640 1 --2- 172.21.15.8:0/2450058354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee41071d0 0x7f1ee4109620 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009ca0 tx=0x7f1edc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.598+0000 7f1eea73e640 1 -- 172.21.15.8:0/2450058354 shutdown_connections 2024-02-19T16:20:36.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.598+0000 7f1eea73e640 1 --2- 172.21.15.8:0/2450058354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee41071d0 0x7f1ee4109620 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.598+0000 7f1eea73e640 1 --2- 172.21.15.8:0/2450058354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 0x7f1ee4106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.598+0000 7f1eea73e640 1 --2- 172.21.15.8:0/2450058354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee4101f70 0x7f1ee4104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- 172.21.15.8:0/2450058354 >> 172.21.15.8:0/2450058354 conn(0x7f1ee40fb980 msgr2=0x7f1ee40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- 172.21.15.8:0/2450058354 shutdown_connections 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- 172.21.15.8:0/2450058354 wait complete. 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 Processor -- start 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- start start 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 0x7f1ee4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 0x7f1ee4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee41071d0 0x7f1ee419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee419f060 con 0x7f1ee4101f70 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee419f1d0 con 0x7f1ee41048a0 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.599+0000 7f1eea73e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee419f390 con 0x7f1ee41071d0 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 0x7f1ee4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 0x7f1ee4197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32852/0 (socket says 172.21.15.8:32852) 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 -- 172.21.15.8:0/938817237 learned_addr learned my addr 172.21.15.8:0/938817237 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee8cdb640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee41071d0 0x7f1ee419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee37fe640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 0x7f1ee4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 -- 172.21.15.8:0/938817237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee41071d0 msgr2=0x7f1ee419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee41071d0 0x7f1ee419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 -- 172.21.15.8:0/938817237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 msgr2=0x7f1ee4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 0x7f1ee4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 -- 172.21.15.8:0/938817237 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed4009490 con 0x7f1ee4101f70 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee37fe640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 0x7f1ee4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee3fff640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 0x7f1ee4197570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400ea60 tx=0x7f1ed400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400cde0 con 0x7f1ee4101f70 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ed40035f0 con 0x7f1ee4101f70 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc009900 con 0x7f1ee4101f70 2024-02-19T16:20:36.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4010690 con 0x7f1ee4101f70 2024-02-19T16:20:36.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.600+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee419f9f0 con 0x7f1ee4101f70 2024-02-19T16:20:36.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.601+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee40fd470 con 0x7f1ee4101f70 2024-02-19T16:20:36.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.602+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1ed4003af0 con 0x7f1ee4101f70 2024-02-19T16:20:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.603+0000 7f1ee17fa640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ecc0a20c0 0x7f1ecc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.603+0000 7f1ee37fe640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ecc0a20c0 0x7f1ecc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.603+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1ed4014070 con 0x7f1ee4101f70 2024-02-19T16:20:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.603+0000 7f1ee37fe640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ecc0a20c0 0x7f1ecc0a4580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4198a80 tx=0x7f1ed8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:36.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.607+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1ed4097180 con 0x7f1ee4101f70 2024-02-19T16:20:36.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.790+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1ee410cf00 con 0x7f1ee4101f70 2024-02-19T16:20:36.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.790+0000 7f1ee17fa640 1 -- 172.21.15.8:0/938817237 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1ed409a840 con 0x7f1ee4101f70 2024-02-19T16:20:36.791 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ecc0a20c0 msgr2=0x7f1ecc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ecc0a20c0 0x7f1ecc0a4580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4198a80 tx=0x7f1ed8005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 msgr2=0x7f1ee4197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 0x7f1ee4197570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400ea60 tx=0x7f1ed400ef20 comp rx=0 tx=0).stop 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 shutdown_connections 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ecc0a20c0 0x7f1ecc0a4580 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee41071d0 0x7f1ee419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ee41048a0 0x7f1ee4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 --2- 172.21.15.8:0/938817237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ee4101f70 0x7f1ee4197570 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 >> 172.21.15.8:0/938817237 conn(0x7f1ee40fb980 msgr2=0x7f1ee41028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:36.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 shutdown_connections 2024-02-19T16:20:36.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.792+0000 7f1eea73e640 1 -- 172.21.15.8:0/938817237 wait complete. 2024-02-19T16:20:36.804 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-19T16:20:36.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 -- 172.21.15.8:0/4147132811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1026f0 msgr2=0x7f82cc102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 --2- 172.21.15.8:0/4147132811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1026f0 0x7f82cc102b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009900 tx=0x7f82b402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:36.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 -- 172.21.15.8:0/4147132811 shutdown_connections 2024-02-19T16:20:36.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 --2- 172.21.15.8:0/4147132811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82cc1030b0 0x7f82cc109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 --2- 172.21.15.8:0/4147132811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1026f0 0x7f82cc102b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 --2- 172.21.15.8:0/4147132811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 0x7f82cc101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 -- 172.21.15.8:0/4147132811 >> 172.21.15.8:0/4147132811 conn(0x7f82cc0fca90 msgr2=0x7f82cc0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 -- 172.21.15.8:0/4147132811 shutdown_connections 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.907+0000 7f82d101c640 1 -- 172.21.15.8:0/4147132811 wait complete. 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 Processor -- start 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 -- start start 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 0x7f82cc197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82cc1026f0 0x7f82cc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 0x7f82cc19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc19f050 con 0x7f82cc1030b0 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc19f1c0 con 0x7f82cc1026f0 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82d101c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc19f380 con 0x7f82cc101500 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cb59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 0x7f82cc19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cb59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 0x7f82cc19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32872/0 (socket says 172.21.15.8:32872) 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cb59e640 1 -- 172.21.15.8:0/1365146358 learned_addr learned my addr 172.21.15.8:0/1365146358 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:36.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cad9d640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 0x7f82cc197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82c3fff640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82cc1026f0 0x7f82cc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cad9d640 1 -- 172.21.15.8:0/1365146358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82cc1026f0 msgr2=0x7f82cc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cad9d640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82cc1026f0 0x7f82cc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cad9d640 1 -- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 msgr2=0x7f82cc19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cad9d640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 0x7f82cc19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cad9d640 1 -- 172.21.15.8:0/1365146358 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009560 con 0x7f82cc101500 2024-02-19T16:20:36.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.908+0000 7f82cb59e640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 0x7f82cc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.909+0000 7f82cad9d640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 0x7f82cc197560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9c0 tx=0x7f82bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.909+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00ccc0 con 0x7f82cc101500 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.909+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc19f660 con 0x7f82cc101500 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.909+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82bc00ce20 con 0x7f82cc101500 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.909+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc010430 con 0x7f82cc101500 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.909+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc19fb20 con 0x7f82cc101500 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.910+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc1088a0 con 0x7f82cc101500 2024-02-19T16:20:36.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.911+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f82bc004000 con 0x7f82cc101500 2024-02-19T16:20:36.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.911+0000 7f82c8d99640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82b80a20c0 0x7f82b80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:36.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.911+0000 7f82c3fff640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82b80a20c0 0x7f82b80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:36.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.911+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f82bc014070 con 0x7f82cc101500 2024-02-19T16:20:36.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.912+0000 7f82c3fff640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82b80a20c0 0x7f82b80a4580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82cc198a70 tx=0x7f82b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:36.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:36.915+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f82bc097080 con 0x7f82cc101500 2024-02-19T16:20:37.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:36 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4292096580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:36 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/789564259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:36 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/226981450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:36 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4292096580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:36 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/789564259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:36 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/226981450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.112+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82cc10cf10 con 0x7f82cc101500 2024-02-19T16:20:37.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.112+0000 7f82c8d99640 1 -- 172.21.15.8:0/1365146358 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+579 (secure 0 0 0) 0x7f82bc09a740 con 0x7f82cc101500 2024-02-19T16:20:37.113 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3227434625 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:37.113 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/886492473 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:37.113 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/886492473 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:37.113 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3125167978 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:37.113 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1092270301 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:37.114 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2946360420 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:37.114 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3125167978 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:37.114 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/414318988 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:37.114 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2746045673 2024-02-20T16:12:53.197791+0000 2024-02-19T16:20:37.114 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/4121837085 2024-02-20T16:13:20.630755+0000 2024-02-19T16:20:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:listed 10 entries 2024-02-19T16:20:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82b80a20c0 msgr2=0x7f82b80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82b80a20c0 0x7f82b80a4580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82cc198a70 tx=0x7f82b4000bc0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 msgr2=0x7f82cc197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 0x7f82cc197560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9c0 tx=0x7f82bc00ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 shutdown_connections 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82b80a20c0 0x7f82b80a4580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82cc1030b0 0x7f82cc19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82cc1026f0 0x7f82cc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 --2- 172.21.15.8:0/1365146358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82cc101500 0x7f82cc197560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 >> 172.21.15.8:0/1365146358 conn(0x7f82cc0fca90 msgr2=0x7f82cc105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 shutdown_connections 2024-02-19T16:20:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.114+0000 7f82d101c640 1 -- 172.21.15.8:0/1365146358 wait complete. 2024-02-19T16:20:37.125 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:36 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4292096580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.125 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:36 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/789564259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.125 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:36 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/226981450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.126 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi008.front.sepia.ceph.com... 2024-02-19T16:20:37.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:37.127 DEBUG:teuthology.orchestra.run.smithi008:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-19T16:20:37.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:ceph-fuse[74641]: fuse finished with error 0 and tester_r 0 2024-02-19T16:20:37.154 INFO:teuthology.orchestra.run:waiting for 300 2024-02-19T16:20:37.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:20:37.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:20:37.203 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-19T16:20:37.228 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:37.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3768859787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0100490 msgr2=0x7f8cb01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3768859787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0100490 0x7f8cb01008b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009900 tx=0x7f8ca802bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:37.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3768859787 shutdown_connections 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3768859787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0102040 0x7f8cb01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3768859787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb0101680 0x7f8cb0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3768859787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0100490 0x7f8cb01008b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3768859787 >> 172.21.15.8:0/3768859787 conn(0x7f8cb00fba20 msgr2=0x7f8cb00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3768859787 shutdown_connections 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3768859787 wait complete. 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.370+0000 7f8cb83c9640 1 Processor -- start 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 -- start start 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0100490 0x7f8cb0197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 0x7f8cb0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb0102040 0x7f8cb019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019f150 con 0x7f8cb0101680 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019f2c0 con 0x7f8cb0102040 2024-02-19T16:20:37.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb83c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019f480 con 0x7f8cb0100490 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 0x7f8cb0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb6165640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0100490 0x7f8cb0197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 0x7f8cb0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32894/0 (socket says 172.21.15.8:32894) 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 -- 172.21.15.8:0/3525347843 learned_addr learned my addr 172.21.15.8:0/3525347843 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb6966640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb0102040 0x7f8cb019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 -- 172.21.15.8:0/3525347843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0100490 msgr2=0x7f8cb0197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0100490 0x7f8cb0197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 -- 172.21.15.8:0/3525347843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb0102040 msgr2=0x7f8cb019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb0102040 0x7f8cb019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 -- 172.21.15.8:0/3525347843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009560 con 0x7f8cb0101680 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb6165640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0100490 0x7f8cb0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8cb5964640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 0x7f8cb0197ba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b4a0 tx=0x7f8ca000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0010610 con 0x7f8cb0101680 2024-02-19T16:20:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.371+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0010c50 con 0x7f8cb0101680 2024-02-19T16:20:37.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.372+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb019f760 con 0x7f8cb0101680 2024-02-19T16:20:37.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.372+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0017e20 con 0x7f8cb0101680 2024-02-19T16:20:37.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.372+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb019fd40 con 0x7f8cb0101680 2024-02-19T16:20:37.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.372+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb010be90 con 0x7f8cb0101680 2024-02-19T16:20:37.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.374+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8ca0017480 con 0x7f8cb0101680 2024-02-19T16:20:37.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.374+0000 7f8ca77fe640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8c9c0a2140 0x7f8c9c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.374+0000 7f8cb6165640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8c9c0a2140 0x7f8c9c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.374+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8ca00cd900 con 0x7f8cb0101680 2024-02-19T16:20:37.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.375+0000 7f8cb6165640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8c9c0a2140 0x7f8c9c0a4600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005ab0 tx=0x7f8ca8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:37.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.378+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8ca0096690 con 0x7f8cb0101680 2024-02-19T16:20:37.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.562+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cb010c020 con 0x7f8cb0101680 2024-02-19T16:20:37.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.564+0000 7f8ca77fe640 1 -- 172.21.15.8:0/3525347843 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f8ca00092b0 con 0x7f8cb0101680 2024-02-19T16:20:37.565 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:37.565 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:37.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.565+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8c9c0a2140 msgr2=0x7f8c9c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.565+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8c9c0a2140 0x7f8c9c0a4600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005ab0 tx=0x7f8ca8005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.565+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 msgr2=0x7f8cb0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.565+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 0x7f8cb0197ba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b4a0 tx=0x7f8ca000b960 comp rx=0 tx=0).stop 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 shutdown_connections 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8c9c0a2140 0x7f8c9c0a4600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb0102040 0x7f8cb019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cb0101680 0x7f8cb0197ba0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 --2- 172.21.15.8:0/3525347843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cb0100490 0x7f8cb0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 >> 172.21.15.8:0/3525347843 conn(0x7f8cb00fba20 msgr2=0x7f8cb0069b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 shutdown_connections 2024-02-19T16:20:37.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.566+0000 7f8cb83c9640 1 -- 172.21.15.8:0/3525347843 wait complete. 2024-02-19T16:20:37.578 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 -- 172.21.15.8:0/2190098702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc1015f0 msgr2=0x7f83fc101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 --2- 172.21.15.8:0/2190098702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc1015f0 0x7f83fc101a70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009900 tx=0x7f83f002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 -- 172.21.15.8:0/2190098702 shutdown_connections 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 --2- 172.21.15.8:0/2190098702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83fc101fb0 0x7f83fc108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 --2- 172.21.15.8:0/2190098702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc1015f0 0x7f83fc101a70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 --2- 172.21.15.8:0/2190098702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc100400 0x7f83fc100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 -- 172.21.15.8:0/2190098702 >> 172.21.15.8:0/2190098702 conn(0x7f83fc0fb990 msgr2=0x7f83fc0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:37.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 -- 172.21.15.8:0/2190098702 shutdown_connections 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 -- 172.21.15.8:0/2190098702 wait complete. 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 Processor -- start 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.682+0000 7f8403d48640 1 -- start start 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 0x7f83fc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83fc1015f0 0x7f83fc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc101fb0 0x7f83fc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc19f0d0 con 0x7f83fc100400 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc19f240 con 0x7f83fc1015f0 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc19f400 con 0x7f83fc101fb0 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 0x7f83fc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 0x7f83fc1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32914/0 (socket says 172.21.15.8:32914) 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 -- 172.21.15.8:0/2259100232 learned_addr learned my addr 172.21.15.8:0/2259100232 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f84022e5640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc101fb0 0x7f83fc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 -- 172.21.15.8:0/2259100232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc101fb0 msgr2=0x7f83fc19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc101fb0 0x7f83fc19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f84012e3640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83fc1015f0 0x7f83fc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 -- 172.21.15.8:0/2259100232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83fc1015f0 msgr2=0x7f83fc197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83fc1015f0 0x7f83fc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 -- 172.21.15.8:0/2259100232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0009560 con 0x7f83fc100400 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8401ae4640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 0x7f83fc1975e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e9c0 tx=0x7f83f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40096e0 con 0x7f83fc100400 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83f400cd80 con 0x7f83fc100400 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc19f6e0 con 0x7f83fc100400 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.683+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4010430 con 0x7f83fc100400 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.684+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc19fc20 con 0x7f83fc100400 2024-02-19T16:20:37.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.684+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc10be70 con 0x7f83fc100400 2024-02-19T16:20:37.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.685+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f83f4003680 con 0x7f83fc100400 2024-02-19T16:20:37.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.686+0000 7f83faffd640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f83e40a20c0 0x7f83e40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:37.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.686+0000 7f84012e3640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f83e40a20c0 0x7f83e40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:37.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.686+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f83f4014070 con 0x7f83fc100400 2024-02-19T16:20:37.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.686+0000 7f84012e3640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f83e40a20c0 0x7f83e40a4580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83f002c6a0 tx=0x7f83f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:37.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.690+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f83f40970b0 con 0x7f83fc100400 2024-02-19T16:20:37.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.876+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83fc10c050 con 0x7f83fc100400 2024-02-19T16:20:37.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.878+0000 7f83faffd640 1 -- 172.21.15.8:0/2259100232 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f83f409a770 con 0x7f83fc100400 2024-02-19T16:20:37.879 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:37.879 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f83e40a20c0 msgr2=0x7f83e40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f83e40a20c0 0x7f83e40a4580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83f002c6a0 tx=0x7f83f0005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 msgr2=0x7f83fc1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 0x7f83fc1975e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e9c0 tx=0x7f83f400ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 shutdown_connections 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f83e40a20c0 0x7f83e40a4580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc101fb0 0x7f83fc19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83fc1015f0 0x7f83fc197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 --2- 172.21.15.8:0/2259100232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83fc100400 0x7f83fc1975e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 >> 172.21.15.8:0/2259100232 conn(0x7f83fc0fb990 msgr2=0x7f83fc0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 shutdown_connections 2024-02-19T16:20:37.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.880+0000 7f8403d48640 1 -- 172.21.15.8:0/2259100232 wait complete. 2024-02-19T16:20:37.890 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:37 smithi008 ceph-mon[32573]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-02-19T16:20:37.890 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/938817237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:20:37.891 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1365146358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:20:37.891 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3525347843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:37.892 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-19T16:20:37.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 -- 172.21.15.8:0/186286029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780ff590 msgr2=0x7f4b780ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:37.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/186286029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780ff590 0x7f4b780ffa10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009900 tx=0x7f4b6c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 -- 172.21.15.8:0/186286029 shutdown_connections 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/186286029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b780fff50 0x7f4b78109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/186286029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780ff590 0x7f4b780ffa10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/186286029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b78102b20 0x7f4b78102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 -- 172.21.15.8:0/186286029 >> 172.21.15.8:0/186286029 conn(0x7f4b780fb0f0 msgr2=0x7f4b780fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 -- 172.21.15.8:0/186286029 shutdown_connections 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.996+0000 7f4b7e22a640 1 -- 172.21.15.8:0/186286029 wait complete. 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 Processor -- start 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 -- start start 2024-02-19T16:20:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b780ff590 0x7f4b78102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 0x7f4b78100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b78102b20 0x7f4b78100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78101500 con 0x7f4b780fff50 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78101670 con 0x7f4b78102b20 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b7e22a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781a15c0 con 0x7f4b780ff590 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b76ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 0x7f4b78100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b76ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 0x7f4b78100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32932/0 (socket says 172.21.15.8:32932) 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b76ffd640 1 -- 172.21.15.8:0/1489024408 learned_addr learned my addr 172.21.15.8:0/1489024408 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b777fe640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b780ff590 0x7f4b78102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.997+0000 7f4b77fff640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b78102b20 0x7f4b78100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b76ffd640 1 -- 172.21.15.8:0/1489024408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b780ff590 msgr2=0x7f4b78102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b76ffd640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b780ff590 0x7f4b78102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b76ffd640 1 -- 172.21.15.8:0/1489024408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b78102b20 msgr2=0x7f4b78100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b76ffd640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b78102b20 0x7f4b78100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b76ffd640 1 -- 172.21.15.8:0/1489024408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c009560 con 0x7f4b780fff50 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b76ffd640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 0x7f4b78100860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c036040 tx=0x7f4b6c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0044c0 con 0x7f4b780fff50 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c0052d0 con 0x7f4b780fff50 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b781a1810 con 0x7f4b780fff50 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003e10 con 0x7f4b780fff50 2024-02-19T16:20:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.998+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b781a1d70 con 0x7f4b780fff50 2024-02-19T16:20:38.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:37.999+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b781089e0 con 0x7f4b780fff50 2024-02-19T16:20:38.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.000+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4b6c005440 con 0x7f4b780fff50 2024-02-19T16:20:38.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.001+0000 7f4b74ff9640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4b600a2140 0x7f4b600a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.001+0000 7f4b777fe640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4b600a2140 0x7f4b600a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.001+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4b6c0f0250 con 0x7f4b780fff50 2024-02-19T16:20:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.001+0000 7f4b777fe640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4b600a2140 0x7f4b600a4600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b78069800 tx=0x7f4b68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:38.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.005+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f4b6c0b8f80 con 0x7f4b780fff50 2024-02-19T16:20:38.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:37 smithi191 ceph-mon[37203]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-02-19T16:20:38.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/938817237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:20:38.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1365146358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:20:38.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3525347843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:38.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:37 smithi156 ceph-mon[36502]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-02-19T16:20:38.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/938817237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:20:38.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1365146358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:20:38.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3525347843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:38.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.228+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b7810cf80 con 0x7f4b780fff50 2024-02-19T16:20:38.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.779+0000 7f4b74ff9640 1 -- 172.21.15.8:0/1489024408 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b6c0bc640 con 0x7f4b780fff50 2024-02-19T16:20:38.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4b600a2140 msgr2=0x7f4b600a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4b600a2140 0x7f4b600a4600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b78069800 tx=0x7f4b68005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 msgr2=0x7f4b78100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 0x7f4b78100860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c036040 tx=0x7f4b6c0040c0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 shutdown_connections 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4b600a2140 0x7f4b600a4600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b78102b20 0x7f4b78100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780fff50 0x7f4b78100860 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 --2- 172.21.15.8:0/1489024408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b780ff590 0x7f4b78102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 >> 172.21.15.8:0/1489024408 conn(0x7f4b780fb0f0 msgr2=0x7f4b78105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 shutdown_connections 2024-02-19T16:20:38.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.782+0000 7f4b7e22a640 1 -- 172.21.15.8:0/1489024408 wait complete. 2024-02-19T16:20:38.796 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-19T16:20:38.796 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-19T16:20:38.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 -- 172.21.15.8:0/2860728983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c1015f0 msgr2=0x7f655c101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 --2- 172.21.15.8:0/2860728983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c1015f0 0x7f655c101a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 -- 172.21.15.8:0/2860728983 shutdown_connections 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 --2- 172.21.15.8:0/2860728983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f655c101fb0 0x7f655c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 --2- 172.21.15.8:0/2860728983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c1015f0 0x7f655c101a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 --2- 172.21.15.8:0/2860728983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c100400 0x7f655c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 -- 172.21.15.8:0/2860728983 >> 172.21.15.8:0/2860728983 conn(0x7f655c0fb990 msgr2=0x7f655c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 -- 172.21.15.8:0/2860728983 shutdown_connections 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 -- 172.21.15.8:0/2860728983 wait complete. 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 Processor -- start 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.907+0000 7f65617b6640 1 -- start start 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c100400 0x7f655c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f655c1015f0 0x7f655c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 0x7f655c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c19f0a0 con 0x7f655c101fb0 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c19f210 con 0x7f655c100400 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c19f3d0 con 0x7f655c1015f0 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 0x7f655c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 0x7f655c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32940/0 (socket says 172.21.15.8:32940) 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 -- 172.21.15.8:0/3914217536 learned_addr learned my addr 172.21.15.8:0/3914217536 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655a7fc640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f655c1015f0 0x7f655c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655affd640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c100400 0x7f655c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 -- 172.21.15.8:0/3914217536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f655c1015f0 msgr2=0x7f655c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f655c1015f0 0x7f655c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 -- 172.21.15.8:0/3914217536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c100400 msgr2=0x7f655c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c100400 0x7f655c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:38.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 -- 172.21.15.8:0/3914217536 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009840 con 0x7f655c101fb0 2024-02-19T16:20:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f655b7fe640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 0x7f655c19eb60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f654c00efd0 tx=0x7f654c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c009b40 con 0x7f655c101fb0 2024-02-19T16:20:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f654c003b70 con 0x7f655c101fb0 2024-02-19T16:20:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6544009560 con 0x7f655c101fb0 2024-02-19T16:20:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0108f0 con 0x7f655c101fb0 2024-02-19T16:20:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.908+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c19fa00 con 0x7f655c101fb0 2024-02-19T16:20:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.909+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c1077c0 con 0x7f655c101fb0 2024-02-19T16:20:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.911+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f654c003ce0 con 0x7f655c101fb0 2024-02-19T16:20:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.911+0000 7f6553fff640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f65480a2140 0x7f65480a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.911+0000 7f655affd640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f65480a2140 0x7f65480a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.911+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f654c014070 con 0x7f655c101fb0 2024-02-19T16:20:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.912+0000 7f655affd640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f65480a2140 0x7f65480a4600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f6554005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:38.915+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f654c0975f0 con 0x7f655c101fb0 2024-02-19T16:20:39.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:38 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2259100232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:39.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:38 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1489024408' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-19T16:20:39.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.140+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f655c02d000 con 0x7f655c101fb0 2024-02-19T16:20:39.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:38 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2259100232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:39.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:38 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1489024408' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-19T16:20:39.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:38 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2259100232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:39.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:38 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1489024408' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-19T16:20:39.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.785+0000 7f6553fff640 1 -- 172.21.15.8:0/3914217536 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f654c09acb0 con 0x7f655c101fb0 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f65480a2140 msgr2=0x7f65480a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f65480a2140 0x7f65480a4600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f6554005850 comp rx=0 tx=0).stop 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 msgr2=0x7f655c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 0x7f655c19eb60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f654c00efd0 tx=0x7f654c00c6a0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 shutdown_connections 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f65480a2140 0x7f65480a4600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f655c101fb0 0x7f655c19eb60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f655c1015f0 0x7f655c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 --2- 172.21.15.8:0/3914217536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c100400 0x7f655c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 >> 172.21.15.8:0/3914217536 conn(0x7f655c0fb990 msgr2=0x7f655c104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 shutdown_connections 2024-02-19T16:20:39.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.788+0000 7f65617b6640 1 -- 172.21.15.8:0/3914217536 wait complete. 2024-02-19T16:20:39.805 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 -- 172.21.15.8:0/1646059127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450100400 msgr2=0x7f1450100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 --2- 172.21.15.8:0/1646059127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450100400 0x7f1450100820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f144c009900 tx=0x7f144c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 -- 172.21.15.8:0/1646059127 shutdown_connections 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 --2- 172.21.15.8:0/1646059127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1450101fb0 0x7f1450108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 --2- 172.21.15.8:0/1646059127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14501015f0 0x7f1450101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 --2- 172.21.15.8:0/1646059127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450100400 0x7f1450100820 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 -- 172.21.15.8:0/1646059127 >> 172.21.15.8:0/1646059127 conn(0x7f14500fb990 msgr2=0x7f14500fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 -- 172.21.15.8:0/1646059127 shutdown_connections 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 -- 172.21.15.8:0/1646059127 wait complete. 2024-02-19T16:20:39.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 Processor -- start 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.916+0000 7f1457ced640 1 -- start start 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1450100400 0x7f14501964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14501015f0 0x7f1450196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 0x7f145019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145019dfc0 con 0x7f1450101fb0 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145019e130 con 0x7f14501015f0 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145019e2f0 con 0x7f1450100400 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 0x7f145019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 0x7f145019da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32946/0 (socket says 172.21.15.8:32946) 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 -- 172.21.15.8:0/3644820002 learned_addr learned my addr 172.21.15.8:0/3644820002 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1455a89640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1450100400 0x7f14501964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1455288640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14501015f0 0x7f1450196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:39.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 -- 172.21.15.8:0/3644820002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1450100400 msgr2=0x7f14501964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1450100400 0x7f14501964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 -- 172.21.15.8:0/3644820002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14501015f0 msgr2=0x7f1450196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14501015f0 0x7f1450196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 -- 172.21.15.8:0/3644820002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c009560 con 0x7f1450101fb0 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f145628a640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 0x7f145019da80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f144800eb10 tx=0x7f144800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448004b20 con 0x7f1450101fb0 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1448004c80 con 0x7f1450101fb0 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145019e5d0 con 0x7f1450101fb0 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448010650 con 0x7f1450101fb0 2024-02-19T16:20:39.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.917+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145019ea90 con 0x7f1450101fb0 2024-02-19T16:20:39.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.918+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1450107790 con 0x7f1450101fb0 2024-02-19T16:20:39.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.920+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1448005020 con 0x7f1450101fb0 2024-02-19T16:20:39.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.920+0000 7f1446ffd640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f143c0a20c0 0x7f143c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:39.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.920+0000 7f1455a89640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f143c0a20c0 0x7f143c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:39.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.920+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1448014070 con 0x7f1450101fb0 2024-02-19T16:20:39.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.921+0000 7f1455a89640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f143c0a20c0 0x7f143c0a4580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f144c005ab0 tx=0x7f144c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:39.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:39.925+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1448097260 con 0x7f1450101fb0 2024-02-19T16:20:40.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:39 smithi156 ceph-mon[36502]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 269 B/s wr, 0 op/s 2024-02-19T16:20:40.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1489024408' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-19T16:20:40.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:39 smithi156 ceph-mon[36502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:40.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3914217536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-19T16:20:40.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.148+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f145002d000 con 0x7f1450101fb0 2024-02-19T16:20:40.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:39 smithi008 ceph-mon[32573]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 269 B/s wr, 0 op/s 2024-02-19T16:20:40.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1489024408' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-19T16:20:40.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:39 smithi008 ceph-mon[32573]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:40.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3914217536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-19T16:20:40.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:39 smithi191 ceph-mon[37203]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 269 B/s wr, 0 op/s 2024-02-19T16:20:40.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1489024408' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-19T16:20:40.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:39 smithi191 ceph-mon[37203]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:40.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3914217536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-19T16:20:40.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.793+0000 7f1446ffd640 1 -- 172.21.15.8:0/3644820002 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f144809a920 con 0x7f1450101fb0 2024-02-19T16:20:40.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f143c0a20c0 msgr2=0x7f143c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:40.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f143c0a20c0 0x7f143c0a4580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f144c005ab0 tx=0x7f144c005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 msgr2=0x7f145019da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:40.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 0x7f145019da80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f144800eb10 tx=0x7f144800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 shutdown_connections 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f143c0a20c0 0x7f143c0a4580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1450101fb0 0x7f145019da80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14501015f0 0x7f1450196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 --2- 172.21.15.8:0/3644820002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1450100400 0x7f14501964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 >> 172.21.15.8:0/3644820002 conn(0x7f14500fb990 msgr2=0x7f1450104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 shutdown_connections 2024-02-19T16:20:40.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.795+0000 7f1457ced640 1 -- 172.21.15.8:0/3644820002 wait complete. 2024-02-19T16:20:40.814 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-19T16:20:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 -- 172.21.15.8:0/259548681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd641030b0 msgr2=0x7fbd64109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/259548681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd641030b0 0x7fbd64109940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009ca0 tx=0x7fbd5c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 -- 172.21.15.8:0/259548681 shutdown_connections 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/259548681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd641030b0 0x7fbd64109940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/259548681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641026f0 0x7fbd64102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/259548681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd64101500 0x7fbd64101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 -- 172.21.15.8:0/259548681 >> 172.21.15.8:0/259548681 conn(0x7fbd640fca90 msgr2=0x7fbd640feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 -- 172.21.15.8:0/259548681 shutdown_connections 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 -- 172.21.15.8:0/259548681 wait complete. 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.922+0000 7fbd6b55e640 1 Processor -- start 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 -- start start 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 0x7fbd640752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd641026f0 0x7fbd640738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641030b0 0x7fbd64073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64074580 con 0x7fbd64101500 2024-02-19T16:20:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640746f0 con 0x7fbd641030b0 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd6b55e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641a15c0 con 0x7fbd641026f0 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 0x7fbd640752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 0x7fbd640752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32960/0 (socket says 172.21.15.8:32960) 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 -- 172.21.15.8:0/3618097425 learned_addr learned my addr 172.21.15.8:0/3618097425 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd68af9640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd641026f0 0x7fbd640738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd69afb640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641030b0 0x7fbd64073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 -- 172.21.15.8:0/3618097425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd641026f0 msgr2=0x7fbd640738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd641026f0 0x7fbd640738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 -- 172.21.15.8:0/3618097425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641030b0 msgr2=0x7fbd64073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641030b0 0x7fbd64073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 -- 172.21.15.8:0/3618097425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009900 con 0x7fbd64101500 2024-02-19T16:20:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd68af9640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd641026f0 0x7fbd640738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd692fa640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 0x7fbd640752a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000e9f0 tx=0x7fbd6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.923+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60009700 con 0x7fbd64101500 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.924+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd6000cd80 con 0x7fbd64101500 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.924+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd641a1870 con 0x7fbd64101500 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.924+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60010690 con 0x7fbd64101500 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.924+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd641a1d80 con 0x7fbd64101500 2024-02-19T16:20:40.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.924+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd641088a0 con 0x7fbd64101500 2024-02-19T16:20:40.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.926+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fbd60003710 con 0x7fbd64101500 2024-02-19T16:20:40.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.926+0000 7fbd5a7fc640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd500a20c0 0x7fbd500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:40.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.926+0000 7fbd68af9640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd500a20c0 0x7fbd500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:40.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.926+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbd60014070 con 0x7fbd64101500 2024-02-19T16:20:40.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.927+0000 7fbd68af9640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd500a20c0 0x7fbd500a4580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd64075010 tx=0x7fbd54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:40.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:40.930+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbd60096f90 con 0x7fbd64101500 2024-02-19T16:20:41.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T16:20:41.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3914217536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-19T16:20:41.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-19T16:20:41.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:41.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-02-19T16:20:41.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-02-19T16:20:41.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3644820002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-19T16:20:41.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.151+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fbd6402d030 con 0x7fbd64101500 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3914217536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-02-19T16:20:41.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3644820002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-19T16:20:41.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T16:20:41.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3914217536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-19T16:20:41.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-19T16:20:41.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-02-19T16:20:41.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-02-19T16:20:41.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-02-19T16:20:41.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3644820002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-19T16:20:41.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.314+0000 7fbd5a7fc640 1 -- 172.21.15.8:0/3618097425 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fbd6009a650 con 0x7fbd64101500 2024-02-19T16:20:41.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd500a20c0 msgr2=0x7fbd500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd500a20c0 0x7fbd500a4580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd64075010 tx=0x7fbd54005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 msgr2=0x7fbd640752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 0x7fbd640752a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000e9f0 tx=0x7fbd6000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 shutdown_connections 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd500a20c0 0x7fbd500a4580 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd641030b0 0x7fbd64073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd641026f0 0x7fbd640738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 --2- 172.21.15.8:0/3618097425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd64101500 0x7fbd640752a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 >> 172.21.15.8:0/3618097425 conn(0x7fbd640fca90 msgr2=0x7fbd64105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 shutdown_connections 2024-02-19T16:20:41.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.317+0000 7fbd6b55e640 1 -- 172.21.15.8:0/3618097425 wait complete. 2024-02-19T16:20:41.330 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-02-19T16:20:41.330 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 -- 172.21.15.8:0/902514548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 msgr2=0x7fb260100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 --2- 172.21.15.8:0/902514548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb260100820 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb250009900 tx=0x7fb25002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 -- 172.21.15.8:0/902514548 shutdown_connections 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 --2- 172.21.15.8:0/902514548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb260101fb0 0x7fb260108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 --2- 172.21.15.8:0/902514548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2601015f0 0x7fb260101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 --2- 172.21.15.8:0/902514548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb260100820 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 -- 172.21.15.8:0/902514548 >> 172.21.15.8:0/902514548 conn(0x7fb2600fb990 msgr2=0x7fb2600fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 -- 172.21.15.8:0/902514548 shutdown_connections 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 -- 172.21.15.8:0/902514548 wait complete. 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.434+0000 7fb266169640 1 Processor -- start 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 -- start start 2024-02-19T16:20:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb2601975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2601015f0 0x7fb260197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb260101fb0 0x7fb26019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f0d0 con 0x7fb260100400 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f240 con 0x7fb2601015f0 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb266169640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f400 con 0x7fb260101fb0 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb2601975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb2601975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32980/0 (socket says 172.21.15.8:32980) 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 -- 172.21.15.8:0/153844572 learned_addr learned my addr 172.21.15.8:0/153844572 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25effd640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2601015f0 0x7fb260197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25ffff640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb260101fb0 0x7fb26019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 -- 172.21.15.8:0/153844572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb260101fb0 msgr2=0x7fb26019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb260101fb0 0x7fb26019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 -- 172.21.15.8:0/153844572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2601015f0 msgr2=0x7fb260197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2601015f0 0x7fb260197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 -- 172.21.15.8:0/153844572 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250009560 con 0x7fb260100400 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25f7fe640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb2601975e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb250005ab0 tx=0x7fb250005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.435+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500043d0 con 0x7fb260100400 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.436+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26019f680 con 0x7fb260100400 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.436+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2500052d0 con 0x7fb260100400 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.436+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250003d40 con 0x7fb260100400 2024-02-19T16:20:41.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.436+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26019fbe0 con 0x7fb260100400 2024-02-19T16:20:41.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.436+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26010be10 con 0x7fb260100400 2024-02-19T16:20:41.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.438+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb250005440 con 0x7fb260100400 2024-02-19T16:20:41.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.438+0000 7fb25cff9640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2480a20c0 0x7fb2480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.438+0000 7fb25effd640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2480a20c0 0x7fb2480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.438+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb2500f0000 con 0x7fb260100400 2024-02-19T16:20:41.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.439+0000 7fb25effd640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2480a20c0 0x7fb2480a4580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb260198af0 tx=0x7fb254005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:41.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.443+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb2500b8d90 con 0x7fb260100400 2024-02-19T16:20:41.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.650+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2600fdbb0 con 0x7fb2480a20c0 2024-02-19T16:20:41.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.671+0000 7fb25cff9640 1 -- 172.21.15.8:0/153844572 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb2600fdbb0 con 0x7fb2480a20c0 2024-02-19T16:20:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2480a20c0 msgr2=0x7fb2480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2480a20c0 0x7fb2480a4580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb260198af0 tx=0x7fb254005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 msgr2=0x7fb2601975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb2601975e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb250005ab0 tx=0x7fb250005e50 comp rx=0 tx=0).stop 2024-02-19T16:20:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 shutdown_connections 2024-02-19T16:20:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2480a20c0 0x7fb2480a4580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb260101fb0 0x7fb26019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2601015f0 0x7fb260197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 --2- 172.21.15.8:0/153844572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb260100400 0x7fb2601975e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 >> 172.21.15.8:0/153844572 conn(0x7fb2600fb990 msgr2=0x7fb2600fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 shutdown_connections 2024-02-19T16:20:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.673+0000 7fb266169640 1 -- 172.21.15.8:0/153844572 wait complete. 2024-02-19T16:20:41.686 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-19T16:20:41.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.797+0000 7f5873a6b640 1 -- 172.21.15.8:0/2275041390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 msgr2=0x7f586c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.797+0000 7f5873a6b640 1 --2- 172.21.15.8:0/2275041390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c102b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f585c009900 tx=0x7f585c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- 172.21.15.8:0/2275041390 shutdown_connections 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 --2- 172.21.15.8:0/2275041390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c1030b0 0x7f586c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 --2- 172.21.15.8:0/2275041390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c102b70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 --2- 172.21.15.8:0/2275041390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c101500 0x7f586c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- 172.21.15.8:0/2275041390 >> 172.21.15.8:0/2275041390 conn(0x7f586c0fca90 msgr2=0x7f586c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- 172.21.15.8:0/2275041390 shutdown_connections 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- 172.21.15.8:0/2275041390 wait complete. 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 Processor -- start 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- start start 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c101500 0x7f586c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c1030b0 0x7f586c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c19f090 con 0x7f586c1026f0 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c19f200 con 0x7f586c1030b0 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.798+0000 7f5873a6b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c19f3c0 con 0x7f586c101500 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33012/0 (socket says 172.21.15.8:33012) 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 -- 172.21.15.8:0/496434392 learned_addr learned my addr 172.21.15.8:0/496434392 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871807640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c101500 0x7f586c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5872008640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c1030b0 0x7f586c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 -- 172.21.15.8:0/496434392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c101500 msgr2=0x7f586c1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c101500 0x7f586c1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 -- 172.21.15.8:0/496434392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c1030b0 msgr2=0x7f586c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:41.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c1030b0 0x7f586c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 -- 172.21.15.8:0/496434392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c009560 con 0x7f586c1026f0 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871807640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c101500 0x7f586c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5871006640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c197ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f585c006010 tx=0x7f585c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c0043d0 con 0x7f586c1026f0 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f585c0052d0 con 0x7f586c1026f0 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c19f640 con 0x7f586c1026f0 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c003cc0 con 0x7f586c1026f0 2024-02-19T16:20:41.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.799+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c19fb50 con 0x7f586c1026f0 2024-02-19T16:20:41.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.800+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c108820 con 0x7f586c1026f0 2024-02-19T16:20:41.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.801+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f585c005440 con 0x7f586c1026f0 2024-02-19T16:20:41.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.802+0000 7f5862ffd640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f58580a20c0 0x7f58580a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:41.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.802+0000 7f5871807640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f58580a20c0 0x7f58580a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:41.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.802+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f585c0f0100 con 0x7f586c1026f0 2024-02-19T16:20:41.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.802+0000 7f5871807640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f58580a20c0 0x7f58580a4580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f586c102550 tx=0x7f5868005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:41.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:41.806+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f585c0b8e90 con 0x7f586c1026f0 2024-02-19T16:20:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.015+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f586c02d070 con 0x7f58580a20c0 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:20:41.654+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:20:41.654+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:20:41.654+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:20:41.654+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:20:41.654+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: pgmap v301: 129 pgs: 129 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-02-19T16:20:42.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3644820002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] up:active 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: Cluster is now healthy 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3618097425' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3618097425' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] up:active 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:20:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:41 smithi008 ceph-mon[32573]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:42.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.036+0000 7f5862ffd640 1 -- 172.21.15.8:0/496434392 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f586c02d070 con 0x7f58580a20c0 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f58580a20c0 msgr2=0x7f58580a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f58580a20c0 0x7f58580a4580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f586c102550 tx=0x7f5868005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 msgr2=0x7f586c197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c197ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f585c006010 tx=0x7f585c005e80 comp rx=0 tx=0).stop 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 shutdown_connections 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f58580a20c0 0x7f58580a4580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c1030b0 0x7f586c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c1026f0 0x7f586c197ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 --2- 172.21.15.8:0/496434392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c101500 0x7f586c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 >> 172.21.15.8:0/496434392 conn(0x7f586c0fca90 msgr2=0x7f586c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:42.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 shutdown_connections 2024-02-19T16:20:42.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.038+0000 7f5873a6b640 1 -- 172.21.15.8:0/496434392 wait complete. 2024-02-19T16:20:42.050 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-19T16:20:42.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: pgmap v301: 129 pgs: 129 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-02-19T16:20:42.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3644820002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-19T16:20:42.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] up:active 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: Cluster is now healthy 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3618097425' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3618097425' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] up:active 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:20:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:41 smithi156 ceph-mon[36502]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 -- 172.21.15.8:0/2980422158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674100410 msgr2=0x7f0674100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 --2- 172.21.15.8:0/2980422158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674100410 0x7f0674100830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f066c009ca0 tx=0x7f066c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 -- 172.21.15.8:0/2980422158 shutdown_connections 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 --2- 172.21.15.8:0/2980422158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0674101fc0 0x7f0674108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 --2- 172.21.15.8:0/2980422158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674101600 0x7f0674101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 --2- 172.21.15.8:0/2980422158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674100410 0x7f0674100830 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 -- 172.21.15.8:0/2980422158 >> 172.21.15.8:0/2980422158 conn(0x7f06740fb9a0 msgr2=0x7f06740fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:42.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 -- 172.21.15.8:0/2980422158 shutdown_connections 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 -- 172.21.15.8:0/2980422158 wait complete. 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 Processor -- start 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.157+0000 7f067acc3640 1 -- start start 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674100410 0x7f067406a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0674101600 0x7f067419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 0x7f067419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067419f0d0 con 0x7f0674101fc0 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067419f240 con 0x7f0674100410 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067419f400 con 0x7f0674101600 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 0x7f067419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 0x7f067419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33028/0 (socket says 172.21.15.8:33028) 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 -- 172.21.15.8:0/4254024245 learned_addr learned my addr 172.21.15.8:0/4254024245 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:42.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0673fff640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0674101600 0x7f067419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0678a5f640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674100410 0x7f067406a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 -- 172.21.15.8:0/4254024245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0674101600 msgr2=0x7f067419c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0674101600 0x7f067419c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 -- 172.21.15.8:0/4254024245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674100410 msgr2=0x7f067406a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674100410 0x7f067406a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 -- 172.21.15.8:0/4254024245 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009900 con 0x7f0674101fc0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0679260640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 0x7f067419e940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f066400e970 tx=0x7f066400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664009720 con 0x7f0674101fc0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0664005500 con 0x7f0674101fc0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067419f6e0 con 0x7f0674101fc0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.158+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664010430 con 0x7f0674101fc0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.159+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067419fbf0 con 0x7f0674101fc0 2024-02-19T16:20:42.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.160+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0674107800 con 0x7f0674101fc0 2024-02-19T16:20:42.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.161+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0664004ff0 con 0x7f0674101fc0 2024-02-19T16:20:42.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.161+0000 7f0671ffb640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06600a20c0 0x7f06600a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.161+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0664014070 con 0x7f0674101fc0 2024-02-19T16:20:42.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.161+0000 7f0678a5f640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06600a20c0 0x7f06600a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.162+0000 7f0678a5f640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06600a20c0 0x7f06600a4580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f066c02c700 tx=0x7f066c0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:42.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.166+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f0664096fb0 con 0x7f0674101fc0 2024-02-19T16:20:42.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: pgmap v301: 129 pgs: 129 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3644820002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] up:active 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: Cluster is now healthy 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3618097425' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3618097425' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] up:active 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:20:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:41 smithi191 ceph-mon[37203]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:42.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.377+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f067402d070 con 0x7f06600a20c0 2024-02-19T16:20:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.381+0000 7f0671ffb640 1 -- 172.21.15.8:0/4254024245 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f067402d070 con 0x7f06600a20c0 2024-02-19T16:20:42.383 INFO:teuthology.orchestra.run.smithi008.stdout:/volumes/qa/sv_0/b100d499-41d4-4cd8-ba5e-f82b78cc0f69 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06600a20c0 msgr2=0x7f06600a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06600a20c0 0x7f06600a4580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f066c02c700 tx=0x7f066c0326b0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 msgr2=0x7f067419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 0x7f067419e940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f066400e970 tx=0x7f066400ed70 comp rx=0 tx=0).stop 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 shutdown_connections 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06600a20c0 0x7f06600a4580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0674101fc0 0x7f067419e940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0674101600 0x7f067419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 --2- 172.21.15.8:0/4254024245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0674100410 0x7f067406a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 >> 172.21.15.8:0/4254024245 conn(0x7f06740fb9a0 msgr2=0x7f06741049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:42.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 shutdown_connections 2024-02-19T16:20:42.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.383+0000 7f067acc3640 1 -- 172.21.15.8:0/4254024245 wait complete. 2024-02-19T16:20:42.396 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-19T16:20:42.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/3239665856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4104960 msgr2=0x7f8fd4106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/3239665856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4104960 0x7f8fd4106d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009900 tx=0x7f8fc402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:42.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/3239665856 shutdown_connections 2024-02-19T16:20:42.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/3239665856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fd4107290 0x7f8fd4109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/3239665856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4104960 0x7f8fd4106d50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/3239665856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fd4102030 0x7f8fd4104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/3239665856 >> 172.21.15.8:0/3239665856 conn(0x7f8fd40fb9a0 msgr2=0x7f8fd40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/3239665856 shutdown_connections 2024-02-19T16:20:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/3239665856 wait complete. 2024-02-19T16:20:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 Processor -- start 2024-02-19T16:20:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 -- start start 2024-02-19T16:20:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fd4102030 0x7f8fd41019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fd4104960 0x7f8fd40fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.502+0000 7f8fdb3f6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 0x7f8fd4100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fdb3f6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4100c80 con 0x7f8fd4107290 2024-02-19T16:20:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fdb3f6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4100df0 con 0x7f8fd4102030 2024-02-19T16:20:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fdb3f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd41a1630 con 0x7f8fd4104960 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 0x7f8fd4100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 0x7f8fd4100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33042/0 (socket says 172.21.15.8:33042) 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd8991640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fd4104960 0x7f8fd40fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 -- 172.21.15.8:0/1369860212 learned_addr learned my addr 172.21.15.8:0/1369860212 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9192640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fd4102030 0x7f8fd41019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 -- 172.21.15.8:0/1369860212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fd4104960 msgr2=0x7f8fd40fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fd4104960 0x7f8fd40fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 -- 172.21.15.8:0/1369860212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fd4102030 msgr2=0x7f8fd41019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fd4102030 0x7f8fd41019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 -- 172.21.15.8:0/1369860212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0009490 con 0x7f8fd4107290 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fd9993640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 0x7f8fd4100520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00eb40 tx=0x7f8fcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc004b20 con 0x7f8fd4107290 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc004c80 con 0x7f8fd4107290 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc4009560 con 0x7f8fd4107290 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc010670 con 0x7f8fd4107290 2024-02-19T16:20:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.503+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd41a1c80 con 0x7f8fd4107290 2024-02-19T16:20:42.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.504+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd40fd490 con 0x7f8fd4107290 2024-02-19T16:20:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.505+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8fcc005020 con 0x7f8fd4107290 2024-02-19T16:20:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.506+0000 7f8fca7fc640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8fc00a2140 0x7f8fc00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.506+0000 7f8fd9192640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8fc00a2140 0x7f8fc00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.506+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8fcc014070 con 0x7f8fd4107290 2024-02-19T16:20:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.506+0000 7f8fd9192640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8fc00a2140 0x7f8fc00a4600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009d60 tx=0x7f8fd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:42.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.511+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8fcc0972c0 con 0x7f8fd4107290 2024-02-19T16:20:42.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.719+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fd402d070 con 0x7f8fc00a2140 2024-02-19T16:20:42.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.741+0000 7f8fca7fc640 1 -- 172.21.15.8:0/1369860212 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8fd402d070 con 0x7f8fc00a2140 2024-02-19T16:20:42.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8fc00a2140 msgr2=0x7f8fc00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8fc00a2140 0x7f8fc00a4600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009d60 tx=0x7f8fd0005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 msgr2=0x7f8fd4100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 0x7f8fd4100520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00eb40 tx=0x7f8fcc00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 shutdown_connections 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8fc00a2140 0x7f8fc00a4600 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8fd4107290 0x7f8fd4100520 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fd4104960 0x7f8fd40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 --2- 172.21.15.8:0/1369860212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fd4102030 0x7f8fd41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 >> 172.21.15.8:0/1369860212 conn(0x7f8fd40fb9a0 msgr2=0x7f8fd4102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 shutdown_connections 2024-02-19T16:20:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.743+0000 7f8fdb3f6640 1 -- 172.21.15.8:0/1369860212 wait complete. 2024-02-19T16:20:42.755 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-19T16:20:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 -- 172.21.15.8:0/3118275310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc0ffba0 msgr2=0x7f16bc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 --2- 172.21.15.8:0/3118275310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc0ffba0 0x7f16bc0fffc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009900 tx=0x7f16b402bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 -- 172.21.15.8:0/3118275310 shutdown_connections 2024-02-19T16:20:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 --2- 172.21.15.8:0/3118275310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16bc101750 0x7f16bc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 --2- 172.21.15.8:0/3118275310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc100d90 0x7f16bc101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 --2- 172.21.15.8:0/3118275310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc0ffba0 0x7f16bc0fffc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 -- 172.21.15.8:0/3118275310 >> 172.21.15.8:0/3118275310 conn(0x7f16bc0fb0f0 msgr2=0x7f16bc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 -- 172.21.15.8:0/3118275310 shutdown_connections 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.862+0000 7f16c18d4640 1 -- 172.21.15.8:0/3118275310 wait complete. 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 Processor -- start 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 -- start start 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16bc0ffba0 0x7f16bc195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc100d90 0x7f16bc195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 0x7f16bc19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19cef0 con 0x7f16bc101750 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19d060 con 0x7f16bc0ffba0 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16c18d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19d220 con 0x7f16bc100d90 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 0x7f16bc19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 0x7f16bc19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33058/0 (socket says 172.21.15.8:33058) 2024-02-19T16:20:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 -- 172.21.15.8:0/2271342760 learned_addr learned my addr 172.21.15.8:0/2271342760 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16ba7fc640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc100d90 0x7f16bc195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16baffd640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16bc0ffba0 0x7f16bc195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 -- 172.21.15.8:0/2271342760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc100d90 msgr2=0x7f16bc195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc100d90 0x7f16bc195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 -- 172.21.15.8:0/2271342760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16bc0ffba0 msgr2=0x7f16bc195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16bc0ffba0 0x7f16bc195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.863+0000 7f16bb7fe640 1 -- 172.21.15.8:0/2271342760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4009560 con 0x7f16bc101750 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.864+0000 7f16bb7fe640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 0x7f16bc19c9b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009fd0 tx=0x7f16ac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.864+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00bea0 con 0x7f16bc101750 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.864+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16ac009940 con 0x7f16bc101750 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.864+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc19d500 con 0x7f16bc101750 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.864+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0106c0 con 0x7f16bc101750 2024-02-19T16:20:42.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.864+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc19d9c0 con 0x7f16bc101750 2024-02-19T16:20:42.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.865+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc106ee0 con 0x7f16bc101750 2024-02-19T16:20:42.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.866+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f16ac004240 con 0x7f16bc101750 2024-02-19T16:20:42.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.867+0000 7f16c08d2640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f16a80a20c0 0x7f16a80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:42.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.867+0000 7f16baffd640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f16a80a20c0 0x7f16a80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:42.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.867+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f16ac0cdb50 con 0x7f16bc101750 2024-02-19T16:20:42.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.867+0000 7f16baffd640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f16a80a20c0 0x7f16a80a4580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005ab0 tx=0x7f16b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:42.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:42.871+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f16ac096990 con 0x7f16bc101750 2024-02-19T16:20:43.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:42 smithi008 ceph-mon[32573]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:43.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:42 smithi008 ceph-mon[32573]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:43.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:42 smithi156 ceph-mon[36502]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:43.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:42 smithi156 ceph-mon[36502]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:43.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.082+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f16bc02d000 con 0x7f16a80a20c0 2024-02-19T16:20:43.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.087+0000 7f16c08d2640 1 -- 172.21.15.8:0/2271342760 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f16bc02d000 con 0x7f16a80a20c0 2024-02-19T16:20:43.089 INFO:teuthology.orchestra.run.smithi008.stdout:/volumes/qa/sv_1/86171f7c-f2df-4e31-a388-f917828b18e5 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f16a80a20c0 msgr2=0x7f16a80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f16a80a20c0 0x7f16a80a4580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005ab0 tx=0x7f16b4005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 msgr2=0x7f16bc19c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 0x7f16bc19c9b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009fd0 tx=0x7f16ac00b260 comp rx=0 tx=0).stop 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 shutdown_connections 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f16a80a20c0 0x7f16a80a4580 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16bc101750 0x7f16bc19c9b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.088+0000 7f16c18d4640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc100d90 0x7f16bc195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.089+0000 7f16c18d4640 1 --2- 172.21.15.8:0/2271342760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16bc0ffba0 0x7f16bc195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.089+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 >> 172.21.15.8:0/2271342760 conn(0x7f16bc0fb0f0 msgr2=0x7f16bc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.089+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 shutdown_connections 2024-02-19T16:20:43.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.089+0000 7f16c18d4640 1 -- 172.21.15.8:0/2271342760 wait complete. 2024-02-19T16:20:43.102 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:20:43.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/2897519504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf581030d0 msgr2=0x7fdf58109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/2897519504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf581030d0 0x7fdf58109960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009ca0 tx=0x7fdf5002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/2897519504 shutdown_connections 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/2897519504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf581030d0 0x7fdf58109960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/2897519504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf58102710 0x7fdf58102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/2897519504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58101520 0x7fdf58101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/2897519504 >> 172.21.15.8:0/2897519504 conn(0x7fdf580fca90 msgr2=0x7fdf580feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/2897519504 shutdown_connections 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/2897519504 wait complete. 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 Processor -- start 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.206+0000 7fdf5cb6b640 1 -- start start 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5cb6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 0x7fdf581975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5cb6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58102710 0x7fdf58197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5cb6b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581030d0 0x7fdf5819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5cb6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819f0b0 con 0x7fdf58101520 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5cb6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819f220 con 0x7fdf58102710 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5cb6b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819f3e0 con 0x7fdf581030d0 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 0x7fdf581975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 0x7fdf581975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33076/0 (socket says 172.21.15.8:33076) 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5659c640 1 -- 172.21.15.8:0/160318948 learned_addr learned my addr 172.21.15.8:0/160318948 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581030d0 0x7fdf5819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf4ffff640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58102710 0x7fdf58197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 -- 172.21.15.8:0/160318948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58102710 msgr2=0x7fdf58197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58102710 0x7fdf58197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 -- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 msgr2=0x7fdf581975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 0x7fdf581975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 -- 172.21.15.8:0/160318948 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48009490 con 0x7fdf581030d0 2024-02-19T16:20:43.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf4ffff640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58102710 0x7fdf58197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf5659c640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 0x7fdf581975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.207+0000 7fdf56d9d640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581030d0 0x7fdf5819eb70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf5002c700 tx=0x7fdf50034a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.208+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500329b0 con 0x7fdf581030d0 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.208+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf50003e50 con 0x7fdf581030d0 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.208+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50009900 con 0x7fdf581030d0 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.208+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50004840 con 0x7fdf581030d0 2024-02-19T16:20:43.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.208+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5819f980 con 0x7fdf581030d0 2024-02-19T16:20:43.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.209+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf581087e0 con 0x7fdf581030d0 2024-02-19T16:20:43.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.210+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fdf5000a040 con 0x7fdf581030d0 2024-02-19T16:20:43.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.210+0000 7fdf4f7fe640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf440a20c0 0x7fdf440a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.210+0000 7fdf5659c640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf440a20c0 0x7fdf440a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.210+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdf500f0c00 con 0x7fdf581030d0 2024-02-19T16:20:43.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.211+0000 7fdf5659c640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf440a20c0 0x7fdf440a4580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf480058c0 tx=0x7fdf48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.215+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fdf500f7080 con 0x7fdf581030d0 2024-02-19T16:20:43.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:42 smithi191 ceph-mon[37203]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:43.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:42 smithi191 ceph-mon[37203]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:43.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.438+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf5802d000 con 0x7fdf581030d0 2024-02-19T16:20:43.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.439+0000 7fdf4f7fe640 1 -- 172.21.15.8:0/160318948 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fdf500dbe60 con 0x7fdf581030d0 2024-02-19T16:20:43.441 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:43.441 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:41.315237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:20:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:20:43.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf440a20c0 msgr2=0x7fdf440a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf440a20c0 0x7fdf440a4580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf480058c0 tx=0x7fdf48005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581030d0 msgr2=0x7fdf5819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581030d0 0x7fdf5819eb70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf5002c700 tx=0x7fdf50034a90 comp rx=0 tx=0).stop 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 shutdown_connections 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdf440a20c0 0x7fdf440a4580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581030d0 0x7fdf5819eb70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.441+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf58102710 0x7fdf58197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.442+0000 7fdf5cb6b640 1 --2- 172.21.15.8:0/160318948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf58101520 0x7fdf581975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.442+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 >> 172.21.15.8:0/160318948 conn(0x7fdf580fca90 msgr2=0x7fdf581059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.442+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 shutdown_connections 2024-02-19T16:20:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.442+0000 7fdf5cb6b640 1 -- 172.21.15.8:0/160318948 wait complete. 2024-02-19T16:20:43.455 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3980130133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68101600 msgr2=0x7fcf68101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3980130133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68101600 0x7fcf68101a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009900 tx=0x7fcf5002bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3980130133 shutdown_connections 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3980130133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf68101fc0 0x7fcf68108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3980130133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68101600 0x7fcf68101a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3980130133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf68100410 0x7fcf68100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3980130133 >> 172.21.15.8:0/3980130133 conn(0x7fcf680fb980 msgr2=0x7fcf680fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3980130133 shutdown_connections 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.557+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3980130133 wait complete. 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 Processor -- start 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 -- start start 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 0x7fcf68197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf68101600 0x7fcf68197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf68101fc0 0x7fcf6819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819f060 con 0x7fcf68100410 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819f1d0 con 0x7fcf68101600 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6ca1b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819f390 con 0x7fcf68101fc0 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 0x7fcf68197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 0x7fcf68197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33096/0 (socket says 172.21.15.8:33096) 2024-02-19T16:20:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 -- 172.21.15.8:0/3223811399 learned_addr learned my addr 172.21.15.8:0/3223811399 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf5ffff640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf68101600 0x7fcf68197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf66d9d640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf68101fc0 0x7fcf6819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 -- 172.21.15.8:0/3223811399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf68101fc0 msgr2=0x7fcf6819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf68101fc0 0x7fcf6819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 -- 172.21.15.8:0/3223811399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf68101600 msgr2=0x7fcf68197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf68101600 0x7fcf68197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.558+0000 7fcf6659c640 1 -- 172.21.15.8:0/3223811399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009560 con 0x7fcf68100410 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.559+0000 7fcf6659c640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 0x7fcf68197570 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e9c0 tx=0x7fcf5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.559+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009700 con 0x7fcf68100410 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.559+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf5800cd80 con 0x7fcf68100410 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.559+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6819f670 con 0x7fcf68100410 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.559+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58010430 con 0x7fcf68100410 2024-02-19T16:20:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.559+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6819fba0 con 0x7fcf68100410 2024-02-19T16:20:43.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.560+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680682e0 con 0x7fcf68100410 2024-02-19T16:20:43.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.561+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fcf58003680 con 0x7fcf68100410 2024-02-19T16:20:43.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.562+0000 7fcf5f7fe640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcf540a2140 0x7fcf540a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.562+0000 7fcf5ffff640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcf540a2140 0x7fcf540a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.562+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcf58014070 con 0x7fcf68100410 2024-02-19T16:20:43.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.562+0000 7fcf5ffff640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcf540a2140 0x7fcf540a4600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:43.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.566+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fcf58096f80 con 0x7fcf68100410 2024-02-19T16:20:43.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.750+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf6819ffe0 con 0x7fcf68100410 2024-02-19T16:20:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.752+0000 7fcf5f7fe640 1 -- 172.21.15.8:0/3223811399 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fcf5809a640 con 0x7fcf68100410 2024-02-19T16:20:43.754 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:43.754 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcf540a2140 msgr2=0x7fcf540a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcf540a2140 0x7fcf540a4600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009230 tx=0x7fcf50005fd0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 msgr2=0x7fcf68197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 0x7fcf68197570 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e9c0 tx=0x7fcf5800ee80 comp rx=0 tx=0).stop 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 shutdown_connections 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcf540a2140 0x7fcf540a4600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf68101fc0 0x7fcf6819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf68101600 0x7fcf68197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 --2- 172.21.15.8:0/3223811399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf68100410 0x7fcf68197570 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 >> 172.21.15.8:0/3223811399 conn(0x7fcf680fb980 msgr2=0x7fcf680fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:43.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 shutdown_connections 2024-02-19T16:20:43.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.754+0000 7fcf6ca1b640 1 -- 172.21.15.8:0/3223811399 wait complete. 2024-02-19T16:20:43.767 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-19T16:20:43.767 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-19T16:20:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 -- 172.21.15.8:0/1678910461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381071f0 msgr2=0x7f1f38109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 --2- 172.21.15.8:0/1678910461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381071f0 0x7f1f38109640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009ca0 tx=0x7f1f3002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 -- 172.21.15.8:0/1678910461 shutdown_connections 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 --2- 172.21.15.8:0/1678910461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381071f0 0x7f1f38109640 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 --2- 172.21.15.8:0/1678910461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f381048c0 0x7f1f38106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 --2- 172.21.15.8:0/1678910461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f38101f90 0x7f1f38104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 -- 172.21.15.8:0/1678910461 >> 172.21.15.8:0/1678910461 conn(0x7f1f380fb980 msgr2=0x7f1f380fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 -- 172.21.15.8:0/1678910461 shutdown_connections 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.876+0000 7f1f405bc640 1 -- 172.21.15.8:0/1678910461 wait complete. 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 Processor -- start 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 -- start start 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38101f90 0x7f1f38199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 0x7f1f38199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f381071f0 0x7f1f381a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381a1260 con 0x7f1f381048c0 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381a13d0 con 0x7f1f38101f90 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f405bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381a1590 con 0x7f1f381071f0 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3db57640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 0x7f1f38199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3db57640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 0x7f1f38199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33110/0 (socket says 172.21.15.8:33110) 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3db57640 1 -- 172.21.15.8:0/3288397714 learned_addr learned my addr 172.21.15.8:0/3288397714 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3e358640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38101f90 0x7f1f38199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3eb59640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f381071f0 0x7f1f381a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3e358640 1 -- 172.21.15.8:0/3288397714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f381071f0 msgr2=0x7f1f381a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3e358640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f381071f0 0x7f1f381a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3e358640 1 -- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 msgr2=0x7f1f38199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3e358640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 0x7f1f38199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3e358640 1 -- 172.21.15.8:0/3288397714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009900 con 0x7f1f38101f90 2024-02-19T16:20:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.877+0000 7f1f3db57640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 0x7f1f38199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.878+0000 7f1f3e358640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38101f90 0x7f1f38199770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9f0 tx=0x7f1f3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.878+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400ccc0 con 0x7f1f38101f90 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.878+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f381a1870 con 0x7f1f38101f90 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.878+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f3400ce20 con 0x7f1f38101f90 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.878+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34018920 con 0x7f1f38101f90 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.878+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f381a1d80 con 0x7f1f38101f90 2024-02-19T16:20:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.879+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f38108ce0 con 0x7f1f38101f90 2024-02-19T16:20:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.880+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f1f34018a80 con 0x7f1f38101f90 2024-02-19T16:20:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.880+0000 7f1f2f7fe640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1f240a2150 0x7f1f240a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.880+0000 7f1f3db57640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1f240a2150 0x7f1f240a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.880+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1f34014070 con 0x7f1f38101f90 2024-02-19T16:20:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.881+0000 7f1f3db57640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1f240a2150 0x7f1f240a4610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f3819ac80 tx=0x7f1f28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:43.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:43.885+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1f34053230 con 0x7f1f38101f90 2024-02-19T16:20:44.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:43 smithi008 ceph-mon[32573]: pgmap v302: 129 pgs: 129 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-19T16:20:44.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:43 smithi008 ceph-mon[32573]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:44.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:43 smithi008 ceph-mon[32573]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:44.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/160318948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:44.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3223811399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:44.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.063+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f3802d070 con 0x7f1f240a2150 2024-02-19T16:20:44.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.077+0000 7f1f2f7fe640 1 -- 172.21.15.8:0/3288397714 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424710 (secure 0 0 0) 0x7f1f3802d070 con 0x7f1f240a2150 2024-02-19T16:20:44.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:43 smithi156 ceph-mon[36502]: pgmap v302: 129 pgs: 129 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-19T16:20:44.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:43 smithi156 ceph-mon[36502]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:44.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:43 smithi156 ceph-mon[36502]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:44.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/160318948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:44.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3223811399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:44.086 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:44.089 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1f240a2150 msgr2=0x7f1f240a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1f240a2150 0x7f1f240a4610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f3819ac80 tx=0x7f1f28005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38101f90 msgr2=0x7f1f38199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38101f90 0x7f1f38199770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9f0 tx=0x7f1f3400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 shutdown_connections 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1f240a2150 0x7f1f240a4610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f381071f0 0x7f1f381a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f381048c0 0x7f1f38199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 --2- 172.21.15.8:0/3288397714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38101f90 0x7f1f38199770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 >> 172.21.15.8:0/3288397714 conn(0x7f1f380fb980 msgr2=0x7f1f38101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 shutdown_connections 2024-02-19T16:20:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.081+0000 7f1f405bc640 1 -- 172.21.15.8:0/3288397714 wait complete. 2024-02-19T16:20:44.094 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-02-19T16:20:42.716737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597148,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":293,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373480,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568344,"statfs":{"total":1151940427776,"available":1151557984256,"internally_reserved":0,"allocated":12111872,"data_stored":7664188,"data_compressed":319710,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4370,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":67,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003670"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.572038+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:20:27.572038+0000","last_peered":"2024-02-19T16:20:27.572038+0000","last_clean":"2024-02-19T16:20:27.572038+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:20:27.572038+0000","last_undegraded":"2024-02-19T16:20:27.572038+0000","last_fullsized":"2024-02-19T16:20:27.572038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544543+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:20:27.544543+0000","last_peered":"2024-02-19T16:20:27.544543+0000","last_clean":"2024-02-19T16:20:27.544543+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:20:27.544543+0000","last_undegraded":"2024-02-19T16:20:27.544543+0000","last_fullsized":"2024-02-19T16:20:27.544543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544440+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:20:27.544440+0000","last_peered":"2024-02-19T16:20:27.544440+0000","last_clean":"2024-02-19T16:20:27.544440+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:20:27.544440+0000","last_undegraded":"2024-02-19T16:20:27.544440+0000","last_fullsized":"2024-02-19T16:20:27.544440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327628+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:20:27.327628+0000","last_peered":"2024-02-19T16:20:27.327628+0000","last_clean":"2024-02-19T16:20:27.327628+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:20:27.327628+0000","last_undegraded":"2024-02-19T16:20:27.327628+0000","last_fullsized":"2024-02-19T16:20:27.327628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998322+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:20:27.998322+0000","last_peered":"2024-02-19T16:20:27.998322+0000","last_clean":"2024-02-19T16:20:27.998322+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:20:27.998322+0000","last_undegraded":"2024-02-19T16:20:27.998322+0000","last_fullsized":"2024-02-19T16:20:27.998322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545733+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:20:27.545733+0000","last_peered":"2024-02-19T16:20:27.545733+0000","last_clean":"2024-02-19T16:20:27.545733+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:20:27.545733+0000","last_undegraded":"2024-02-19T16:20:27.545733+0000","last_fullsized":"2024-02-19T16:20:27.545733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735104+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:20:27.735104+0000","last_peered":"2024-02-19T16:20:27.735104+0000","last_clean":"2024-02-19T16:20:27.735104+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:20:27.735104+0000","last_undegraded":"2024-02-19T16:20:27.735104+0000","last_fullsized":"2024-02-19T16:20:27.735104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703761+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:20:28.703761+0000","last_peered":"2024-02-19T16:20:28.703761+0000","last_clean":"2024-02-19T16:20:28.703761+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:20:28.703761+0000","last_undegraded":"2024-02-19T16:20:28.703761+0000","last_fullsized":"2024-02-19T16:20:28.703761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673644+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:20:28.673644+0000","last_peered":"2024-02-19T16:20:28.673644+0000","last_clean":"2024-02-19T16:20:28.673644+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:20:28.673644+0000","last_undegraded":"2024-02-19T16:20:28.673644+0000","last_fullsized":"2024-02-19T16:20:28.673644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.546117+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:20:27.546117+0000","last_peered":"2024-02-19T16:20:27.546117+0000","last_clean":"2024-02-19T16:20:27.546117+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:20:27.546117+0000","last_undegraded":"2024-02-19T16:20:27.546117+0000","last_fullsized":"2024-02-19T16:20:27.546117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735668+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:20:27.735668+0000","last_peered":"2024-02-19T16:20:27.735668+0000","last_clean":"2024-02-19T16:20:27.735668+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:20:27.735668+0000","last_undegraded":"2024-02-19T16:20:27.735668+0000","last_fullsized":"2024-02-19T16:20:27.735668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327425+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:20:27.327425+0000","last_peered":"2024-02-19T16:20:27.327425+0000","last_clean":"2024-02-19T16:20:27.327425+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:20:27.327425+0000","last_undegraded":"2024-02-19T16:20:27.327425+0000","last_fullsized":"2024-02-19T16:20:27.327425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672705+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:20:28.672705+0000","last_peered":"2024-02-19T16:20:28.672705+0000","last_clean":"2024-02-19T16:20:28.672705+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:20:28.672705+0000","last_undegraded":"2024-02-19T16:20:28.672705+0000","last_fullsized":"2024-02-19T16:20:28.672705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544243+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:20:27.544243+0000","last_peered":"2024-02-19T16:20:27.544243+0000","last_clean":"2024-02-19T16:20:27.544243+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:20:27.544243+0000","last_undegraded":"2024-02-19T16:20:27.544243+0000","last_fullsized":"2024-02-19T16:20:27.544243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544226+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:20:27.544226+0000","last_peered":"2024-02-19T16:20:27.544226+0000","last_clean":"2024-02-19T16:20:27.544226+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:20:27.544226+0000","last_undegraded":"2024-02-19T16:20:27.544226+0000","last_fullsized":"2024-02-19T16:20:27.544226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672970+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:20:28.672970+0000","last_peered":"2024-02-19T16:20:28.672970+0000","last_clean":"2024-02-19T16:20:28.672970+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:20:28.672970+0000","last_undegraded":"2024-02-19T16:20:28.672970+0000","last_fullsized":"2024-02-19T16:20:28.672970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571874+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:20:27.571874+0000","last_peered":"2024-02-19T16:20:27.571874+0000","last_clean":"2024-02-19T16:20:27.571874+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:20:27.571874+0000","last_undegraded":"2024-02-19T16:20:27.571874+0000","last_fullsized":"2024-02-19T16:20:27.571874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326877+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:20:27.326877+0000","last_peered":"2024-02-19T16:20:27.326877+0000","last_clean":"2024-02-19T16:20:27.326877+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:20:27.326877+0000","last_undegraded":"2024-02-19T16:20:27.326877+0000","last_fullsized":"2024-02-19T16:20:27.326877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327610+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:20:27.327610+0000","last_peered":"2024-02-19T16:20:27.327610+0000","last_clean":"2024-02-19T16:20:27.327610+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:20:27.327610+0000","last_undegraded":"2024-02-19T16:20:27.327610+0000","last_fullsized":"2024-02-19T16:20:27.327610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327374+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:20:27.327374+0000","last_peered":"2024-02-19T16:20:27.327374+0000","last_clean":"2024-02-19T16:20:27.327374+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:20:27.327374+0000","last_undegraded":"2024-02-19T16:20:27.327374+0000","last_fullsized":"2024-02-19T16:20:27.327374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.810358+0000","last_change":"2024-02-19T16:20:20.638225+0000","last_active":"2024-02-19T16:20:40.810358+0000","last_peered":"2024-02-19T16:20:40.810358+0000","last_clean":"2024-02-19T16:20:40.810358+0000","last_became_active":"2024-02-19T16:20:20.637923+0000","last_became_peered":"2024-02-19T16:20:20.637923+0000","last_unstale":"2024-02-19T16:20:40.810358+0000","last_undegraded":"2024-02-19T16:20:40.810358+0000","last_fullsized":"2024-02-19T16:20:40.810358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:42:28.514081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704202+0000","last_change":"2024-02-19T16:20:21.662054+0000","last_active":"2024-02-19T16:20:28.704202+0000","last_peered":"2024-02-19T16:20:28.704202+0000","last_clean":"2024-02-19T16:20:28.704202+0000","last_became_active":"2024-02-19T16:20:21.661721+0000","last_became_peered":"2024-02-19T16:20:21.661721+0000","last_unstale":"2024-02-19T16:20:28.704202+0000","last_undegraded":"2024-02-19T16:20:28.704202+0000","last_fullsized":"2024-02-19T16:20:28.704202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:27:13.322445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326924+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:20:27.326924+0000","last_peered":"2024-02-19T16:20:27.326924+0000","last_clean":"2024-02-19T16:20:27.326924+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:20:27.326924+0000","last_undegraded":"2024-02-19T16:20:27.326924+0000","last_fullsized":"2024-02-19T16:20:27.326924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703625+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:20:28.703625+0000","last_peered":"2024-02-19T16:20:28.703625+0000","last_clean":"2024-02-19T16:20:28.703625+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:20:28.703625+0000","last_undegraded":"2024-02-19T16:20:28.703625+0000","last_fullsized":"2024-02-19T16:20:28.703625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544532+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:20:27.544532+0000","last_peered":"2024-02-19T16:20:27.544532+0000","last_clean":"2024-02-19T16:20:27.544532+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:20:27.544532+0000","last_undegraded":"2024-02-19T16:20:27.544532+0000","last_fullsized":"2024-02-19T16:20:27.544532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327037+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:20:27.327037+0000","last_peered":"2024-02-19T16:20:27.327037+0000","last_clean":"2024-02-19T16:20:27.327037+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:20:27.327037+0000","last_undegraded":"2024-02-19T16:20:27.327037+0000","last_fullsized":"2024-02-19T16:20:27.327037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.807310+0000","last_change":"2024-02-19T16:20:20.634795+0000","last_active":"2024-02-19T16:20:40.807310+0000","last_peered":"2024-02-19T16:20:40.807310+0000","last_clean":"2024-02-19T16:20:40.807310+0000","last_became_active":"2024-02-19T16:20:20.634475+0000","last_became_peered":"2024-02-19T16:20:20.634475+0000","last_unstale":"2024-02-19T16:20:40.807310+0000","last_undegraded":"2024-02-19T16:20:40.807310+0000","last_fullsized":"2024-02-19T16:20:40.807310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:23:19.541767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673230+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:20:28.673230+0000","last_peered":"2024-02-19T16:20:28.673230+0000","last_clean":"2024-02-19T16:20:28.673230+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:20:28.673230+0000","last_undegraded":"2024-02-19T16:20:28.673230+0000","last_fullsized":"2024-02-19T16:20:28.673230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327010+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:20:27.327010+0000","last_peered":"2024-02-19T16:20:27.327010+0000","last_clean":"2024-02-19T16:20:27.327010+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:20:27.327010+0000","last_undegraded":"2024-02-19T16:20:27.327010+0000","last_fullsized":"2024-02-19T16:20:27.327010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998397+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:20:27.998397+0000","last_peered":"2024-02-19T16:20:27.998397+0000","last_clean":"2024-02-19T16:20:27.998397+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:20:27.998397+0000","last_undegraded":"2024-02-19T16:20:27.998397+0000","last_fullsized":"2024-02-19T16:20:27.998397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672970+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:20:28.672970+0000","last_peered":"2024-02-19T16:20:28.672970+0000","last_clean":"2024-02-19T16:20:28.672970+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:20:28.672970+0000","last_undegraded":"2024-02-19T16:20:28.672970+0000","last_fullsized":"2024-02-19T16:20:28.672970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327531+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:20:27.327531+0000","last_peered":"2024-02-19T16:20:27.327531+0000","last_clean":"2024-02-19T16:20:27.327531+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:20:27.327531+0000","last_undegraded":"2024-02-19T16:20:27.327531+0000","last_fullsized":"2024-02-19T16:20:27.327531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802449+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:20:39.802449+0000","last_peered":"2024-02-19T16:20:39.802449+0000","last_clean":"2024-02-19T16:20:39.802449+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:20:39.802449+0000","last_undegraded":"2024-02-19T16:20:39.802449+0000","last_fullsized":"2024-02-19T16:20:39.802449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998350+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:20:27.998350+0000","last_peered":"2024-02-19T16:20:27.998350+0000","last_clean":"2024-02-19T16:20:27.998350+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:20:27.998350+0000","last_undegraded":"2024-02-19T16:20:27.998350+0000","last_fullsized":"2024-02-19T16:20:27.998350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327742+0000","last_change":"2024-02-19T16:20:25.757507+0000","last_active":"2024-02-19T16:20:27.327742+0000","last_peered":"2024-02-19T16:20:27.327742+0000","last_clean":"2024-02-19T16:20:27.327742+0000","last_became_active":"2024-02-19T16:20:25.757101+0000","last_became_peered":"2024-02-19T16:20:25.757101+0000","last_unstale":"2024-02-19T16:20:27.327742+0000","last_undegraded":"2024-02-19T16:20:27.327742+0000","last_fullsized":"2024-02-19T16:20:27.327742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:40:32.118424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545654+0000","last_change":"2024-02-19T16:20:20.638520+0000","last_active":"2024-02-19T16:20:27.545654+0000","last_peered":"2024-02-19T16:20:27.545654+0000","last_clean":"2024-02-19T16:20:27.545654+0000","last_became_active":"2024-02-19T16:20:20.638204+0000","last_became_peered":"2024-02-19T16:20:20.638204+0000","last_unstale":"2024-02-19T16:20:27.545654+0000","last_undegraded":"2024-02-19T16:20:27.545654+0000","last_fullsized":"2024-02-19T16:20:27.545654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:42:49.275623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672807+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:20:28.672807+0000","last_peered":"2024-02-19T16:20:28.672807+0000","last_clean":"2024-02-19T16:20:28.672807+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:20:28.672807+0000","last_undegraded":"2024-02-19T16:20:28.672807+0000","last_fullsized":"2024-02-19T16:20:28.672807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571918+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:20:27.571918+0000","last_peered":"2024-02-19T16:20:27.571918+0000","last_clean":"2024-02-19T16:20:27.571918+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:20:27.571918+0000","last_undegraded":"2024-02-19T16:20:27.571918+0000","last_fullsized":"2024-02-19T16:20:27.571918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.805823+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:20:40.805823+0000","last_peered":"2024-02-19T16:20:40.805823+0000","last_clean":"2024-02-19T16:20:40.805823+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:20:40.805823+0000","last_undegraded":"2024-02-19T16:20:40.805823+0000","last_fullsized":"2024-02-19T16:20:40.805823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703876+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:20:28.703876+0000","last_peered":"2024-02-19T16:20:28.703876+0000","last_clean":"2024-02-19T16:20:28.703876+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:20:28.703876+0000","last_undegraded":"2024-02-19T16:20:28.703876+0000","last_fullsized":"2024-02-19T16:20:28.703876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544582+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:20:27.544582+0000","last_peered":"2024-02-19T16:20:27.544582+0000","last_clean":"2024-02-19T16:20:27.544582+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:20:27.544582+0000","last_undegraded":"2024-02-19T16:20:27.544582+0000","last_fullsized":"2024-02-19T16:20:27.544582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672281+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:20:28.672281+0000","last_peered":"2024-02-19T16:20:28.672281+0000","last_clean":"2024-02-19T16:20:28.672281+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:20:28.672281+0000","last_undegraded":"2024-02-19T16:20:28.672281+0000","last_fullsized":"2024-02-19T16:20:28.672281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326084+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:20:27.326084+0000","last_peered":"2024-02-19T16:20:27.326084+0000","last_clean":"2024-02-19T16:20:27.326084+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:20:27.326084+0000","last_undegraded":"2024-02-19T16:20:27.326084+0000","last_fullsized":"2024-02-19T16:20:27.326084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703928+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:20:28.703928+0000","last_peered":"2024-02-19T16:20:28.703928+0000","last_clean":"2024-02-19T16:20:28.703928+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:20:28.703928+0000","last_undegraded":"2024-02-19T16:20:28.703928+0000","last_fullsized":"2024-02-19T16:20:28.703928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.546027+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:20:27.546027+0000","last_peered":"2024-02-19T16:20:27.546027+0000","last_clean":"2024-02-19T16:20:27.546027+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:20:27.546027+0000","last_undegraded":"2024-02-19T16:20:27.546027+0000","last_fullsized":"2024-02-19T16:20:27.546027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673595+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:20:28.673595+0000","last_peered":"2024-02-19T16:20:28.673595+0000","last_clean":"2024-02-19T16:20:28.673595+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:20:28.673595+0000","last_undegraded":"2024-02-19T16:20:28.673595+0000","last_fullsized":"2024-02-19T16:20:28.673595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326201+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:20:27.326201+0000","last_peered":"2024-02-19T16:20:27.326201+0000","last_clean":"2024-02-19T16:20:27.326201+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:20:27.326201+0000","last_undegraded":"2024-02-19T16:20:27.326201+0000","last_fullsized":"2024-02-19T16:20:27.326201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802661+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:20:39.802661+0000","last_peered":"2024-02-19T16:20:39.802661+0000","last_clean":"2024-02-19T16:20:39.802661+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:20:39.802661+0000","last_undegraded":"2024-02-19T16:20:39.802661+0000","last_fullsized":"2024-02-19T16:20:39.802661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544558+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:20:27.544558+0000","last_peered":"2024-02-19T16:20:27.544558+0000","last_clean":"2024-02-19T16:20:27.544558+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:20:27.544558+0000","last_undegraded":"2024-02-19T16:20:27.544558+0000","last_fullsized":"2024-02-19T16:20:27.544558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327331+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:20:27.327331+0000","last_peered":"2024-02-19T16:20:27.327331+0000","last_clean":"2024-02-19T16:20:27.327331+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:20:27.327331+0000","last_undegraded":"2024-02-19T16:20:27.327331+0000","last_fullsized":"2024-02-19T16:20:27.327331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672459+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:20:28.672459+0000","last_peered":"2024-02-19T16:20:28.672459+0000","last_clean":"2024-02-19T16:20:28.672459+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:20:28.672459+0000","last_undegraded":"2024-02-19T16:20:28.672459+0000","last_fullsized":"2024-02-19T16:20:28.672459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673458+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:20:28.673458+0000","last_peered":"2024-02-19T16:20:28.673458+0000","last_clean":"2024-02-19T16:20:28.673458+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:20:28.673458+0000","last_undegraded":"2024-02-19T16:20:28.673458+0000","last_fullsized":"2024-02-19T16:20:28.673458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326348+0000","last_change":"2024-02-19T16:20:25.763053+0000","last_active":"2024-02-19T16:20:27.326348+0000","last_peered":"2024-02-19T16:20:27.326348+0000","last_clean":"2024-02-19T16:20:27.326348+0000","last_became_active":"2024-02-19T16:20:25.762571+0000","last_became_peered":"2024-02-19T16:20:25.762571+0000","last_unstale":"2024-02-19T16:20:27.326348+0000","last_undegraded":"2024-02-19T16:20:27.326348+0000","last_fullsized":"2024-02-19T16:20:27.326348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:16:47.050677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:37.154975+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:20:37.154975+0000","last_peered":"2024-02-19T16:20:37.154975+0000","last_clean":"2024-02-19T16:20:37.154975+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:20:37.154975+0000","last_undegraded":"2024-02-19T16:20:37.154975+0000","last_fullsized":"2024-02-19T16:20:37.154975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998655+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:20:27.998655+0000","last_peered":"2024-02-19T16:20:27.998655+0000","last_clean":"2024-02-19T16:20:27.998655+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:20:27.998655+0000","last_undegraded":"2024-02-19T16:20:27.998655+0000","last_fullsized":"2024-02-19T16:20:27.998655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673176+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:20:28.673176+0000","last_peered":"2024-02-19T16:20:28.673176+0000","last_clean":"2024-02-19T16:20:28.673176+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:20:28.673176+0000","last_undegraded":"2024-02-19T16:20:28.673176+0000","last_fullsized":"2024-02-19T16:20:28.673176+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.800466+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:20:39.800466+0000","last_peered":"2024-02-19T16:20:39.800466+0000","last_clean":"2024-02-19T16:20:39.800466+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:20:39.800466+0000","last_undegraded":"2024-02-19T16:20:39.800466+0000","last_fullsized":"2024-02-19T16:20:39.800466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672607+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:20:28.672607+0000","last_peered":"2024-02-19T16:20:28.672607+0000","last_clean":"2024-02-19T16:20:28.672607+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:20:28.672607+0000","last_undegraded":"2024-02-19T16:20:28.672607+0000","last_fullsized":"2024-02-19T16:20:28.672607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571726+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:20:27.571726+0000","last_peered":"2024-02-19T16:20:27.571726+0000","last_clean":"2024-02-19T16:20:27.571726+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:20:27.571726+0000","last_undegraded":"2024-02-19T16:20:27.571726+0000","last_fullsized":"2024-02-19T16:20:27.571726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.800891+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:20:39.800891+0000","last_peered":"2024-02-19T16:20:39.800891+0000","last_clean":"2024-02-19T16:20:39.800891+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:20:39.800891+0000","last_undegraded":"2024-02-19T16:20:39.800891+0000","last_fullsized":"2024-02-19T16:20:39.800891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735335+0000","last_change":"2024-02-19T16:20:21.660468+0000","last_active":"2024-02-19T16:20:27.735335+0000","last_peered":"2024-02-19T16:20:27.735335+0000","last_clean":"2024-02-19T16:20:27.735335+0000","last_became_active":"2024-02-19T16:20:21.659988+0000","last_became_peered":"2024-02-19T16:20:21.659988+0000","last_unstale":"2024-02-19T16:20:27.735335+0000","last_undegraded":"2024-02-19T16:20:27.735335+0000","last_fullsized":"2024-02-19T16:20:27.735335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:57:08.316143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326598+0000","last_change":"2024-02-19T16:20:25.756660+0000","last_active":"2024-02-19T16:20:27.326598+0000","last_peered":"2024-02-19T16:20:27.326598+0000","last_clean":"2024-02-19T16:20:27.326598+0000","last_became_active":"2024-02-19T16:20:25.755644+0000","last_became_peered":"2024-02-19T16:20:25.755644+0000","last_unstale":"2024-02-19T16:20:27.326598+0000","last_undegraded":"2024-02-19T16:20:27.326598+0000","last_fullsized":"2024-02-19T16:20:27.326598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:52:07.717537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.808440+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:20:40.808440+0000","last_peered":"2024-02-19T16:20:40.808440+0000","last_clean":"2024-02-19T16:20:40.808440+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:20:40.808440+0000","last_undegraded":"2024-02-19T16:20:40.808440+0000","last_fullsized":"2024-02-19T16:20:40.808440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735086+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:20:27.735086+0000","last_peered":"2024-02-19T16:20:27.735086+0000","last_clean":"2024-02-19T16:20:27.735086+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:20:27.735086+0000","last_undegraded":"2024-02-19T16:20:27.735086+0000","last_fullsized":"2024-02-19T16:20:27.735086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672956+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:20:28.672956+0000","last_peered":"2024-02-19T16:20:28.672956+0000","last_clean":"2024-02-19T16:20:28.672956+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:20:28.672956+0000","last_undegraded":"2024-02-19T16:20:28.672956+0000","last_fullsized":"2024-02-19T16:20:28.672956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.808634+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:20:40.808634+0000","last_peered":"2024-02-19T16:20:40.808634+0000","last_clean":"2024-02-19T16:20:40.808634+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:20:40.808634+0000","last_undegraded":"2024-02-19T16:20:40.808634+0000","last_fullsized":"2024-02-19T16:20:40.808634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672674+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:20:28.672674+0000","last_peered":"2024-02-19T16:20:28.672674+0000","last_clean":"2024-02-19T16:20:28.672674+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:20:28.672674+0000","last_undegraded":"2024-02-19T16:20:28.672674+0000","last_fullsized":"2024-02-19T16:20:28.672674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672708+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:20:28.672708+0000","last_peered":"2024-02-19T16:20:28.672708+0000","last_clean":"2024-02-19T16:20:28.672708+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:20:28.672708+0000","last_undegraded":"2024-02-19T16:20:28.672708+0000","last_fullsized":"2024-02-19T16:20:28.672708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998660+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:20:27.998660+0000","last_peered":"2024-02-19T16:20:27.998660+0000","last_clean":"2024-02-19T16:20:27.998660+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:20:27.998660+0000","last_undegraded":"2024-02-19T16:20:27.998660+0000","last_fullsized":"2024-02-19T16:20:27.998660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571854+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:20:27.571854+0000","last_peered":"2024-02-19T16:20:27.571854+0000","last_clean":"2024-02-19T16:20:27.571854+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:20:27.571854+0000","last_undegraded":"2024-02-19T16:20:27.571854+0000","last_fullsized":"2024-02-19T16:20:27.571854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998348+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:20:27.998348+0000","last_peered":"2024-02-19T16:20:27.998348+0000","last_clean":"2024-02-19T16:20:27.998348+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:20:27.998348+0000","last_undegraded":"2024-02-19T16:20:27.998348+0000","last_fullsized":"2024-02-19T16:20:27.998348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326398+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:20:27.326398+0000","last_peered":"2024-02-19T16:20:27.326398+0000","last_clean":"2024-02-19T16:20:27.326398+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:20:27.326398+0000","last_undegraded":"2024-02-19T16:20:27.326398+0000","last_fullsized":"2024-02-19T16:20:27.326398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545899+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:20:27.545899+0000","last_peered":"2024-02-19T16:20:27.545899+0000","last_clean":"2024-02-19T16:20:27.545899+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:20:27.545899+0000","last_undegraded":"2024-02-19T16:20:27.545899+0000","last_fullsized":"2024-02-19T16:20:27.545899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672805+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:20:28.672805+0000","last_peered":"2024-02-19T16:20:28.672805+0000","last_clean":"2024-02-19T16:20:28.672805+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:20:28.672805+0000","last_undegraded":"2024-02-19T16:20:28.672805+0000","last_fullsized":"2024-02-19T16:20:28.672805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735253+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:20:27.735253+0000","last_peered":"2024-02-19T16:20:27.735253+0000","last_clean":"2024-02-19T16:20:27.735253+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:20:27.735253+0000","last_undegraded":"2024-02-19T16:20:27.735253+0000","last_fullsized":"2024-02-19T16:20:27.735253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545804+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:20:27.545804+0000","last_peered":"2024-02-19T16:20:27.545804+0000","last_clean":"2024-02-19T16:20:27.545804+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:20:27.545804+0000","last_undegraded":"2024-02-19T16:20:27.545804+0000","last_fullsized":"2024-02-19T16:20:27.545804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672900+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:20:28.672900+0000","last_peered":"2024-02-19T16:20:28.672900+0000","last_clean":"2024-02-19T16:20:28.672900+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:20:28.672900+0000","last_undegraded":"2024-02-19T16:20:28.672900+0000","last_fullsized":"2024-02-19T16:20:28.672900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672569+0000","last_change":"2024-02-19T16:20:21.671091+0000","last_active":"2024-02-19T16:20:28.672569+0000","last_peered":"2024-02-19T16:20:28.672569+0000","last_clean":"2024-02-19T16:20:28.672569+0000","last_became_active":"2024-02-19T16:20:21.670762+0000","last_became_peered":"2024-02-19T16:20:21.670762+0000","last_unstale":"2024-02-19T16:20:28.672569+0000","last_undegraded":"2024-02-19T16:20:28.672569+0000","last_fullsized":"2024-02-19T16:20:28.672569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:14:35.707812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.809032+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:20:40.809032+0000","last_peered":"2024-02-19T16:20:40.809032+0000","last_clean":"2024-02-19T16:20:40.809032+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:20:40.809032+0000","last_undegraded":"2024-02-19T16:20:40.809032+0000","last_fullsized":"2024-02-19T16:20:40.809032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735547+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:20:27.735547+0000","last_peered":"2024-02-19T16:20:27.735547+0000","last_clean":"2024-02-19T16:20:27.735547+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:20:27.735547+0000","last_undegraded":"2024-02-19T16:20:27.735547+0000","last_fullsized":"2024-02-19T16:20:27.735547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703942+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:20:28.703942+0000","last_peered":"2024-02-19T16:20:28.703942+0000","last_clean":"2024-02-19T16:20:28.703942+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:20:28.703942+0000","last_undegraded":"2024-02-19T16:20:28.703942+0000","last_fullsized":"2024-02-19T16:20:28.703942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.801843+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:20:39.801843+0000","last_peered":"2024-02-19T16:20:39.801843+0000","last_clean":"2024-02-19T16:20:39.801843+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:20:39.801843+0000","last_undegraded":"2024-02-19T16:20:39.801843+0000","last_fullsized":"2024-02-19T16:20:39.801843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.734959+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:20:27.734959+0000","last_peered":"2024-02-19T16:20:27.734959+0000","last_clean":"2024-02-19T16:20:27.734959+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:20:27.734959+0000","last_undegraded":"2024-02-19T16:20:27.734959+0000","last_fullsized":"2024-02-19T16:20:27.734959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673126+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:20:28.673126+0000","last_peered":"2024-02-19T16:20:28.673126+0000","last_clean":"2024-02-19T16:20:28.673126+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:20:28.673126+0000","last_undegraded":"2024-02-19T16:20:28.673126+0000","last_fullsized":"2024-02-19T16:20:28.673126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672385+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:20:28.672385+0000","last_peered":"2024-02-19T16:20:28.672385+0000","last_clean":"2024-02-19T16:20:28.672385+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:20:28.672385+0000","last_undegraded":"2024-02-19T16:20:28.672385+0000","last_fullsized":"2024-02-19T16:20:28.672385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.806469+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:20:40.806469+0000","last_peered":"2024-02-19T16:20:40.806469+0000","last_clean":"2024-02-19T16:20:40.806469+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:20:40.806469+0000","last_undegraded":"2024-02-19T16:20:40.806469+0000","last_fullsized":"2024-02-19T16:20:40.806469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571693+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:20:27.571693+0000","last_peered":"2024-02-19T16:20:27.571693+0000","last_clean":"2024-02-19T16:20:27.571693+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:20:27.571693+0000","last_undegraded":"2024-02-19T16:20:27.571693+0000","last_fullsized":"2024-02-19T16:20:27.571693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998729+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:20:27.998729+0000","last_peered":"2024-02-19T16:20:27.998729+0000","last_clean":"2024-02-19T16:20:27.998729+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:20:27.998729+0000","last_undegraded":"2024-02-19T16:20:27.998729+0000","last_fullsized":"2024-02-19T16:20:27.998729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672801+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:20:28.672801+0000","last_peered":"2024-02-19T16:20:28.672801+0000","last_clean":"2024-02-19T16:20:28.672801+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:20:28.672801+0000","last_undegraded":"2024-02-19T16:20:28.672801+0000","last_fullsized":"2024-02-19T16:20:28.672801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703919+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:20:28.703919+0000","last_peered":"2024-02-19T16:20:28.703919+0000","last_clean":"2024-02-19T16:20:28.703919+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:20:28.703919+0000","last_undegraded":"2024-02-19T16:20:28.703919+0000","last_fullsized":"2024-02-19T16:20:28.703919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.810029+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:20:40.810029+0000","last_peered":"2024-02-19T16:20:40.810029+0000","last_clean":"2024-02-19T16:20:40.810029+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:20:40.810029+0000","last_undegraded":"2024-02-19T16:20:40.810029+0000","last_fullsized":"2024-02-19T16:20:40.810029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704333+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:20:28.704333+0000","last_peered":"2024-02-19T16:20:28.704333+0000","last_clean":"2024-02-19T16:20:28.704333+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:20:28.704333+0000","last_undegraded":"2024-02-19T16:20:28.704333+0000","last_fullsized":"2024-02-19T16:20:28.704333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327232+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:20:27.327232+0000","last_peered":"2024-02-19T16:20:27.327232+0000","last_clean":"2024-02-19T16:20:27.327232+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:20:27.327232+0000","last_undegraded":"2024-02-19T16:20:27.327232+0000","last_fullsized":"2024-02-19T16:20:27.327232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.798712+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:20:39.798712+0000","last_peered":"2024-02-19T16:20:39.798712+0000","last_clean":"2024-02-19T16:20:39.798712+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:20:39.798712+0000","last_undegraded":"2024-02-19T16:20:39.798712+0000","last_fullsized":"2024-02-19T16:20:39.798712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544636+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:20:27.544636+0000","last_peered":"2024-02-19T16:20:27.544636+0000","last_clean":"2024-02-19T16:20:27.544636+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:20:27.544636+0000","last_undegraded":"2024-02-19T16:20:27.544636+0000","last_fullsized":"2024-02-19T16:20:27.544636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327335+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:20:27.327335+0000","last_peered":"2024-02-19T16:20:27.327335+0000","last_clean":"2024-02-19T16:20:27.327335+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:20:27.327335+0000","last_undegraded":"2024-02-19T16:20:27.327335+0000","last_fullsized":"2024-02-19T16:20:27.327335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802228+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:20:39.802228+0000","last_peered":"2024-02-19T16:20:39.802228+0000","last_clean":"2024-02-19T16:20:39.802228+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:20:39.802228+0000","last_undegraded":"2024-02-19T16:20:39.802228+0000","last_fullsized":"2024-02-19T16:20:39.802228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998255+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:20:27.998255+0000","last_peered":"2024-02-19T16:20:27.998255+0000","last_clean":"2024-02-19T16:20:27.998255+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:20:27.998255+0000","last_undegraded":"2024-02-19T16:20:27.998255+0000","last_fullsized":"2024-02-19T16:20:27.998255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326696+0000","last_change":"2024-02-19T16:20:25.756990+0000","last_active":"2024-02-19T16:20:27.326696+0000","last_peered":"2024-02-19T16:20:27.326696+0000","last_clean":"2024-02-19T16:20:27.326696+0000","last_became_active":"2024-02-19T16:20:25.756255+0000","last_became_peered":"2024-02-19T16:20:25.756255+0000","last_unstale":"2024-02-19T16:20:27.326696+0000","last_undegraded":"2024-02-19T16:20:27.326696+0000","last_fullsized":"2024-02-19T16:20:27.326696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:56:20.728617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.801668+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:20:39.801668+0000","last_peered":"2024-02-19T16:20:39.801668+0000","last_clean":"2024-02-19T16:20:39.801668+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:20:39.801668+0000","last_undegraded":"2024-02-19T16:20:39.801668+0000","last_fullsized":"2024-02-19T16:20:39.801668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571523+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:20:27.571523+0000","last_peered":"2024-02-19T16:20:27.571523+0000","last_clean":"2024-02-19T16:20:27.571523+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:20:27.571523+0000","last_undegraded":"2024-02-19T16:20:27.571523+0000","last_fullsized":"2024-02-19T16:20:27.571523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327345+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:20:27.327345+0000","last_peered":"2024-02-19T16:20:27.327345+0000","last_clean":"2024-02-19T16:20:27.327345+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:20:27.327345+0000","last_undegraded":"2024-02-19T16:20:27.327345+0000","last_fullsized":"2024-02-19T16:20:27.327345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327284+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:20:27.327284+0000","last_peered":"2024-02-19T16:20:27.327284+0000","last_clean":"2024-02-19T16:20:27.327284+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:20:27.327284+0000","last_undegraded":"2024-02-19T16:20:27.327284+0000","last_fullsized":"2024-02-19T16:20:27.327284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704257+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:20:28.704257+0000","last_peered":"2024-02-19T16:20:28.704257+0000","last_clean":"2024-02-19T16:20:28.704257+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:20:28.704257+0000","last_undegraded":"2024-02-19T16:20:28.704257+0000","last_fullsized":"2024-02-19T16:20:28.704257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327302+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:20:27.327302+0000","last_peered":"2024-02-19T16:20:27.327302+0000","last_clean":"2024-02-19T16:20:27.327302+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:20:27.327302+0000","last_undegraded":"2024-02-19T16:20:27.327302+0000","last_fullsized":"2024-02-19T16:20:27.327302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802713+0000","last_change":"2024-02-19T16:20:20.636038+0000","last_active":"2024-02-19T16:20:39.802713+0000","last_peered":"2024-02-19T16:20:39.802713+0000","last_clean":"2024-02-19T16:20:39.802713+0000","last_became_active":"2024-02-19T16:20:20.635260+0000","last_became_peered":"2024-02-19T16:20:20.635260+0000","last_unstale":"2024-02-19T16:20:39.802713+0000","last_undegraded":"2024-02-19T16:20:39.802713+0000","last_fullsized":"2024-02-19T16:20:39.802713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:52:11.483671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735321+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:20:27.735321+0000","last_peered":"2024-02-19T16:20:27.735321+0000","last_clean":"2024-02-19T16:20:27.735321+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:20:27.735321+0000","last_undegraded":"2024-02-19T16:20:27.735321+0000","last_fullsized":"2024-02-19T16:20:27.735321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673354+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:20:28.673354+0000","last_peered":"2024-02-19T16:20:28.673354+0000","last_clean":"2024-02-19T16:20:28.673354+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:20:28.673354+0000","last_undegraded":"2024-02-19T16:20:28.673354+0000","last_fullsized":"2024-02-19T16:20:28.673354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327587+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:20:27.327587+0000","last_peered":"2024-02-19T16:20:27.327587+0000","last_clean":"2024-02-19T16:20:27.327587+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:20:27.327587+0000","last_undegraded":"2024-02-19T16:20:27.327587+0000","last_fullsized":"2024-02-19T16:20:27.327587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735708+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:20:27.735708+0000","last_peered":"2024-02-19T16:20:27.735708+0000","last_clean":"2024-02-19T16:20:27.735708+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:20:27.735708+0000","last_undegraded":"2024-02-19T16:20:27.735708+0000","last_fullsized":"2024-02-19T16:20:27.735708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544354+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:20:27.544354+0000","last_peered":"2024-02-19T16:20:27.544354+0000","last_clean":"2024-02-19T16:20:27.544354+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:20:27.544354+0000","last_undegraded":"2024-02-19T16:20:27.544354+0000","last_fullsized":"2024-02-19T16:20:27.544354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672316+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:20:28.672316+0000","last_peered":"2024-02-19T16:20:28.672316+0000","last_clean":"2024-02-19T16:20:28.672316+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:20:28.672316+0000","last_undegraded":"2024-02-19T16:20:28.672316+0000","last_fullsized":"2024-02-19T16:20:28.672316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735417+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:20:27.735417+0000","last_peered":"2024-02-19T16:20:27.735417+0000","last_clean":"2024-02-19T16:20:27.735417+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:20:27.735417+0000","last_undegraded":"2024-02-19T16:20:27.735417+0000","last_fullsized":"2024-02-19T16:20:27.735417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672431+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:20:28.672431+0000","last_peered":"2024-02-19T16:20:28.672431+0000","last_clean":"2024-02-19T16:20:28.672431+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:20:28.672431+0000","last_undegraded":"2024-02-19T16:20:28.672431+0000","last_fullsized":"2024-02-19T16:20:28.672431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672131+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:20:28.672131+0000","last_peered":"2024-02-19T16:20:28.672131+0000","last_clean":"2024-02-19T16:20:28.672131+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:20:28.672131+0000","last_undegraded":"2024-02-19T16:20:28.672131+0000","last_fullsized":"2024-02-19T16:20:28.672131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735646+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:20:27.735646+0000","last_peered":"2024-02-19T16:20:27.735646+0000","last_clean":"2024-02-19T16:20:27.735646+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:20:27.735646+0000","last_undegraded":"2024-02-19T16:20:27.735646+0000","last_fullsized":"2024-02-19T16:20:27.735646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571589+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:20:27.571589+0000","last_peered":"2024-02-19T16:20:27.571589+0000","last_clean":"2024-02-19T16:20:27.571589+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:20:27.571589+0000","last_undegraded":"2024-02-19T16:20:27.571589+0000","last_fullsized":"2024-02-19T16:20:27.571589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327587+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:20:27.327587+0000","last_peered":"2024-02-19T16:20:27.327587+0000","last_clean":"2024-02-19T16:20:27.327587+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:20:27.327587+0000","last_undegraded":"2024-02-19T16:20:27.327587+0000","last_fullsized":"2024-02-19T16:20:27.327587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327218+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:20:27.327218+0000","last_peered":"2024-02-19T16:20:27.327218+0000","last_clean":"2024-02-19T16:20:27.327218+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:20:27.327218+0000","last_undegraded":"2024-02-19T16:20:27.327218+0000","last_fullsized":"2024-02-19T16:20:27.327218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544556+0000","last_change":"2024-02-19T16:20:21.663952+0000","last_active":"2024-02-19T16:20:27.544556+0000","last_peered":"2024-02-19T16:20:27.544556+0000","last_clean":"2024-02-19T16:20:27.544556+0000","last_became_active":"2024-02-19T16:20:21.663244+0000","last_became_peered":"2024-02-19T16:20:21.663244+0000","last_unstale":"2024-02-19T16:20:27.544556+0000","last_undegraded":"2024-02-19T16:20:27.544556+0000","last_fullsized":"2024-02-19T16:20:27.544556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:13:37.033049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672907+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:20:28.672907+0000","last_peered":"2024-02-19T16:20:28.672907+0000","last_clean":"2024-02-19T16:20:28.672907+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:20:28.672907+0000","last_undegraded":"2024-02-19T16:20:28.672907+0000","last_fullsized":"2024-02-19T16:20:28.672907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571870+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:20:27.571870+0000","last_peered":"2024-02-19T16:20:27.571870+0000","last_clean":"2024-02-19T16:20:27.571870+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:20:27.571870+0000","last_undegraded":"2024-02-19T16:20:27.571870+0000","last_fullsized":"2024-02-19T16:20:27.571870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327691+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:20:27.327691+0000","last_peered":"2024-02-19T16:20:27.327691+0000","last_clean":"2024-02-19T16:20:27.327691+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:20:27.327691+0000","last_undegraded":"2024-02-19T16:20:27.327691+0000","last_fullsized":"2024-02-19T16:20:27.327691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704413+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:20:28.704413+0000","last_peered":"2024-02-19T16:20:28.704413+0000","last_clean":"2024-02-19T16:20:28.704413+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:20:28.704413+0000","last_undegraded":"2024-02-19T16:20:28.704413+0000","last_fullsized":"2024-02-19T16:20:28.704413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672580+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:20:28.672580+0000","last_peered":"2024-02-19T16:20:28.672580+0000","last_clean":"2024-02-19T16:20:28.672580+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:20:28.672580+0000","last_undegraded":"2024-02-19T16:20:28.672580+0000","last_fullsized":"2024-02-19T16:20:28.672580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703863+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:20:28.703863+0000","last_peered":"2024-02-19T16:20:28.703863+0000","last_clean":"2024-02-19T16:20:28.703863+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:20:28.703863+0000","last_undegraded":"2024-02-19T16:20:28.703863+0000","last_fullsized":"2024-02-19T16:20:28.703863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673496+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:20:28.673496+0000","last_peered":"2024-02-19T16:20:28.673496+0000","last_clean":"2024-02-19T16:20:28.673496+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:20:28.673496+0000","last_undegraded":"2024-02-19T16:20:28.673496+0000","last_fullsized":"2024-02-19T16:20:28.673496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672597+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:20:28.672597+0000","last_peered":"2024-02-19T16:20:28.672597+0000","last_clean":"2024-02-19T16:20:28.672597+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:20:28.672597+0000","last_undegraded":"2024-02-19T16:20:28.672597+0000","last_fullsized":"2024-02-19T16:20:28.672597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571787+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:20:27.571787+0000","last_peered":"2024-02-19T16:20:27.571787+0000","last_clean":"2024-02-19T16:20:27.571787+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:20:27.571787+0000","last_undegraded":"2024-02-19T16:20:27.571787+0000","last_fullsized":"2024-02-19T16:20:27.571787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6780,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":112,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":18699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":489673,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.441}]},{"osd":6,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.622}]},{"osd":7,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":9,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60899999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55400000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1084077,"data_compressed":53461,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.623}]},{"osd":3,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.747}]},{"osd":4,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81499999999999995}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76700000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.86199999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88200000000000001}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":490491,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.309},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40100000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.441},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.749}]},{"osd":5,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.63700000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.50700000000000001}]},{"osd":10,"last update":"Mon Feb 19 16:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":490411,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.375},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.28599999999999998}]},{"osd":1,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64900000000000002,"5min":0.875,"15min":0.875},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73699999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.432,"15min":0.434},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.30399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.246}]},{"osd":3,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.438}]},{"osd":4,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57899999999999996}]},{"osd":5,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58399999999999996,"15min":0.59599999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72499999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40500000000000003}]},{"osd":6,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.496},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78700000000000003,"5min":1.393,"15min":1.393},"last":0.56599999999999995}]},{"osd":9,"last update":"Mon Feb 19 16:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46600000000000003}]},{"osd":10,"last update":"Mon Feb 19 16:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1081833,"data_compressed":53461,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40500000000000003,"15min":0.39900000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.437,"15min":0.435},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67300000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75800000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48399999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.623},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.498},{"interface":"front","average":{"1min":0.504,"5min":0.45700000000000002,"15min":0.44900000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46899999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72699999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51200000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64700000000000002}]},{"osd":8,"last update":"Mon Feb 19 16:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64500000000000002,"15min":0.63800000000000001},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.84499999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65800000000000003}]},{"osd":10,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.34899999999999998},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.29399999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77700000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83699999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55800000000000005}]},{"osd":3,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71699999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64700000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.443}]},{"osd":4,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":5,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86699999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.878,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51400000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73399999999999999}]},{"osd":8,"last update":"Mon Feb 19 16:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89900000000000002,"5min":1.014,"15min":1.014},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68100000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.70699999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":493515,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.75700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77600000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Feb 19 16:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.433,"5min":0.315,"15min":0.315},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51800000000000002,"15min":0.49299999999999999},"min":{"1min":0.40999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.80200000000000005}]},{"osd":4,"last update":"Mon Feb 19 16:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.39600000000000002,"15min":0.37},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.51000000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86199999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.625},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.876,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Feb 19 16:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.82499999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.875},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58899999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.752}]},{"osd":10,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.441,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Mon Feb 19 16:19:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.443},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.49099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.53200000000000003}]},{"osd":3,"last update":"Mon Feb 19 16:19:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40600000000000003,"15min":0.38800000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79600000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71699999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.51500000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Feb 19 16:20:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66700000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Feb 19 16:19:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.76600000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64900000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.75}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78900000000000003}]},{"osd":11,"last update":"Mon Feb 19 16:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":494171,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.39900000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51500000000000001,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.623}]},{"osd":3,"last update":"Mon Feb 19 16:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.46700000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.436}]},{"osd":4,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.47899999999999998},"min":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.30399999999999999}]},{"osd":6,"last update":"Mon Feb 19 16:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.628,"15min":0.625},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47499999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.496}]},{"osd":8,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.371}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52800000000000002}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.60999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":489867,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.373,"15min":0.35299999999999998},"min":{"1min":0.29799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53800000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.38800000000000001,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.69299999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.378,"5min":0.318,"15min":0.318},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.623},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.42399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.42299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.82499999999999996,"15min":0.66300000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84699999999999998,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Feb 19 16:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.48099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.746}]},{"osd":6,"last update":"Mon Feb 19 16:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73499999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85699999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.51700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.95299999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.98399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":489695,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.372},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60799999999999998,"5min":0.754,"15min":0.754},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.378,"15min":0.371},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.623}]},{"osd":2,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.44900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73499999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61199999999999999}]},{"osd":6,"last update":"Mon Feb 19 16:20:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88800000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.51600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Feb 19 16:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78000000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66100000000000003,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.76400000000000001}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85099999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1080929,"data_compressed":53461,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.47199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.435}]},{"osd":2,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30199999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.23300000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.871,"15min":0.871},"last":0.495},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71999999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66200000000000003,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76500000000000001,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.55900000000000005}]},{"osd":7,"last update":"Mon Feb 19 16:19:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.50800000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78000000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.313}]},{"osd":9,"last update":"Mon Feb 19 16:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60899999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:20:44.103 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 -- 172.21.15.8:0/1431963555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 msgr2=0x7f6908101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 --2- 172.21.15.8:0/1431963555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f6908101a80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009900 tx=0x7f68f802bc30 comp rx=0 tx=0).stop 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 -- 172.21.15.8:0/1431963555 shutdown_connections 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 --2- 172.21.15.8:0/1431963555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6908101fc0 0x7f6908108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 --2- 172.21.15.8:0/1431963555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f6908101a80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 --2- 172.21.15.8:0/1431963555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6908100410 0x7f6908100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 -- 172.21.15.8:0/1431963555 >> 172.21.15.8:0/1431963555 conn(0x7f69080fb9a0 msgr2=0x7f69080fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:44.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 -- 172.21.15.8:0/1431963555 shutdown_connections 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.236+0000 7f690f8ff640 1 -- 172.21.15.8:0/1431963555 wait complete. 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 Processor -- start 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 -- start start 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6908100410 0x7f69081043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f69081029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6908101fc0 0x7f6908102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690819b400 con 0x7f6908101600 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6908103460 con 0x7f6908101fc0 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690f8ff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6908103620 con 0x7f6908100410 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f69081029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690d69b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6908100410 0x7f69081043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f69081029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33128/0 (socket says 172.21.15.8:33128) 2024-02-19T16:20:44.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 -- 172.21.15.8:0/875443256 learned_addr learned my addr 172.21.15.8:0/875443256 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690de9c640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6908101fc0 0x7f6908102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 -- 172.21.15.8:0/875443256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6908100410 msgr2=0x7f69081043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6908100410 0x7f69081043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 -- 172.21.15.8:0/875443256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6908101fc0 msgr2=0x7f6908102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6908101fc0 0x7f6908102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.237+0000 7f690ce9a640 1 -- 172.21.15.8:0/875443256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8009560 con 0x7f6908101600 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.238+0000 7f690ce9a640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f69081029e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f68f8006010 tx=0x7f68f8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.238+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f80043d0 con 0x7f6908101600 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.238+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68f80052d0 con 0x7f6908101600 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.238+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8003cc0 con 0x7f6908101600 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.238+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6908103840 con 0x7f6908101600 2024-02-19T16:20:44.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.238+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69081a61c0 con 0x7f6908101600 2024-02-19T16:20:44.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.239+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690810be20 con 0x7f6908101600 2024-02-19T16:20:44.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.240+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f68f8005440 con 0x7f6908101600 2024-02-19T16:20:44.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.241+0000 7f68fe7fc640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f68f40a21d0 0x7f68f40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:44.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.241+0000 7f690d69b640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f68f40a21d0 0x7f68f40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:44.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.241+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f68f80f0130 con 0x7f6908101600 2024-02-19T16:20:44.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.241+0000 7f690d69b640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f68f40a21d0 0x7f68f40a4690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6908101460 tx=0x7f6904005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:44.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.245+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f68f80b8e70 con 0x7f6908101600 2024-02-19T16:20:44.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:43 smithi191 ceph-mon[37203]: pgmap v302: 129 pgs: 129 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-19T16:20:44.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:43 smithi191 ceph-mon[37203]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:44.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:43 smithi191 ceph-mon[37203]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:20:44.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/160318948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:20:44.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3223811399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:44.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.425+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69081a6410 con 0x7f68f40a21d0 2024-02-19T16:20:44.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.437+0000 7f68fe7fc640 1 -- 172.21.15.8:0/875443256 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424710 (secure 0 0 0) 0x7f69081a6410 con 0x7f68f40a21d0 2024-02-19T16:20:44.443 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:44.446 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:20:44.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f68f40a21d0 msgr2=0x7f68f40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f68f40a21d0 0x7f68f40a4690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6908101460 tx=0x7f6904005f90 comp rx=0 tx=0).stop 2024-02-19T16:20:44.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 msgr2=0x7f69081029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:44.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f69081029e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f68f8006010 tx=0x7f68f8005e80 comp rx=0 tx=0).stop 2024-02-19T16:20:44.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 shutdown_connections 2024-02-19T16:20:44.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f68f40a21d0 0x7f68f40a4690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6908101fc0 0x7f6908102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6908101600 0x7f69081029e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 --2- 172.21.15.8:0/875443256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6908100410 0x7f69081043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:44.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 >> 172.21.15.8:0/875443256 conn(0x7f69080fb9a0 msgr2=0x7f69080fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:44.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 shutdown_connections 2024-02-19T16:20:44.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:44.441+0000 7f690f8ff640 1 -- 172.21.15.8:0/875443256 wait complete. 2024-02-19T16:20:44.453 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-02-19T16:20:42.716737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597148,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":293,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373480,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568344,"statfs":{"total":1151940427776,"available":1151557984256,"internally_reserved":0,"allocated":12111872,"data_stored":7664188,"data_compressed":319710,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4370,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":67,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003670"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.572038+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:20:27.572038+0000","last_peered":"2024-02-19T16:20:27.572038+0000","last_clean":"2024-02-19T16:20:27.572038+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:20:27.572038+0000","last_undegraded":"2024-02-19T16:20:27.572038+0000","last_fullsized":"2024-02-19T16:20:27.572038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544543+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:20:27.544543+0000","last_peered":"2024-02-19T16:20:27.544543+0000","last_clean":"2024-02-19T16:20:27.544543+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:20:27.544543+0000","last_undegraded":"2024-02-19T16:20:27.544543+0000","last_fullsized":"2024-02-19T16:20:27.544543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544440+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:20:27.544440+0000","last_peered":"2024-02-19T16:20:27.544440+0000","last_clean":"2024-02-19T16:20:27.544440+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:20:27.544440+0000","last_undegraded":"2024-02-19T16:20:27.544440+0000","last_fullsized":"2024-02-19T16:20:27.544440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327628+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:20:27.327628+0000","last_peered":"2024-02-19T16:20:27.327628+0000","last_clean":"2024-02-19T16:20:27.327628+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:20:27.327628+0000","last_undegraded":"2024-02-19T16:20:27.327628+0000","last_fullsized":"2024-02-19T16:20:27.327628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998322+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:20:27.998322+0000","last_peered":"2024-02-19T16:20:27.998322+0000","last_clean":"2024-02-19T16:20:27.998322+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:20:27.998322+0000","last_undegraded":"2024-02-19T16:20:27.998322+0000","last_fullsized":"2024-02-19T16:20:27.998322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545733+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:20:27.545733+0000","last_peered":"2024-02-19T16:20:27.545733+0000","last_clean":"2024-02-19T16:20:27.545733+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:20:27.545733+0000","last_undegraded":"2024-02-19T16:20:27.545733+0000","last_fullsized":"2024-02-19T16:20:27.545733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735104+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:20:27.735104+0000","last_peered":"2024-02-19T16:20:27.735104+0000","last_clean":"2024-02-19T16:20:27.735104+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:20:27.735104+0000","last_undegraded":"2024-02-19T16:20:27.735104+0000","last_fullsized":"2024-02-19T16:20:27.735104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703761+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:20:28.703761+0000","last_peered":"2024-02-19T16:20:28.703761+0000","last_clean":"2024-02-19T16:20:28.703761+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:20:28.703761+0000","last_undegraded":"2024-02-19T16:20:28.703761+0000","last_fullsized":"2024-02-19T16:20:28.703761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673644+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:20:28.673644+0000","last_peered":"2024-02-19T16:20:28.673644+0000","last_clean":"2024-02-19T16:20:28.673644+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:20:28.673644+0000","last_undegraded":"2024-02-19T16:20:28.673644+0000","last_fullsized":"2024-02-19T16:20:28.673644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.546117+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:20:27.546117+0000","last_peered":"2024-02-19T16:20:27.546117+0000","last_clean":"2024-02-19T16:20:27.546117+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:20:27.546117+0000","last_undegraded":"2024-02-19T16:20:27.546117+0000","last_fullsized":"2024-02-19T16:20:27.546117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735668+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:20:27.735668+0000","last_peered":"2024-02-19T16:20:27.735668+0000","last_clean":"2024-02-19T16:20:27.735668+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:20:27.735668+0000","last_undegraded":"2024-02-19T16:20:27.735668+0000","last_fullsized":"2024-02-19T16:20:27.735668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327425+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:20:27.327425+0000","last_peered":"2024-02-19T16:20:27.327425+0000","last_clean":"2024-02-19T16:20:27.327425+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:20:27.327425+0000","last_undegraded":"2024-02-19T16:20:27.327425+0000","last_fullsized":"2024-02-19T16:20:27.327425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672705+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:20:28.672705+0000","last_peered":"2024-02-19T16:20:28.672705+0000","last_clean":"2024-02-19T16:20:28.672705+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:20:28.672705+0000","last_undegraded":"2024-02-19T16:20:28.672705+0000","last_fullsized":"2024-02-19T16:20:28.672705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544243+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:20:27.544243+0000","last_peered":"2024-02-19T16:20:27.544243+0000","last_clean":"2024-02-19T16:20:27.544243+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:20:27.544243+0000","last_undegraded":"2024-02-19T16:20:27.544243+0000","last_fullsized":"2024-02-19T16:20:27.544243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544226+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:20:27.544226+0000","last_peered":"2024-02-19T16:20:27.544226+0000","last_clean":"2024-02-19T16:20:27.544226+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:20:27.544226+0000","last_undegraded":"2024-02-19T16:20:27.544226+0000","last_fullsized":"2024-02-19T16:20:27.544226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672970+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:20:28.672970+0000","last_peered":"2024-02-19T16:20:28.672970+0000","last_clean":"2024-02-19T16:20:28.672970+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:20:28.672970+0000","last_undegraded":"2024-02-19T16:20:28.672970+0000","last_fullsized":"2024-02-19T16:20:28.672970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571874+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:20:27.571874+0000","last_peered":"2024-02-19T16:20:27.571874+0000","last_clean":"2024-02-19T16:20:27.571874+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:20:27.571874+0000","last_undegraded":"2024-02-19T16:20:27.571874+0000","last_fullsized":"2024-02-19T16:20:27.571874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326877+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:20:27.326877+0000","last_peered":"2024-02-19T16:20:27.326877+0000","last_clean":"2024-02-19T16:20:27.326877+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:20:27.326877+0000","last_undegraded":"2024-02-19T16:20:27.326877+0000","last_fullsized":"2024-02-19T16:20:27.326877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327610+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:20:27.327610+0000","last_peered":"2024-02-19T16:20:27.327610+0000","last_clean":"2024-02-19T16:20:27.327610+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:20:27.327610+0000","last_undegraded":"2024-02-19T16:20:27.327610+0000","last_fullsized":"2024-02-19T16:20:27.327610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327374+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:20:27.327374+0000","last_peered":"2024-02-19T16:20:27.327374+0000","last_clean":"2024-02-19T16:20:27.327374+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:20:27.327374+0000","last_undegraded":"2024-02-19T16:20:27.327374+0000","last_fullsized":"2024-02-19T16:20:27.327374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.810358+0000","last_change":"2024-02-19T16:20:20.638225+0000","last_active":"2024-02-19T16:20:40.810358+0000","last_peered":"2024-02-19T16:20:40.810358+0000","last_clean":"2024-02-19T16:20:40.810358+0000","last_became_active":"2024-02-19T16:20:20.637923+0000","last_became_peered":"2024-02-19T16:20:20.637923+0000","last_unstale":"2024-02-19T16:20:40.810358+0000","last_undegraded":"2024-02-19T16:20:40.810358+0000","last_fullsized":"2024-02-19T16:20:40.810358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:42:28.514081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704202+0000","last_change":"2024-02-19T16:20:21.662054+0000","last_active":"2024-02-19T16:20:28.704202+0000","last_peered":"2024-02-19T16:20:28.704202+0000","last_clean":"2024-02-19T16:20:28.704202+0000","last_became_active":"2024-02-19T16:20:21.661721+0000","last_became_peered":"2024-02-19T16:20:21.661721+0000","last_unstale":"2024-02-19T16:20:28.704202+0000","last_undegraded":"2024-02-19T16:20:28.704202+0000","last_fullsized":"2024-02-19T16:20:28.704202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:27:13.322445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326924+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:20:27.326924+0000","last_peered":"2024-02-19T16:20:27.326924+0000","last_clean":"2024-02-19T16:20:27.326924+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:20:27.326924+0000","last_undegraded":"2024-02-19T16:20:27.326924+0000","last_fullsized":"2024-02-19T16:20:27.326924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703625+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:20:28.703625+0000","last_peered":"2024-02-19T16:20:28.703625+0000","last_clean":"2024-02-19T16:20:28.703625+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:20:28.703625+0000","last_undegraded":"2024-02-19T16:20:28.703625+0000","last_fullsized":"2024-02-19T16:20:28.703625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544532+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:20:27.544532+0000","last_peered":"2024-02-19T16:20:27.544532+0000","last_clean":"2024-02-19T16:20:27.544532+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:20:27.544532+0000","last_undegraded":"2024-02-19T16:20:27.544532+0000","last_fullsized":"2024-02-19T16:20:27.544532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327037+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:20:27.327037+0000","last_peered":"2024-02-19T16:20:27.327037+0000","last_clean":"2024-02-19T16:20:27.327037+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:20:27.327037+0000","last_undegraded":"2024-02-19T16:20:27.327037+0000","last_fullsized":"2024-02-19T16:20:27.327037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.807310+0000","last_change":"2024-02-19T16:20:20.634795+0000","last_active":"2024-02-19T16:20:40.807310+0000","last_peered":"2024-02-19T16:20:40.807310+0000","last_clean":"2024-02-19T16:20:40.807310+0000","last_became_active":"2024-02-19T16:20:20.634475+0000","last_became_peered":"2024-02-19T16:20:20.634475+0000","last_unstale":"2024-02-19T16:20:40.807310+0000","last_undegraded":"2024-02-19T16:20:40.807310+0000","last_fullsized":"2024-02-19T16:20:40.807310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:23:19.541767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673230+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:20:28.673230+0000","last_peered":"2024-02-19T16:20:28.673230+0000","last_clean":"2024-02-19T16:20:28.673230+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:20:28.673230+0000","last_undegraded":"2024-02-19T16:20:28.673230+0000","last_fullsized":"2024-02-19T16:20:28.673230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327010+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:20:27.327010+0000","last_peered":"2024-02-19T16:20:27.327010+0000","last_clean":"2024-02-19T16:20:27.327010+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:20:27.327010+0000","last_undegraded":"2024-02-19T16:20:27.327010+0000","last_fullsized":"2024-02-19T16:20:27.327010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998397+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:20:27.998397+0000","last_peered":"2024-02-19T16:20:27.998397+0000","last_clean":"2024-02-19T16:20:27.998397+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:20:27.998397+0000","last_undegraded":"2024-02-19T16:20:27.998397+0000","last_fullsized":"2024-02-19T16:20:27.998397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672970+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:20:28.672970+0000","last_peered":"2024-02-19T16:20:28.672970+0000","last_clean":"2024-02-19T16:20:28.672970+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:20:28.672970+0000","last_undegraded":"2024-02-19T16:20:28.672970+0000","last_fullsized":"2024-02-19T16:20:28.672970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327531+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:20:27.327531+0000","last_peered":"2024-02-19T16:20:27.327531+0000","last_clean":"2024-02-19T16:20:27.327531+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:20:27.327531+0000","last_undegraded":"2024-02-19T16:20:27.327531+0000","last_fullsized":"2024-02-19T16:20:27.327531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802449+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:20:39.802449+0000","last_peered":"2024-02-19T16:20:39.802449+0000","last_clean":"2024-02-19T16:20:39.802449+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:20:39.802449+0000","last_undegraded":"2024-02-19T16:20:39.802449+0000","last_fullsized":"2024-02-19T16:20:39.802449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998350+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:20:27.998350+0000","last_peered":"2024-02-19T16:20:27.998350+0000","last_clean":"2024-02-19T16:20:27.998350+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:20:27.998350+0000","last_undegraded":"2024-02-19T16:20:27.998350+0000","last_fullsized":"2024-02-19T16:20:27.998350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327742+0000","last_change":"2024-02-19T16:20:25.757507+0000","last_active":"2024-02-19T16:20:27.327742+0000","last_peered":"2024-02-19T16:20:27.327742+0000","last_clean":"2024-02-19T16:20:27.327742+0000","last_became_active":"2024-02-19T16:20:25.757101+0000","last_became_peered":"2024-02-19T16:20:25.757101+0000","last_unstale":"2024-02-19T16:20:27.327742+0000","last_undegraded":"2024-02-19T16:20:27.327742+0000","last_fullsized":"2024-02-19T16:20:27.327742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:40:32.118424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545654+0000","last_change":"2024-02-19T16:20:20.638520+0000","last_active":"2024-02-19T16:20:27.545654+0000","last_peered":"2024-02-19T16:20:27.545654+0000","last_clean":"2024-02-19T16:20:27.545654+0000","last_became_active":"2024-02-19T16:20:20.638204+0000","last_became_peered":"2024-02-19T16:20:20.638204+0000","last_unstale":"2024-02-19T16:20:27.545654+0000","last_undegraded":"2024-02-19T16:20:27.545654+0000","last_fullsized":"2024-02-19T16:20:27.545654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:42:49.275623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672807+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:20:28.672807+0000","last_peered":"2024-02-19T16:20:28.672807+0000","last_clean":"2024-02-19T16:20:28.672807+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:20:28.672807+0000","last_undegraded":"2024-02-19T16:20:28.672807+0000","last_fullsized":"2024-02-19T16:20:28.672807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571918+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:20:27.571918+0000","last_peered":"2024-02-19T16:20:27.571918+0000","last_clean":"2024-02-19T16:20:27.571918+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:20:27.571918+0000","last_undegraded":"2024-02-19T16:20:27.571918+0000","last_fullsized":"2024-02-19T16:20:27.571918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.805823+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:20:40.805823+0000","last_peered":"2024-02-19T16:20:40.805823+0000","last_clean":"2024-02-19T16:20:40.805823+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:20:40.805823+0000","last_undegraded":"2024-02-19T16:20:40.805823+0000","last_fullsized":"2024-02-19T16:20:40.805823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703876+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:20:28.703876+0000","last_peered":"2024-02-19T16:20:28.703876+0000","last_clean":"2024-02-19T16:20:28.703876+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:20:28.703876+0000","last_undegraded":"2024-02-19T16:20:28.703876+0000","last_fullsized":"2024-02-19T16:20:28.703876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544582+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:20:27.544582+0000","last_peered":"2024-02-19T16:20:27.544582+0000","last_clean":"2024-02-19T16:20:27.544582+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:20:27.544582+0000","last_undegraded":"2024-02-19T16:20:27.544582+0000","last_fullsized":"2024-02-19T16:20:27.544582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672281+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:20:28.672281+0000","last_peered":"2024-02-19T16:20:28.672281+0000","last_clean":"2024-02-19T16:20:28.672281+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:20:28.672281+0000","last_undegraded":"2024-02-19T16:20:28.672281+0000","last_fullsized":"2024-02-19T16:20:28.672281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326084+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:20:27.326084+0000","last_peered":"2024-02-19T16:20:27.326084+0000","last_clean":"2024-02-19T16:20:27.326084+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:20:27.326084+0000","last_undegraded":"2024-02-19T16:20:27.326084+0000","last_fullsized":"2024-02-19T16:20:27.326084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703928+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:20:28.703928+0000","last_peered":"2024-02-19T16:20:28.703928+0000","last_clean":"2024-02-19T16:20:28.703928+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:20:28.703928+0000","last_undegraded":"2024-02-19T16:20:28.703928+0000","last_fullsized":"2024-02-19T16:20:28.703928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.546027+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:20:27.546027+0000","last_peered":"2024-02-19T16:20:27.546027+0000","last_clean":"2024-02-19T16:20:27.546027+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:20:27.546027+0000","last_undegraded":"2024-02-19T16:20:27.546027+0000","last_fullsized":"2024-02-19T16:20:27.546027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673595+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:20:28.673595+0000","last_peered":"2024-02-19T16:20:28.673595+0000","last_clean":"2024-02-19T16:20:28.673595+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:20:28.673595+0000","last_undegraded":"2024-02-19T16:20:28.673595+0000","last_fullsized":"2024-02-19T16:20:28.673595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326201+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:20:27.326201+0000","last_peered":"2024-02-19T16:20:27.326201+0000","last_clean":"2024-02-19T16:20:27.326201+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:20:27.326201+0000","last_undegraded":"2024-02-19T16:20:27.326201+0000","last_fullsized":"2024-02-19T16:20:27.326201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802661+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:20:39.802661+0000","last_peered":"2024-02-19T16:20:39.802661+0000","last_clean":"2024-02-19T16:20:39.802661+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:20:39.802661+0000","last_undegraded":"2024-02-19T16:20:39.802661+0000","last_fullsized":"2024-02-19T16:20:39.802661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544558+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:20:27.544558+0000","last_peered":"2024-02-19T16:20:27.544558+0000","last_clean":"2024-02-19T16:20:27.544558+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:20:27.544558+0000","last_undegraded":"2024-02-19T16:20:27.544558+0000","last_fullsized":"2024-02-19T16:20:27.544558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327331+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:20:27.327331+0000","last_peered":"2024-02-19T16:20:27.327331+0000","last_clean":"2024-02-19T16:20:27.327331+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:20:27.327331+0000","last_undegraded":"2024-02-19T16:20:27.327331+0000","last_fullsized":"2024-02-19T16:20:27.327331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672459+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:20:28.672459+0000","last_peered":"2024-02-19T16:20:28.672459+0000","last_clean":"2024-02-19T16:20:28.672459+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:20:28.672459+0000","last_undegraded":"2024-02-19T16:20:28.672459+0000","last_fullsized":"2024-02-19T16:20:28.672459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673458+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:20:28.673458+0000","last_peered":"2024-02-19T16:20:28.673458+0000","last_clean":"2024-02-19T16:20:28.673458+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:20:28.673458+0000","last_undegraded":"2024-02-19T16:20:28.673458+0000","last_fullsized":"2024-02-19T16:20:28.673458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326348+0000","last_change":"2024-02-19T16:20:25.763053+0000","last_active":"2024-02-19T16:20:27.326348+0000","last_peered":"2024-02-19T16:20:27.326348+0000","last_clean":"2024-02-19T16:20:27.326348+0000","last_became_active":"2024-02-19T16:20:25.762571+0000","last_became_peered":"2024-02-19T16:20:25.762571+0000","last_unstale":"2024-02-19T16:20:27.326348+0000","last_undegraded":"2024-02-19T16:20:27.326348+0000","last_fullsized":"2024-02-19T16:20:27.326348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:16:47.050677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:37.154975+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:20:37.154975+0000","last_peered":"2024-02-19T16:20:37.154975+0000","last_clean":"2024-02-19T16:20:37.154975+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:20:37.154975+0000","last_undegraded":"2024-02-19T16:20:37.154975+0000","last_fullsized":"2024-02-19T16:20:37.154975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998655+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:20:27.998655+0000","last_peered":"2024-02-19T16:20:27.998655+0000","last_clean":"2024-02-19T16:20:27.998655+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:20:27.998655+0000","last_undegraded":"2024-02-19T16:20:27.998655+0000","last_fullsized":"2024-02-19T16:20:27.998655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673176+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:20:28.673176+0000","last_peered":"2024-02-19T16:20:28.673176+0000","last_clean":"2024-02-19T16:20:28.673176+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:20:28.673176+0000","last_undegraded":"2024-02-19T16:20:28.673176+0000","last_fullsized":"2024-02-19T16:20:28.673176+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.800466+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:20:39.800466+0000","last_peered":"2024-02-19T16:20:39.800466+0000","last_clean":"2024-02-19T16:20:39.800466+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:20:39.800466+0000","last_undegraded":"2024-02-19T16:20:39.800466+0000","last_fullsized":"2024-02-19T16:20:39.800466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672607+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:20:28.672607+0000","last_peered":"2024-02-19T16:20:28.672607+0000","last_clean":"2024-02-19T16:20:28.672607+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:20:28.672607+0000","last_undegraded":"2024-02-19T16:20:28.672607+0000","last_fullsized":"2024-02-19T16:20:28.672607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571726+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:20:27.571726+0000","last_peered":"2024-02-19T16:20:27.571726+0000","last_clean":"2024-02-19T16:20:27.571726+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:20:27.571726+0000","last_undegraded":"2024-02-19T16:20:27.571726+0000","last_fullsized":"2024-02-19T16:20:27.571726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.800891+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:20:39.800891+0000","last_peered":"2024-02-19T16:20:39.800891+0000","last_clean":"2024-02-19T16:20:39.800891+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:20:39.800891+0000","last_undegraded":"2024-02-19T16:20:39.800891+0000","last_fullsized":"2024-02-19T16:20:39.800891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735335+0000","last_change":"2024-02-19T16:20:21.660468+0000","last_active":"2024-02-19T16:20:27.735335+0000","last_peered":"2024-02-19T16:20:27.735335+0000","last_clean":"2024-02-19T16:20:27.735335+0000","last_became_active":"2024-02-19T16:20:21.659988+0000","last_became_peered":"2024-02-19T16:20:21.659988+0000","last_unstale":"2024-02-19T16:20:27.735335+0000","last_undegraded":"2024-02-19T16:20:27.735335+0000","last_fullsized":"2024-02-19T16:20:27.735335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:57:08.316143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326598+0000","last_change":"2024-02-19T16:20:25.756660+0000","last_active":"2024-02-19T16:20:27.326598+0000","last_peered":"2024-02-19T16:20:27.326598+0000","last_clean":"2024-02-19T16:20:27.326598+0000","last_became_active":"2024-02-19T16:20:25.755644+0000","last_became_peered":"2024-02-19T16:20:25.755644+0000","last_unstale":"2024-02-19T16:20:27.326598+0000","last_undegraded":"2024-02-19T16:20:27.326598+0000","last_fullsized":"2024-02-19T16:20:27.326598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:52:07.717537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.808440+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:20:40.808440+0000","last_peered":"2024-02-19T16:20:40.808440+0000","last_clean":"2024-02-19T16:20:40.808440+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:20:40.808440+0000","last_undegraded":"2024-02-19T16:20:40.808440+0000","last_fullsized":"2024-02-19T16:20:40.808440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735086+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:20:27.735086+0000","last_peered":"2024-02-19T16:20:27.735086+0000","last_clean":"2024-02-19T16:20:27.735086+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:20:27.735086+0000","last_undegraded":"2024-02-19T16:20:27.735086+0000","last_fullsized":"2024-02-19T16:20:27.735086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672956+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:20:28.672956+0000","last_peered":"2024-02-19T16:20:28.672956+0000","last_clean":"2024-02-19T16:20:28.672956+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:20:28.672956+0000","last_undegraded":"2024-02-19T16:20:28.672956+0000","last_fullsized":"2024-02-19T16:20:28.672956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.808634+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:20:40.808634+0000","last_peered":"2024-02-19T16:20:40.808634+0000","last_clean":"2024-02-19T16:20:40.808634+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:20:40.808634+0000","last_undegraded":"2024-02-19T16:20:40.808634+0000","last_fullsized":"2024-02-19T16:20:40.808634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672674+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:20:28.672674+0000","last_peered":"2024-02-19T16:20:28.672674+0000","last_clean":"2024-02-19T16:20:28.672674+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:20:28.672674+0000","last_undegraded":"2024-02-19T16:20:28.672674+0000","last_fullsized":"2024-02-19T16:20:28.672674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672708+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:20:28.672708+0000","last_peered":"2024-02-19T16:20:28.672708+0000","last_clean":"2024-02-19T16:20:28.672708+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:20:28.672708+0000","last_undegraded":"2024-02-19T16:20:28.672708+0000","last_fullsized":"2024-02-19T16:20:28.672708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998660+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:20:27.998660+0000","last_peered":"2024-02-19T16:20:27.998660+0000","last_clean":"2024-02-19T16:20:27.998660+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:20:27.998660+0000","last_undegraded":"2024-02-19T16:20:27.998660+0000","last_fullsized":"2024-02-19T16:20:27.998660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571854+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:20:27.571854+0000","last_peered":"2024-02-19T16:20:27.571854+0000","last_clean":"2024-02-19T16:20:27.571854+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:20:27.571854+0000","last_undegraded":"2024-02-19T16:20:27.571854+0000","last_fullsized":"2024-02-19T16:20:27.571854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998348+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:20:27.998348+0000","last_peered":"2024-02-19T16:20:27.998348+0000","last_clean":"2024-02-19T16:20:27.998348+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:20:27.998348+0000","last_undegraded":"2024-02-19T16:20:27.998348+0000","last_fullsized":"2024-02-19T16:20:27.998348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326398+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:20:27.326398+0000","last_peered":"2024-02-19T16:20:27.326398+0000","last_clean":"2024-02-19T16:20:27.326398+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:20:27.326398+0000","last_undegraded":"2024-02-19T16:20:27.326398+0000","last_fullsized":"2024-02-19T16:20:27.326398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545899+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:20:27.545899+0000","last_peered":"2024-02-19T16:20:27.545899+0000","last_clean":"2024-02-19T16:20:27.545899+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:20:27.545899+0000","last_undegraded":"2024-02-19T16:20:27.545899+0000","last_fullsized":"2024-02-19T16:20:27.545899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672805+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:20:28.672805+0000","last_peered":"2024-02-19T16:20:28.672805+0000","last_clean":"2024-02-19T16:20:28.672805+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:20:28.672805+0000","last_undegraded":"2024-02-19T16:20:28.672805+0000","last_fullsized":"2024-02-19T16:20:28.672805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735253+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:20:27.735253+0000","last_peered":"2024-02-19T16:20:27.735253+0000","last_clean":"2024-02-19T16:20:27.735253+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:20:27.735253+0000","last_undegraded":"2024-02-19T16:20:27.735253+0000","last_fullsized":"2024-02-19T16:20:27.735253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.545804+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:20:27.545804+0000","last_peered":"2024-02-19T16:20:27.545804+0000","last_clean":"2024-02-19T16:20:27.545804+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:20:27.545804+0000","last_undegraded":"2024-02-19T16:20:27.545804+0000","last_fullsized":"2024-02-19T16:20:27.545804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672900+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:20:28.672900+0000","last_peered":"2024-02-19T16:20:28.672900+0000","last_clean":"2024-02-19T16:20:28.672900+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:20:28.672900+0000","last_undegraded":"2024-02-19T16:20:28.672900+0000","last_fullsized":"2024-02-19T16:20:28.672900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672569+0000","last_change":"2024-02-19T16:20:21.671091+0000","last_active":"2024-02-19T16:20:28.672569+0000","last_peered":"2024-02-19T16:20:28.672569+0000","last_clean":"2024-02-19T16:20:28.672569+0000","last_became_active":"2024-02-19T16:20:21.670762+0000","last_became_peered":"2024-02-19T16:20:21.670762+0000","last_unstale":"2024-02-19T16:20:28.672569+0000","last_undegraded":"2024-02-19T16:20:28.672569+0000","last_fullsized":"2024-02-19T16:20:28.672569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:14:35.707812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.809032+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:20:40.809032+0000","last_peered":"2024-02-19T16:20:40.809032+0000","last_clean":"2024-02-19T16:20:40.809032+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:20:40.809032+0000","last_undegraded":"2024-02-19T16:20:40.809032+0000","last_fullsized":"2024-02-19T16:20:40.809032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735547+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:20:27.735547+0000","last_peered":"2024-02-19T16:20:27.735547+0000","last_clean":"2024-02-19T16:20:27.735547+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:20:27.735547+0000","last_undegraded":"2024-02-19T16:20:27.735547+0000","last_fullsized":"2024-02-19T16:20:27.735547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703942+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:20:28.703942+0000","last_peered":"2024-02-19T16:20:28.703942+0000","last_clean":"2024-02-19T16:20:28.703942+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:20:28.703942+0000","last_undegraded":"2024-02-19T16:20:28.703942+0000","last_fullsized":"2024-02-19T16:20:28.703942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.801843+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:20:39.801843+0000","last_peered":"2024-02-19T16:20:39.801843+0000","last_clean":"2024-02-19T16:20:39.801843+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:20:39.801843+0000","last_undegraded":"2024-02-19T16:20:39.801843+0000","last_fullsized":"2024-02-19T16:20:39.801843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.734959+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:20:27.734959+0000","last_peered":"2024-02-19T16:20:27.734959+0000","last_clean":"2024-02-19T16:20:27.734959+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:20:27.734959+0000","last_undegraded":"2024-02-19T16:20:27.734959+0000","last_fullsized":"2024-02-19T16:20:27.734959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673126+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:20:28.673126+0000","last_peered":"2024-02-19T16:20:28.673126+0000","last_clean":"2024-02-19T16:20:28.673126+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:20:28.673126+0000","last_undegraded":"2024-02-19T16:20:28.673126+0000","last_fullsized":"2024-02-19T16:20:28.673126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672385+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:20:28.672385+0000","last_peered":"2024-02-19T16:20:28.672385+0000","last_clean":"2024-02-19T16:20:28.672385+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:20:28.672385+0000","last_undegraded":"2024-02-19T16:20:28.672385+0000","last_fullsized":"2024-02-19T16:20:28.672385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.806469+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:20:40.806469+0000","last_peered":"2024-02-19T16:20:40.806469+0000","last_clean":"2024-02-19T16:20:40.806469+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:20:40.806469+0000","last_undegraded":"2024-02-19T16:20:40.806469+0000","last_fullsized":"2024-02-19T16:20:40.806469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571693+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:20:27.571693+0000","last_peered":"2024-02-19T16:20:27.571693+0000","last_clean":"2024-02-19T16:20:27.571693+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:20:27.571693+0000","last_undegraded":"2024-02-19T16:20:27.571693+0000","last_fullsized":"2024-02-19T16:20:27.571693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998729+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:20:27.998729+0000","last_peered":"2024-02-19T16:20:27.998729+0000","last_clean":"2024-02-19T16:20:27.998729+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:20:27.998729+0000","last_undegraded":"2024-02-19T16:20:27.998729+0000","last_fullsized":"2024-02-19T16:20:27.998729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672801+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:20:28.672801+0000","last_peered":"2024-02-19T16:20:28.672801+0000","last_clean":"2024-02-19T16:20:28.672801+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:20:28.672801+0000","last_undegraded":"2024-02-19T16:20:28.672801+0000","last_fullsized":"2024-02-19T16:20:28.672801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703919+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:20:28.703919+0000","last_peered":"2024-02-19T16:20:28.703919+0000","last_clean":"2024-02-19T16:20:28.703919+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:20:28.703919+0000","last_undegraded":"2024-02-19T16:20:28.703919+0000","last_fullsized":"2024-02-19T16:20:28.703919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:40.810029+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:20:40.810029+0000","last_peered":"2024-02-19T16:20:40.810029+0000","last_clean":"2024-02-19T16:20:40.810029+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:20:40.810029+0000","last_undegraded":"2024-02-19T16:20:40.810029+0000","last_fullsized":"2024-02-19T16:20:40.810029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704333+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:20:28.704333+0000","last_peered":"2024-02-19T16:20:28.704333+0000","last_clean":"2024-02-19T16:20:28.704333+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:20:28.704333+0000","last_undegraded":"2024-02-19T16:20:28.704333+0000","last_fullsized":"2024-02-19T16:20:28.704333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327232+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:20:27.327232+0000","last_peered":"2024-02-19T16:20:27.327232+0000","last_clean":"2024-02-19T16:20:27.327232+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:20:27.327232+0000","last_undegraded":"2024-02-19T16:20:27.327232+0000","last_fullsized":"2024-02-19T16:20:27.327232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.798712+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:20:39.798712+0000","last_peered":"2024-02-19T16:20:39.798712+0000","last_clean":"2024-02-19T16:20:39.798712+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:20:39.798712+0000","last_undegraded":"2024-02-19T16:20:39.798712+0000","last_fullsized":"2024-02-19T16:20:39.798712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544636+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:20:27.544636+0000","last_peered":"2024-02-19T16:20:27.544636+0000","last_clean":"2024-02-19T16:20:27.544636+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:20:27.544636+0000","last_undegraded":"2024-02-19T16:20:27.544636+0000","last_fullsized":"2024-02-19T16:20:27.544636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327335+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:20:27.327335+0000","last_peered":"2024-02-19T16:20:27.327335+0000","last_clean":"2024-02-19T16:20:27.327335+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:20:27.327335+0000","last_undegraded":"2024-02-19T16:20:27.327335+0000","last_fullsized":"2024-02-19T16:20:27.327335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802228+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:20:39.802228+0000","last_peered":"2024-02-19T16:20:39.802228+0000","last_clean":"2024-02-19T16:20:39.802228+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:20:39.802228+0000","last_undegraded":"2024-02-19T16:20:39.802228+0000","last_fullsized":"2024-02-19T16:20:39.802228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.998255+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:20:27.998255+0000","last_peered":"2024-02-19T16:20:27.998255+0000","last_clean":"2024-02-19T16:20:27.998255+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:20:27.998255+0000","last_undegraded":"2024-02-19T16:20:27.998255+0000","last_fullsized":"2024-02-19T16:20:27.998255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.326696+0000","last_change":"2024-02-19T16:20:25.756990+0000","last_active":"2024-02-19T16:20:27.326696+0000","last_peered":"2024-02-19T16:20:27.326696+0000","last_clean":"2024-02-19T16:20:27.326696+0000","last_became_active":"2024-02-19T16:20:25.756255+0000","last_became_peered":"2024-02-19T16:20:25.756255+0000","last_unstale":"2024-02-19T16:20:27.326696+0000","last_undegraded":"2024-02-19T16:20:27.326696+0000","last_fullsized":"2024-02-19T16:20:27.326696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:56:20.728617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.801668+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:20:39.801668+0000","last_peered":"2024-02-19T16:20:39.801668+0000","last_clean":"2024-02-19T16:20:39.801668+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:20:39.801668+0000","last_undegraded":"2024-02-19T16:20:39.801668+0000","last_fullsized":"2024-02-19T16:20:39.801668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571523+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:20:27.571523+0000","last_peered":"2024-02-19T16:20:27.571523+0000","last_clean":"2024-02-19T16:20:27.571523+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:20:27.571523+0000","last_undegraded":"2024-02-19T16:20:27.571523+0000","last_fullsized":"2024-02-19T16:20:27.571523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327345+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:20:27.327345+0000","last_peered":"2024-02-19T16:20:27.327345+0000","last_clean":"2024-02-19T16:20:27.327345+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:20:27.327345+0000","last_undegraded":"2024-02-19T16:20:27.327345+0000","last_fullsized":"2024-02-19T16:20:27.327345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327284+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:20:27.327284+0000","last_peered":"2024-02-19T16:20:27.327284+0000","last_clean":"2024-02-19T16:20:27.327284+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:20:27.327284+0000","last_undegraded":"2024-02-19T16:20:27.327284+0000","last_fullsized":"2024-02-19T16:20:27.327284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704257+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:20:28.704257+0000","last_peered":"2024-02-19T16:20:28.704257+0000","last_clean":"2024-02-19T16:20:28.704257+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:20:28.704257+0000","last_undegraded":"2024-02-19T16:20:28.704257+0000","last_fullsized":"2024-02-19T16:20:28.704257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327302+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:20:27.327302+0000","last_peered":"2024-02-19T16:20:27.327302+0000","last_clean":"2024-02-19T16:20:27.327302+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:20:27.327302+0000","last_undegraded":"2024-02-19T16:20:27.327302+0000","last_fullsized":"2024-02-19T16:20:27.327302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:39.802713+0000","last_change":"2024-02-19T16:20:20.636038+0000","last_active":"2024-02-19T16:20:39.802713+0000","last_peered":"2024-02-19T16:20:39.802713+0000","last_clean":"2024-02-19T16:20:39.802713+0000","last_became_active":"2024-02-19T16:20:20.635260+0000","last_became_peered":"2024-02-19T16:20:20.635260+0000","last_unstale":"2024-02-19T16:20:39.802713+0000","last_undegraded":"2024-02-19T16:20:39.802713+0000","last_fullsized":"2024-02-19T16:20:39.802713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:52:11.483671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735321+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:20:27.735321+0000","last_peered":"2024-02-19T16:20:27.735321+0000","last_clean":"2024-02-19T16:20:27.735321+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:20:27.735321+0000","last_undegraded":"2024-02-19T16:20:27.735321+0000","last_fullsized":"2024-02-19T16:20:27.735321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673354+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:20:28.673354+0000","last_peered":"2024-02-19T16:20:28.673354+0000","last_clean":"2024-02-19T16:20:28.673354+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:20:28.673354+0000","last_undegraded":"2024-02-19T16:20:28.673354+0000","last_fullsized":"2024-02-19T16:20:28.673354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327587+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:20:27.327587+0000","last_peered":"2024-02-19T16:20:27.327587+0000","last_clean":"2024-02-19T16:20:27.327587+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:20:27.327587+0000","last_undegraded":"2024-02-19T16:20:27.327587+0000","last_fullsized":"2024-02-19T16:20:27.327587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735708+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:20:27.735708+0000","last_peered":"2024-02-19T16:20:27.735708+0000","last_clean":"2024-02-19T16:20:27.735708+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:20:27.735708+0000","last_undegraded":"2024-02-19T16:20:27.735708+0000","last_fullsized":"2024-02-19T16:20:27.735708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544354+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:20:27.544354+0000","last_peered":"2024-02-19T16:20:27.544354+0000","last_clean":"2024-02-19T16:20:27.544354+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:20:27.544354+0000","last_undegraded":"2024-02-19T16:20:27.544354+0000","last_fullsized":"2024-02-19T16:20:27.544354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672316+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:20:28.672316+0000","last_peered":"2024-02-19T16:20:28.672316+0000","last_clean":"2024-02-19T16:20:28.672316+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:20:28.672316+0000","last_undegraded":"2024-02-19T16:20:28.672316+0000","last_fullsized":"2024-02-19T16:20:28.672316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735417+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:20:27.735417+0000","last_peered":"2024-02-19T16:20:27.735417+0000","last_clean":"2024-02-19T16:20:27.735417+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:20:27.735417+0000","last_undegraded":"2024-02-19T16:20:27.735417+0000","last_fullsized":"2024-02-19T16:20:27.735417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672431+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:20:28.672431+0000","last_peered":"2024-02-19T16:20:28.672431+0000","last_clean":"2024-02-19T16:20:28.672431+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:20:28.672431+0000","last_undegraded":"2024-02-19T16:20:28.672431+0000","last_fullsized":"2024-02-19T16:20:28.672431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672131+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:20:28.672131+0000","last_peered":"2024-02-19T16:20:28.672131+0000","last_clean":"2024-02-19T16:20:28.672131+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:20:28.672131+0000","last_undegraded":"2024-02-19T16:20:28.672131+0000","last_fullsized":"2024-02-19T16:20:28.672131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.735646+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:20:27.735646+0000","last_peered":"2024-02-19T16:20:27.735646+0000","last_clean":"2024-02-19T16:20:27.735646+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:20:27.735646+0000","last_undegraded":"2024-02-19T16:20:27.735646+0000","last_fullsized":"2024-02-19T16:20:27.735646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571589+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:20:27.571589+0000","last_peered":"2024-02-19T16:20:27.571589+0000","last_clean":"2024-02-19T16:20:27.571589+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:20:27.571589+0000","last_undegraded":"2024-02-19T16:20:27.571589+0000","last_fullsized":"2024-02-19T16:20:27.571589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327587+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:20:27.327587+0000","last_peered":"2024-02-19T16:20:27.327587+0000","last_clean":"2024-02-19T16:20:27.327587+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:20:27.327587+0000","last_undegraded":"2024-02-19T16:20:27.327587+0000","last_fullsized":"2024-02-19T16:20:27.327587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327218+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:20:27.327218+0000","last_peered":"2024-02-19T16:20:27.327218+0000","last_clean":"2024-02-19T16:20:27.327218+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:20:27.327218+0000","last_undegraded":"2024-02-19T16:20:27.327218+0000","last_fullsized":"2024-02-19T16:20:27.327218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.544556+0000","last_change":"2024-02-19T16:20:21.663952+0000","last_active":"2024-02-19T16:20:27.544556+0000","last_peered":"2024-02-19T16:20:27.544556+0000","last_clean":"2024-02-19T16:20:27.544556+0000","last_became_active":"2024-02-19T16:20:21.663244+0000","last_became_peered":"2024-02-19T16:20:21.663244+0000","last_unstale":"2024-02-19T16:20:27.544556+0000","last_undegraded":"2024-02-19T16:20:27.544556+0000","last_fullsized":"2024-02-19T16:20:27.544556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:13:37.033049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672907+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:20:28.672907+0000","last_peered":"2024-02-19T16:20:28.672907+0000","last_clean":"2024-02-19T16:20:28.672907+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:20:28.672907+0000","last_undegraded":"2024-02-19T16:20:28.672907+0000","last_fullsized":"2024-02-19T16:20:28.672907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571870+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:20:27.571870+0000","last_peered":"2024-02-19T16:20:27.571870+0000","last_clean":"2024-02-19T16:20:27.571870+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:20:27.571870+0000","last_undegraded":"2024-02-19T16:20:27.571870+0000","last_fullsized":"2024-02-19T16:20:27.571870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.327691+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:20:27.327691+0000","last_peered":"2024-02-19T16:20:27.327691+0000","last_clean":"2024-02-19T16:20:27.327691+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:20:27.327691+0000","last_undegraded":"2024-02-19T16:20:27.327691+0000","last_fullsized":"2024-02-19T16:20:27.327691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.704413+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:20:28.704413+0000","last_peered":"2024-02-19T16:20:28.704413+0000","last_clean":"2024-02-19T16:20:28.704413+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:20:28.704413+0000","last_undegraded":"2024-02-19T16:20:28.704413+0000","last_fullsized":"2024-02-19T16:20:28.704413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672580+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:20:28.672580+0000","last_peered":"2024-02-19T16:20:28.672580+0000","last_clean":"2024-02-19T16:20:28.672580+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:20:28.672580+0000","last_undegraded":"2024-02-19T16:20:28.672580+0000","last_fullsized":"2024-02-19T16:20:28.672580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.703863+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:20:28.703863+0000","last_peered":"2024-02-19T16:20:28.703863+0000","last_clean":"2024-02-19T16:20:28.703863+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:20:28.703863+0000","last_undegraded":"2024-02-19T16:20:28.703863+0000","last_fullsized":"2024-02-19T16:20:28.703863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.673496+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:20:28.673496+0000","last_peered":"2024-02-19T16:20:28.673496+0000","last_clean":"2024-02-19T16:20:28.673496+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:20:28.673496+0000","last_undegraded":"2024-02-19T16:20:28.673496+0000","last_fullsized":"2024-02-19T16:20:28.673496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:28.672597+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:20:28.672597+0000","last_peered":"2024-02-19T16:20:28.672597+0000","last_clean":"2024-02-19T16:20:28.672597+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:20:28.672597+0000","last_undegraded":"2024-02-19T16:20:28.672597+0000","last_fullsized":"2024-02-19T16:20:28.672597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-19T16:20:27.571787+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:20:27.571787+0000","last_peered":"2024-02-19T16:20:27.571787+0000","last_clean":"2024-02-19T16:20:27.571787+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:20:27.571787+0000","last_undegraded":"2024-02-19T16:20:27.571787+0000","last_fullsized":"2024-02-19T16:20:27.571787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6780,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":112,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":18699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":489673,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.441}]},{"osd":6,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.622}]},{"osd":7,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":9,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60899999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55400000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1084077,"data_compressed":53461,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.623}]},{"osd":3,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.747}]},{"osd":4,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81499999999999995}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76700000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.86199999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88200000000000001}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":490491,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.309},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40100000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.441},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.749}]},{"osd":5,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.63700000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.50700000000000001}]},{"osd":10,"last update":"Mon Feb 19 16:20:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":490411,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.375},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.28599999999999998}]},{"osd":1,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64900000000000002,"5min":0.875,"15min":0.875},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73699999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.432,"15min":0.434},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.30399999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.246}]},{"osd":3,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.438}]},{"osd":4,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57899999999999996}]},{"osd":5,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55600000000000005,"15min":0.56599999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58399999999999996,"15min":0.59599999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72499999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40500000000000003}]},{"osd":6,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.496},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78700000000000003,"5min":1.393,"15min":1.393},"last":0.56599999999999995}]},{"osd":9,"last update":"Mon Feb 19 16:19:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46600000000000003}]},{"osd":10,"last update":"Mon Feb 19 16:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1081833,"data_compressed":53461,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40500000000000003,"15min":0.39900000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.437,"15min":0.435},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67300000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75800000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48399999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.623},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.498},{"interface":"front","average":{"1min":0.504,"5min":0.45700000000000002,"15min":0.44900000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46899999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72699999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51200000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64700000000000002}]},{"osd":8,"last update":"Mon Feb 19 16:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64500000000000002,"15min":0.63800000000000001},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.84499999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65800000000000003}]},{"osd":10,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36299999999999999,"15min":0.34899999999999998},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.29399999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77700000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83699999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55800000000000005}]},{"osd":3,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71699999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64700000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.443}]},{"osd":4,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":5,"last update":"Mon Feb 19 16:20:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86699999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.878,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51400000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73399999999999999}]},{"osd":8,"last update":"Mon Feb 19 16:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89900000000000002,"5min":1.014,"15min":1.014},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68100000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.70699999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":493515,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.75700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77600000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Feb 19 16:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.433,"5min":0.315,"15min":0.315},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52700000000000002,"15min":0.49199999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51800000000000002,"15min":0.49299999999999999},"min":{"1min":0.40999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.80200000000000005}]},{"osd":4,"last update":"Mon Feb 19 16:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.39600000000000002,"15min":0.37},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.51000000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86199999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.625},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.876,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Feb 19 16:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.82499999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.875},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58899999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.752}]},{"osd":10,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.441,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Mon Feb 19 16:19:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.443},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.49099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.53200000000000003}]},{"osd":3,"last update":"Mon Feb 19 16:19:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40600000000000003,"15min":0.38800000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79600000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71699999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.51500000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Feb 19 16:20:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66700000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Feb 19 16:19:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.76600000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64900000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.75}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78900000000000003}]},{"osd":11,"last update":"Mon Feb 19 16:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":494171,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.39900000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51500000000000001,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.623}]},{"osd":3,"last update":"Mon Feb 19 16:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.46700000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.436}]},{"osd":4,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.47899999999999998},"min":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.30399999999999999}]},{"osd":6,"last update":"Mon Feb 19 16:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.628,"15min":0.625},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47499999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.496}]},{"osd":8,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.371}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52800000000000002}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.60999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":489867,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.373,"15min":0.35299999999999998},"min":{"1min":0.29799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53800000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.38800000000000001,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.69299999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.378,"5min":0.318,"15min":0.318},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.623},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.42399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.42299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.82499999999999996,"15min":0.66300000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84699999999999998,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Feb 19 16:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.48099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.746}]},{"osd":6,"last update":"Mon Feb 19 16:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73499999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85699999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.51700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.95299999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.98399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":489695,"data_compressed":17703,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.372},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60799999999999998,"5min":0.754,"15min":0.754},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.378,"15min":0.371},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.623}]},{"osd":2,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.44900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73499999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61199999999999999}]},{"osd":6,"last update":"Mon Feb 19 16:20:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88800000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.51600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Feb 19 16:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78000000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66100000000000003,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.76400000000000001}]},{"osd":9,"last update":"Mon Feb 19 16:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85099999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1080929,"data_compressed":53461,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.748,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.47199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.435}]},{"osd":2,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30199999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.23300000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.871,"15min":0.871},"last":0.495},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71999999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66200000000000003,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76500000000000001,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.55900000000000005}]},{"osd":7,"last update":"Mon Feb 19 16:19:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.50800000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78000000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.313}]},{"osd":9,"last update":"Mon Feb 19 16:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60899999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:20:44.461 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-19T16:20:44.461 INFO:tasks.cephadm:Setting up client nodes... 2024-02-19T16:20:44.462 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-19T16:20:44.740 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:45.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:44 smithi156 ceph-mon[36502]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-19T16:20:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:44 smithi156 ceph-mon[36502]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:20:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:44 smithi156 ceph-mon[36502]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:20:45.109 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:44 smithi191 ceph-mon[37203]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-19T16:20:45.109 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:44 smithi191 ceph-mon[37203]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:20:45.109 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:44 smithi191 ceph-mon[37203]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:20:45.155 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:44 smithi008 ceph-mon[32573]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-19T16:20:45.155 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:44 smithi008 ceph-mon[32573]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:20:45.155 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:44 smithi008 ceph-mon[32573]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:20:46.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:45 smithi156 ceph-mon[36502]: pgmap v303: 129 pgs: 129 active+clean; 619 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-02-19T16:20:46.223 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:45 smithi008 ceph-mon[32573]: pgmap v303: 129 pgs: 129 active+clean; 619 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-02-19T16:20:46.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:45 smithi191 ceph-mon[37203]: pgmap v303: 129 pgs: 129 active+clean; 619 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-02-19T16:20:46.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 -- 172.21.15.8:0/4044397266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68102030 msgr2=0x7fdc68102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:46.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 --2- 172.21.15.8:0/4044397266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68102030 0x7fdc68102450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdc640099c0 tx=0x7fdc64009cd0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 -- 172.21.15.8:0/4044397266 shutdown_connections 2024-02-19T16:20:46.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 --2- 172.21.15.8:0/4044397266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103bd0 0x7fdc6810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 --2- 172.21.15.8:0/4044397266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68102030 0x7fdc68102450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 --2- 172.21.15.8:0/4044397266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68103230 0x7fdc68103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 -- 172.21.15.8:0/4044397266 >> 172.21.15.8:0/4044397266 conn(0x7fdc680fd5f0 msgr2=0x7fdc680ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 -- 172.21.15.8:0/4044397266 shutdown_connections 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.617+0000 7fdc70199700 1 -- 172.21.15.8:0/4044397266 wait complete. 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 Processor -- start 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 -- start start 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68102030 0x7fdc68075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103230 0x7fdc680743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 0x7fdc68074910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68075090 con 0x7fdc68103bd0 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68075200 con 0x7fdc68103230 2024-02-19T16:20:46.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc70199700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc681a1f70 con 0x7fdc68102030 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc6e736700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 0x7fdc68074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc6e736700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 0x7fdc68074910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49244/0 (socket says 172.21.15.8:49244) 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc6d734700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103230 0x7fdc680743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc6e736700 1 -- 172.21.15.8:0/2086074747 learned_addr learned my addr 172.21.15.8:0/2086074747 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.618+0000 7fdc6df35700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68102030 0x7fdc68075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6d734700 1 -- 172.21.15.8:0/2086074747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68102030 msgr2=0x7fdc68075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6d734700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68102030 0x7fdc68075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6d734700 1 -- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 msgr2=0x7fdc68074910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6d734700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 0x7fdc68074910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6d734700 1 -- 172.21.15.8:0/2086074747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640095f0 con 0x7fdc68103230 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6e736700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 0x7fdc68074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc6d734700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103230 0x7fdc680743d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800e9f0 tx=0x7fdc5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:46.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800c9c0 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc58010430 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc681a2250 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58018680 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.619+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc681a2750 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.620+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc681093f0 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.621+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fdc58010910 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.622+0000 7fdc5effd700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc540a2290 0x7fdc540a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.622+0000 7fdc6df35700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc540a2290 0x7fdc540a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.622+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdc58014070 con 0x7fdc68103230 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.622+0000 7fdc6df35700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc540a2290 0x7fdc540a4750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc640036c0 tx=0x7fdc64005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:46.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.625+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fdc58096e00 con 0x7fdc68103230 2024-02-19T16:20:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.918+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fdc680504f0 con 0x7fdc68103230 2024-02-19T16:20:46.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.922+0000 7fdc5effd700 1 -- 172.21.15.8:0/2086074747 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fdc580105a0 con 0x7fdc68103230 2024-02-19T16:20:46.924 INFO:teuthology.orchestra.run.smithi008.stdout:[client.0] 2024-02-19T16:20:46.924 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDef9NlVl/kNhAAjLsNIXU8FBoEePLrkWD5jw== 2024-02-19T16:20:46.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.924+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc540a2290 msgr2=0x7fdc540a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:46.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.924+0000 7fdc70199700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc540a2290 0x7fdc540a4750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc640036c0 tx=0x7fdc64005ad0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.924+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103230 msgr2=0x7fdc680743d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.924+0000 7fdc70199700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103230 0x7fdc680743d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800e9f0 tx=0x7fdc5800ed00 comp rx=0 tx=0).stop 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 shutdown_connections 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc540a2290 0x7fdc540a4750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc68103bd0 0x7fdc68074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68102030 0x7fdc68075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 --2- 172.21.15.8:0/2086074747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc68103230 0x7fdc680743d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 >> 172.21.15.8:0/2086074747 conn(0x7fdc680fd5f0 msgr2=0x7fdc68106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 shutdown_connections 2024-02-19T16:20:46.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:46.925+0000 7fdc70199700 1 -- 172.21.15.8:0/2086074747 wait complete. 2024-02-19T16:20:47.614 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:20:47.614 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-19T16:20:47.614 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-19T16:20:47.661 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-19T16:20:47.661 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-19T16:20:47.662 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph mgr dump --format=json 2024-02-19T16:20:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:48.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:47 smithi008 ceph-mon[32573]: pgmap v304: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-02-19T16:20:48.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:47 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2086074747' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T16:20:48.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:47 smithi008 ceph-mon[32573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T16:20:48.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:47 smithi008 ceph-mon[32573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T16:20:48.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:47 smithi156 ceph-mon[36502]: pgmap v304: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-02-19T16:20:48.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:47 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2086074747' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T16:20:48.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:47 smithi156 ceph-mon[36502]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T16:20:48.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:47 smithi156 ceph-mon[36502]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T16:20:48.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:47 smithi191 ceph-mon[37203]: pgmap v304: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-02-19T16:20:48.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:47 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2086074747' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T16:20:48.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:47 smithi191 ceph-mon[37203]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T16:20:48.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:47 smithi191 ceph-mon[37203]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 -- 172.21.15.8:0/3093752150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101140 msgr2=0x7fb188101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/3093752150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101140 0x7fb188101520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb178009990 tx=0x7fb178009ca0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 -- 172.21.15.8:0/3093752150 shutdown_connections 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/3093752150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb188106220 0x7fb18810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/3093752150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101140 0x7fb188101520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/3093752150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188101af0 0x7fb188105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 -- 172.21.15.8:0/3093752150 >> 172.21.15.8:0/3093752150 conn(0x7fb188077fb0 msgr2=0x7fb1880783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:49.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 -- 172.21.15.8:0/3093752150 shutdown_connections 2024-02-19T16:20:49.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.696+0000 7fb18eb4b700 1 -- 172.21.15.8:0/3093752150 wait complete. 2024-02-19T16:20:49.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 Processor -- start 2024-02-19T16:20:49.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 -- start start 2024-02-19T16:20:49.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb188101140 0x7fb1881992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 0x7fb188199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188106220 0x7fb1881a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18819a0a0 con 0x7fb188101af0 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1881a48b0 con 0x7fb188101140 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18eb4b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1881a49f0 con 0x7fb188106220 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb187fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 0x7fb188199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb187fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 0x7fb188199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49262/0 (socket says 172.21.15.8:49262) 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb187fff700 1 -- 172.21.15.8:0/274715747 learned_addr learned my addr 172.21.15.8:0/274715747 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18c8e7700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb188101140 0x7fb1881992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb18d0e8700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188106220 0x7fb1881a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.697+0000 7fb187fff700 1 -- 172.21.15.8:0/274715747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188106220 msgr2=0x7fb1881a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb187fff700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188106220 0x7fb1881a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb187fff700 1 -- 172.21.15.8:0/274715747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb188101140 msgr2=0x7fb1881992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:49.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb187fff700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb188101140 0x7fb1881992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb187fff700 1 -- 172.21.15.8:0/274715747 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780095f0 con 0x7fb188101af0 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb187fff700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 0x7fb188199830 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00e970 tx=0x7fb17c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c009790 con 0x7fb188101af0 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb17c00c7e0 con 0x7fb188101af0 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1881a4c80 con 0x7fb188101af0 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c005580 con 0x7fb188101af0 2024-02-19T16:20:49.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.698+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1881a5180 con 0x7fb188101af0 2024-02-19T16:20:49.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.699+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb188109450 con 0x7fb188101af0 2024-02-19T16:20:49.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.700+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb17c00cce0 con 0x7fb188101af0 2024-02-19T16:20:49.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.701+0000 7fb185ffb700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb1700a21e0 0x7fb1700a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:49.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.701+0000 7fb18c8e7700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb1700a21e0 0x7fb1700a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:49.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.701+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb17c014070 con 0x7fb188101af0 2024-02-19T16:20:49.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.701+0000 7fb18c8e7700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb1700a21e0 0x7fb1700a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb17800b2e0 tx=0x7fb178005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:49.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.706+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb17c096f10 con 0x7fb188101af0 2024-02-19T16:20:49.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.972+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb188110f70 con 0x7fb188101af0 2024-02-19T16:20:49.974 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:49 smithi008 ceph-mon[32573]: pgmap v305: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-02-19T16:20:49.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.977+0000 7fb185ffb700 1 -- 172.21.15.8:0/274715747 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261472 (secure 0 0 0) 0x7fb17c09a5d0 con 0x7fb188101af0 2024-02-19T16:20:49.983 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:49.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb1700a21e0 msgr2=0x7fb1700a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:49.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb1700a21e0 0x7fb1700a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb17800b2e0 tx=0x7fb178005960 comp rx=0 tx=0).stop 2024-02-19T16:20:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 msgr2=0x7fb188199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 0x7fb188199830 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00e970 tx=0x7fb17c00ec80 comp rx=0 tx=0).stop 2024-02-19T16:20:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 shutdown_connections 2024-02-19T16:20:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb1700a21e0 0x7fb1700a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188106220 0x7fb1881a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb188101140 0x7fb1881992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 --2- 172.21.15.8:0/274715747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb188101af0 0x7fb188199830 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 >> 172.21.15.8:0/274715747 conn(0x7fb188077fb0 msgr2=0x7fb1881000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 shutdown_connections 2024-02-19T16:20:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:49.980+0000 7fb18eb4b700 1 -- 172.21.15.8:0/274715747 wait complete. 2024-02-19T16:20:50.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:49 smithi156 ceph-mon[36502]: pgmap v305: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-02-19T16:20:50.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:49 smithi191 ceph-mon[37203]: pgmap v305: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-02-19T16:20:50.806 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":354789085},{"type":"v1","addr":"172.21.15.8:6801","nonce":354789085}]},"active_addr":"172.21.15.8:6801/354789085","active_change":"2024-02-19T16:13:20.630869+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.8:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":2140457421}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":2267597929}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":543580195}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":1905912286}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":1064944975}]}]} 2024-02-19T16:20:50.811 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-19T16:20:50.811 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-19T16:20:50.811 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd dump --format=json 2024-02-19T16:20:51.060 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:51.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:50 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/274715747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-19T16:20:51.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:50 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/274715747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-19T16:20:51.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:50 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/274715747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-19T16:20:52.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:51 smithi156 ceph-mon[36502]: pgmap v306: 129 pgs: 129 active+clean; 625 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-02-19T16:20:52.258 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:51 smithi008 ceph-mon[32573]: pgmap v306: 129 pgs: 129 active+clean; 625 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-02-19T16:20:52.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:51 smithi191 ceph-mon[37203]: pgmap v306: 129 pgs: 129 active+clean; 625 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3581541795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 msgr2=0x7f9f9c1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3581541795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c1083f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0099c0 tx=0x7f9f8c009cd0 comp rx=0 tx=0).stop 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3581541795 shutdown_connections 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3581541795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f9c1029f0 0x7f9f9c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3581541795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c1083f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3581541795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c102030 0x7f9f9c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:52.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3581541795 >> 172.21.15.8:0/3581541795 conn(0x7f9f9c079110 msgr2=0x7f9f9c079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3581541795 shutdown_connections 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.815+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3581541795 wait complete. 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 Processor -- start 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 -- start start 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f9c102030 0x7f9f9c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c1029f0 0x7f9f9c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19d3c0 con 0x7f9f9c108010 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19d530 con 0x7f9f9c1029f0 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9fa1f7c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19d6f0 con 0x7f9f9c102030 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9f9bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c1029f0 0x7f9f9c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9f9b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f9c102030 0x7f9f9c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:52.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9f9bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49270/0 (socket says 172.21.15.8:49270) 2024-02-19T16:20:52.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.816+0000 7f9f9bfff700 1 -- 172.21.15.8:0/3600718883 learned_addr learned my addr 172.21.15.8:0/3600718883 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:52.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f9affd700 1 -- 172.21.15.8:0/3600718883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f9c102030 msgr2=0x7f9f9c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f9affd700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f9c102030 0x7f9f9c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f9affd700 1 -- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 msgr2=0x7f9f9c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f9affd700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f9affd700 1 -- 172.21.15.8:0/3600718883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0095f0 con 0x7f9f9c1029f0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f9affd700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c1029f0 0x7f9f9c198a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e9f0 tx=0x7f9f9000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000c9c0 con 0x7f9f9c1029f0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c19d9d0 con 0x7f9f9c1029f0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f90010430 con 0x7f9f9c1029f0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90018660 con 0x7f9f9c1029f0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.817+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c1a5230 con 0x7f9f9c1029f0 2024-02-19T16:20:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.818+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c10a4f0 con 0x7f9f9c1029f0 2024-02-19T16:20:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.819+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f9f90010910 con 0x7f9f9c1029f0 2024-02-19T16:20:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.820+0000 7f9f98ff9700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9f840a21e0 0x7f9f840a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.820+0000 7f9f9b7fe700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9f840a21e0 0x7f9f840a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.820+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9f90014070 con 0x7f9f9c1029f0 2024-02-19T16:20:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.821+0000 7f9f9b7fe700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9f840a21e0 0x7f9f840a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:52.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:52.828+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9f90097310 con 0x7f9f9c1029f0 2024-02-19T16:20:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.033+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f9c1998a0 con 0x7f9f9c1029f0 2024-02-19T16:20:53.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.035+0000 7f9f98ff9700 1 -- 172.21.15.8:0/3600718883 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f9f900105a0 con 0x7f9f9c1029f0 2024-02-19T16:20:53.036 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:53.036 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9f840a21e0 msgr2=0x7f9f840a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9f840a21e0 0x7f9f840a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c005fb0 comp rx=0 tx=0).stop 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c1029f0 msgr2=0x7f9f9c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c1029f0 0x7f9f9c198a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e9f0 tx=0x7f9f9000ed00 comp rx=0 tx=0).stop 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 shutdown_connections 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9f840a21e0 0x7f9f840a46a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:53.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f9c108010 0x7f9f9c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:53.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f9c102030 0x7f9f9c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:53.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 --2- 172.21.15.8:0/3600718883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f9c1029f0 0x7f9f9c198a20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:53.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.037+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 >> 172.21.15.8:0/3600718883 conn(0x7f9f9c079110 msgr2=0x7f9f9c100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:53.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.038+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 shutdown_connections 2024-02-19T16:20:53.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:53.038+0000 7f9fa1f7c700 1 -- 172.21.15.8:0/3600718883 wait complete. 2024-02-19T16:20:53.837 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-19T16:20:53.838 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd dump --format=json 2024-02-19T16:20:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:54.206 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:53 smithi008 ceph-mon[32573]: pgmap v307: 129 pgs: 129 active+clean; 625 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-02-19T16:20:54.206 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:53 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3600718883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:54.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:53 smithi191 ceph-mon[37203]: pgmap v307: 129 pgs: 129 active+clean; 625 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-02-19T16:20:54.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:53 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3600718883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:54.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:53 smithi156 ceph-mon[36502]: pgmap v307: 129 pgs: 129 active+clean; 625 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-02-19T16:20:54.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:53 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3600718883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:55.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/3246342948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01070b0 msgr2=0x7fd6e0107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:55.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/3246342948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01070b0 0x7fd6e0107490 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40099c0 tx=0x7fd6d4009cd0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/3246342948 shutdown_connections 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/3246342948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01013a0 0x7fd6e010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/3246342948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01070b0 0x7fd6e0107490 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/3246342948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e01009e0 0x7fd6e0100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/3246342948 >> 172.21.15.8:0/3246342948 conn(0x7fd6e0076230 msgr2=0x7fd6e0076640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/3246342948 shutdown_connections 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.759+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/3246342948 wait complete. 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.760+0000 7fd6e6ccf700 1 Processor -- start 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.760+0000 7fd6e6ccf700 1 -- start start 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.760+0000 7fd6e6ccf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01009e0 0x7fd6e0101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.760+0000 7fd6e6ccf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 0x7fd6e0102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.760+0000 7fd6e6ccf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e01070b0 0x7fd6e0102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:55.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e6ccf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01a68f0 con 0x7fd6e01013a0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e6ccf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01a6a60 con 0x7fd6e01009e0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e6ccf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01a6c20 con 0x7fd6e01070b0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 0x7fd6e0102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 0x7fd6e0102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43130/0 (socket says 172.21.15.8:43130) 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 -- 172.21.15.8:0/2940048265 learned_addr learned my addr 172.21.15.8:0/2940048265 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e4a6b700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01009e0 0x7fd6e0101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e526c700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e01070b0 0x7fd6e0102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 -- 172.21.15.8:0/2940048265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e01070b0 msgr2=0x7fd6e0102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e01070b0 0x7fd6e0102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 -- 172.21.15.8:0/2940048265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01009e0 msgr2=0x7fd6e0101bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01009e0 0x7fd6e0101bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 -- 172.21.15.8:0/2940048265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40095f0 con 0x7fd6e01013a0 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e4a6b700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01009e0 0x7fd6e0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:20:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6dffff700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 0x7fd6e0102100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00d980 tx=0x7fd6cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc004220 con 0x7fd6e01013a0 2024-02-19T16:20:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6cc007640 con 0x7fd6e01013a0 2024-02-19T16:20:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e01a6f00 con 0x7fd6e01013a0 2024-02-19T16:20:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.761+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00f690 con 0x7fd6e01013a0 2024-02-19T16:20:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.762+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e01a7370 con 0x7fd6e01013a0 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.762+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0068ca0 con 0x7fd6e01013a0 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.764+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fd6cc004410 con 0x7fd6e01013a0 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.764+0000 7fd6ddffb700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd6d00a21e0 0x7fd6d00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.764+0000 7fd6e4a6b700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd6d00a21e0 0x7fd6d00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.764+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd6cc0cddc0 con 0x7fd6e01013a0 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.765+0000 7fd6e4a6b700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd6d00a21e0 0x7fd6d00a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:20:55.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.769+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd6cc0969f0 con 0x7fd6e01013a0 2024-02-19T16:20:55.838 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:55 smithi008 ceph-mon[32573]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-02-19T16:20:55.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.975+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6e01121c0 con 0x7fd6e01013a0 2024-02-19T16:20:55.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.977+0000 7fd6ddffb700 1 -- 172.21.15.8:0/2940048265 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fd6cc09a0b0 con 0x7fd6e01013a0 2024-02-19T16:20:55.978 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:20:55.978 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":91,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:20:27.320240+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:20:55.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd6d00a21e0 msgr2=0x7fd6d00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd6d00a21e0 0x7fd6d00a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d4005fb0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 msgr2=0x7fd6e0102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 0x7fd6e0102100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00d980 tx=0x7fd6cc00dc90 comp rx=0 tx=0).stop 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 shutdown_connections 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd6d00a21e0 0x7fd6d00a46a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e01070b0 0x7fd6e0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6e01009e0 0x7fd6e0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 --2- 172.21.15.8:0/2940048265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e01013a0 0x7fd6e0102100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 >> 172.21.15.8:0/2940048265 conn(0x7fd6e0076230 msgr2=0x7fd6e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 shutdown_connections 2024-02-19T16:20:55.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:20:55.979+0000 7fd6e6ccf700 1 -- 172.21.15.8:0/2940048265 wait complete. 2024-02-19T16:20:56.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:55 smithi191 ceph-mon[37203]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-02-19T16:20:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:55 smithi156 ceph-mon[36502]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-02-19T16:20:56.693 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-19T16:20:56.693 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-19T16:20:56.693 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-19T16:20:56.694 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-19T16:20:56.694 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-19T16:20:56.695 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-19T16:20:56.695 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-19T16:20:56.695 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-19T16:20:56.696 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-19T16:20:56.696 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-19T16:20:56.697 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-19T16:20:56.697 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-19T16:20:57.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:56 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2940048265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:57.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:56 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2940048265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:57.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:56 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2940048265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:20:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.456 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.471 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.472 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.475 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.475 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.545 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.546 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:20:57.935 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:57 smithi008 ceph-mon[32573]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-02-19T16:20:58.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:57 smithi191 ceph-mon[37203]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-02-19T16:20:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:57 smithi156 ceph-mon[36502]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-02-19T16:21:00.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:20:59 smithi008 ceph-mon[32573]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-02-19T16:21:00.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:20:59 smithi191 ceph-mon[37203]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-02-19T16:21:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:20:59 smithi156 ceph-mon[36502]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-02-19T16:21:01.891 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:01 smithi008 ceph-mon[32573]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-19T16:21:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:01 smithi191 ceph-mon[37203]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-19T16:21:02.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:01 smithi156 ceph-mon[36502]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-19T16:21:04.131 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:03 smithi156 ceph-mon[36502]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-19T16:21:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:03 smithi008 ceph-mon[32573]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-19T16:21:04.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:03 smithi191 ceph-mon[37203]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 -- 172.21.15.8:0/207907967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084102290 msgr2=0x7f3084102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 --2- 172.21.15.8:0/207907967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084102290 0x7f3084102670 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3080009990 tx=0x7f3080009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 -- 172.21.15.8:0/207907967 shutdown_connections 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 --2- 172.21.15.8:0/207907967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3084107360 0x7f308410ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 --2- 172.21.15.8:0/207907967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084102290 0x7f3084102670 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 --2- 172.21.15.8:0/207907967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102c40 0x7f3084106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 -- 172.21.15.8:0/207907967 >> 172.21.15.8:0/207907967 conn(0x7f3084079140 msgr2=0x7f3084079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:05.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 -- 172.21.15.8:0/207907967 shutdown_connections 2024-02-19T16:21:05.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.029+0000 7f308b6b8700 1 -- 172.21.15.8:0/207907967 wait complete. 2024-02-19T16:21:05.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 Processor -- start 2024-02-19T16:21:05.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 -- start start 2024-02-19T16:21:05.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102290 0x7f30841985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:05.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3084102c40 0x7f3084198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:05.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084107360 0x7f308419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419d450 con 0x7f3084107360 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419d5c0 con 0x7f3084102290 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f308b6b8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308419d780 con 0x7f3084102c40 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102290 0x7f30841985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089c55700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084107360 0x7f308419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102290 0x7f30841985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:47132/0 (socket says 172.21.15.8:47132) 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 -- 172.21.15.8:0/3894407534 learned_addr learned my addr 172.21.15.8:0/3894407534 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3088c53700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3084102c40 0x7f3084198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 -- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3084102c40 msgr2=0x7f3084198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3084102c40 0x7f3084198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 -- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084107360 msgr2=0x7f308419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3084107360 0x7f308419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.030+0000 7f3089454700 1 -- 172.21.15.8:0/3894407534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800095f0 con 0x7f3084102290 2024-02-19T16:21:05.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.031+0000 7f3089454700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102290 0x7f30841985c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3080000c00 tx=0x7f3080004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:05.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.031+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308001d070 con 0x7f3084102290 2024-02-19T16:21:05.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.031+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f308000b610 con 0x7f3084102290 2024-02-19T16:21:05.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.031+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800173f0 con 0x7f3084102290 2024-02-19T16:21:05.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.031+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308419da00 con 0x7f3084102290 2024-02-19T16:21:05.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.031+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30840743e0 con 0x7f3084102290 2024-02-19T16:21:05.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.032+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3084112380 con 0x7f3084102290 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.033+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f3080022670 con 0x7f3084102290 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.033+0000 7f307a7fc700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f30700a2260 0x7f30700a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.033+0000 7f3088c53700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f30700a2260 0x7f30700a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.033+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f30800cf3b0 con 0x7f3084102290 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.034+0000 7f307a7fc700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] conn(0x7f30700acf20 0x7f30700af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.034+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 --> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30700afa30 con 0x7f30700acf20 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.034+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30800cf720 con 0x7f3084102290 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.034+0000 7f3089c55700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] conn(0x7f30700acf20 0x7f30700af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:05.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.034+0000 7f3088c53700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f30700a2260 0x7f30700a4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3084101c00 tx=0x7f3074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:05.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.034+0000 7f3089c55700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] conn(0x7f30700acf20 0x7f30700af360 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:05.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.035+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== osd.8 v2:172.21.15.191:6816/2037482563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f30700afa30 con 0x7f30700acf20 2024-02-19T16:21:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.051+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 --> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3084062670 con 0x7f30700acf20 2024-02-19T16:21:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f307a7fc700 1 -- 172.21.15.8:0/3894407534 <== osd.8 v2:172.21.15.191:6816/2037482563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3084062670 con 0x7f30700acf20 2024-02-19T16:21:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] conn(0x7f30700acf20 msgr2=0x7f30700af360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] conn(0x7f30700acf20 0x7f30700af360 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f30700a2260 msgr2=0x7f30700a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f30700a2260 0x7f30700a4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3084101c00 tx=0x7f3074009260 comp rx=0 tx=0).stop 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102290 msgr2=0x7f30841985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 --2- 172.21.15.8:0/3894407534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3084102290 0x7f30841985c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3080000c00 tx=0x7f3080004730 comp rx=0 tx=0).stop 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f3089c55700 1 -- 172.21.15.8:0/3894407534 reap_dead start 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 shutdown_connections 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 >> 172.21.15.8:0/3894407534 conn(0x7f3084079140 msgr2=0x7f30841002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 shutdown_connections 2024-02-19T16:21:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:05.052+0000 7f308b6b8700 1 -- 172.21.15.8:0/3894407534 wait complete. 2024-02-19T16:21:06.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:05 smithi008 ceph-mon[32573]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-19T16:21:06.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:05 smithi191 ceph-mon[37203]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-19T16:21:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:05 smithi156 ceph-mon[36502]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-19T16:21:07.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.606+0000 7f42b3a06700 1 -- 172.21.15.8:0/82977882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac074210 msgr2=0x7f42ac0745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:07.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.606+0000 7f42b3a06700 1 --2- 172.21.15.8:0/82977882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac074210 0x7f42ac0745f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009990 tx=0x7f42a8009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.606+0000 7f42b3a06700 1 -- 172.21.15.8:0/82977882 shutdown_connections 2024-02-19T16:21:07.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.606+0000 7f42b3a06700 1 --2- 172.21.15.8:0/82977882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42ac10e9e0 0x7f42ac110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.606+0000 7f42b3a06700 1 --2- 172.21.15.8:0/82977882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac074210 0x7f42ac0745f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.606+0000 7f42b3a06700 1 --2- 172.21.15.8:0/82977882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074b30 0x7f42ac10e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 -- 172.21.15.8:0/82977882 >> 172.21.15.8:0/82977882 conn(0x7f42ac079110 msgr2=0x7f42ac079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 -- 172.21.15.8:0/82977882 shutdown_connections 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 -- 172.21.15.8:0/82977882 wait complete. 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 Processor -- start 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 -- start start 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074210 0x7f42ac19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:07.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42ac074b30 0x7f42ac19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac10e9e0 0x7f42ac19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.607+0000 7f42b3a06700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19f5e0 con 0x7f42ac10e9e0 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b3a06700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19f750 con 0x7f42ac074210 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b3a06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19f910 con 0x7f42ac074b30 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074210 0x7f42ac19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b1fa3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac10e9e0 0x7f42ac19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074210 0x7f42ac19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:38994/0 (socket says 172.21.15.8:38994) 2024-02-19T16:21:07.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 -- 172.21.15.8:0/3142248649 learned_addr learned my addr 172.21.15.8:0/3142248649 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b0fa1700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42ac074b30 0x7f42ac19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 -- 172.21.15.8:0/3142248649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42ac074b30 msgr2=0x7f42ac19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42ac074b30 0x7f42ac19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 -- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac10e9e0 msgr2=0x7f42ac19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42ac10e9e0 0x7f42ac19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 -- 172.21.15.8:0/3142248649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80095f0 con 0x7f42ac074210 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b17a2700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074210 0x7f42ac19a700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42a8000c00 tx=0x7f42a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:07.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a801d070 con 0x7f42ac074210 2024-02-19T16:21:07.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac19fb90 con 0x7f42ac074210 2024-02-19T16:21:07.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42a800b610 con 0x7f42ac074210 2024-02-19T16:21:07.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.608+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80173f0 con 0x7f42ac074210 2024-02-19T16:21:07.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.609+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac1a7450 con 0x7f42ac074210 2024-02-19T16:21:07.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.609+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42ac0758c0 con 0x7f42ac074210 2024-02-19T16:21:07.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.610+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f42a8022670 con 0x7f42ac074210 2024-02-19T16:21:07.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.611+0000 7f42a27fc700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42980a2260 0x7f42980a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:07.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.611+0000 7f42b0fa1700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42980a2260 0x7f42980a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:07.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.611+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f42a80cf8b0 con 0x7f42ac074210 2024-02-19T16:21:07.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.611+0000 7f42a27fc700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] conn(0x7f42980acf20 0x7f42980af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:07.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.611+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 --> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42980afa30 con 0x7f42980acf20 2024-02-19T16:21:07.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.611+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42a80cfc60 con 0x7f42ac074210 2024-02-19T16:21:07.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.612+0000 7f42b1fa3700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] conn(0x7f42980acf20 0x7f42980af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:07.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.612+0000 7f42b0fa1700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42980a2260 0x7f42980a4720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f42ac110700 tx=0x7f429c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:07.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.612+0000 7f42b1fa3700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] conn(0x7f42980acf20 0x7f42980af360 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:07.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.613+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== osd.0 v2:172.21.15.8:6802/94559077 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42980afa30 con 0x7f42980acf20 2024-02-19T16:21:07.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.629+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 --> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42ac114560 con 0x7f42980acf20 2024-02-19T16:21:07.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.629+0000 7f42a27fc700 1 -- 172.21.15.8:0/3142248649 <== osd.0 v2:172.21.15.8:6802/94559077 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f42ac114560 con 0x7f42980acf20 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] conn(0x7f42980acf20 msgr2=0x7f42980af360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] conn(0x7f42980acf20 0x7f42980af360 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42980a2260 msgr2=0x7f42980a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42980a2260 0x7f42980a4720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f42ac110700 tx=0x7f429c009260 comp rx=0 tx=0).stop 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074210 msgr2=0x7f42ac19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 --2- 172.21.15.8:0/3142248649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42ac074210 0x7f42ac19a700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42a8000c00 tx=0x7f42a8004730 comp rx=0 tx=0).stop 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b1fa3700 1 -- 172.21.15.8:0/3142248649 reap_dead start 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 shutdown_connections 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 >> 172.21.15.8:0/3142248649 conn(0x7f42ac079110 msgr2=0x7f42ac0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 shutdown_connections 2024-02-19T16:21:07.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:07.630+0000 7f42b3a06700 1 -- 172.21.15.8:0/3142248649 wait complete. 2024-02-19T16:21:07.995 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:07 smithi008 ceph-mon[32573]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:07 smithi191 ceph-mon[37203]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:08.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:07 smithi156 ceph-mon[36502]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 -- 172.21.15.8:0/1815919346 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54102060 msgr2=0x7fbb541024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 --2- 172.21.15.8:0/1815919346 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54102060 0x7fbb541024e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009990 tx=0x7fbb3c009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 -- 172.21.15.8:0/1815919346 shutdown_connections 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 --2- 172.21.15.8:0/1815919346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb54102a20 0x7fbb5410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 --2- 172.21.15.8:0/1815919346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54108140 0x7fbb54108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 --2- 172.21.15.8:0/1815919346 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54102060 0x7fbb541024e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 -- 172.21.15.8:0/1815919346 >> 172.21.15.8:0/1815919346 conn(0x7fbb540790a0 msgr2=0x7fbb540794b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:08.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 -- 172.21.15.8:0/1815919346 shutdown_connections 2024-02-19T16:21:08.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.803+0000 7fbb596b0700 1 -- 172.21.15.8:0/1815919346 wait complete. 2024-02-19T16:21:08.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 Processor -- start 2024-02-19T16:21:08.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 -- start start 2024-02-19T16:21:08.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb54102060 0x7fbb540742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:08.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54102a20 0x7fbb54074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:08.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54108140 0x7fbb54074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:08.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a6940 con 0x7fbb54108140 2024-02-19T16:21:08.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a6ab0 con 0x7fbb54102060 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb596b0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a6c70 con 0x7fbb54102a20 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb537fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54108140 0x7fbb54074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54102a20 0x7fbb54074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb54102060 0x7fbb540742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54102a20 0x7fbb54074820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:33386/0 (socket says 172.21.15.8:33386) 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb537fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54108140 0x7fbb54074d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34972/0 (socket says 172.21.15.8:34972) 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb527fc700 1 -- 172.21.15.8:0/2058300384 learned_addr learned my addr 172.21.15.8:0/2058300384 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.804+0000 7fbb527fc700 1 -- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb54102060 msgr2=0x7fbb540742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb527fc700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb54102060 0x7fbb540742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb527fc700 1 -- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54108140 msgr2=0x7fbb54074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb527fc700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb54108140 0x7fbb54074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb527fc700 1 -- 172.21.15.8:0/2058300384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0095f0 con 0x7fbb54102a20 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb527fc700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54102a20 0x7fbb54074820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005910 tx=0x7fbb3c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:08.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c01d070 con 0x7fbb54102a20 2024-02-19T16:21:08.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb541a6ef0 con 0x7fbb54102a20 2024-02-19T16:21:08.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c00b5b0 con 0x7fbb54102a20 2024-02-19T16:21:08.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0173f0 con 0x7fbb54102a20 2024-02-19T16:21:08.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.805+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541a7310 con 0x7fbb54102a20 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.806+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb54112350 con 0x7fbb54102a20 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.807+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fbb3c0225e0 con 0x7fbb54102a20 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.807+0000 7fbb4bfff700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb400a21e0 0x7fbb400a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb52ffd700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb400a21e0 0x7fbb400a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbb3c0cf3c0 con 0x7fbb54102a20 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb4bfff700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] conn(0x7fbb400acea0 0x7fbb400af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 --> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb400af9b0 con 0x7fbb400acea0 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb3c0cf7d0 con 0x7fbb54102a20 2024-02-19T16:21:08.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb52ffd700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb400a21e0 0x7fbb400a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009be0 tx=0x7fbb4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:08.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb537fe700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] conn(0x7fbb400acea0 0x7fbb400af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:08.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.808+0000 7fbb537fe700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] conn(0x7fbb400acea0 0x7fbb400af2e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:08.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.810+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== osd.1 v2:172.21.15.156:6800/295266432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbb400af9b0 con 0x7fbb400acea0 2024-02-19T16:21:08.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 --> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb540626f0 con 0x7fbb400acea0 2024-02-19T16:21:08.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb4bfff700 1 -- 172.21.15.8:0/2058300384 <== osd.1 v2:172.21.15.156:6800/295266432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb540626f0 con 0x7fbb400acea0 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] conn(0x7fbb400acea0 msgr2=0x7fbb400af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] conn(0x7fbb400acea0 0x7fbb400af2e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb400a21e0 msgr2=0x7fbb400a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb400a21e0 0x7fbb400a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009be0 tx=0x7fbb4c009260 comp rx=0 tx=0).stop 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54102a20 msgr2=0x7fbb54074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.827+0000 7fbb596b0700 1 --2- 172.21.15.8:0/2058300384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54102a20 0x7fbb54074820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005910 tx=0x7fbb3c004730 comp rx=0 tx=0).stop 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.828+0000 7fbb537fe700 1 -- 172.21.15.8:0/2058300384 reap_dead start 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.828+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 shutdown_connections 2024-02-19T16:21:08.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.828+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 >> 172.21.15.8:0/2058300384 conn(0x7fbb540790a0 msgr2=0x7fbb54100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:08.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.828+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 shutdown_connections 2024-02-19T16:21:08.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:08.828+0000 7fbb596b0700 1 -- 172.21.15.8:0/2058300384 wait complete. 2024-02-19T16:21:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:09 smithi008 ceph-mon[32573]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-19T16:21:10.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:09 smithi191 ceph-mon[37203]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-19T16:21:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:09 smithi156 ceph-mon[36502]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 -- 172.21.15.8:0/1662492537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 msgr2=0x7f8d74074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 --2- 172.21.15.8:0/1662492537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d74074bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00b0a0 tx=0x7f8d6c00b3b0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 -- 172.21.15.8:0/1662492537 shutdown_connections 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 --2- 172.21.15.8:0/1662492537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d74112ca0 0x7f8d74115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 --2- 172.21.15.8:0/1662492537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d74075ff0 0x7f8d74074210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 --2- 172.21.15.8:0/1662492537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d74074bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 -- 172.21.15.8:0/1662492537 >> 172.21.15.8:0/1662492537 conn(0x7f8d74079070 msgr2=0x7f8d74079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 -- 172.21.15.8:0/1662492537 shutdown_connections 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.099+0000 7f8d78a10700 1 -- 172.21.15.8:0/1662492537 wait complete. 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 Processor -- start 2024-02-19T16:21:11.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 -- start start 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d741a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d74075ff0 0x7f8d741a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d74112ca0 0x7f8d7419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7419f0e0 con 0x7f8d74074750 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7419f220 con 0x7f8d74075ff0 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d78a10700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7419f3e0 con 0x7f8d74112ca0 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.100+0000 7f8d7259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d741a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d74075ff0 0x7f8d741a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d7259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d741a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34994/0 (socket says 172.21.15.8:34994) 2024-02-19T16:21:11.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d7259c700 1 -- 172.21.15.8:0/3817122004 learned_addr learned my addr 172.21.15.8:0/3817122004 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d72d9d700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d74112ca0 0x7f8d7419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 -- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d74112ca0 msgr2=0x7f8d7419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d74112ca0 0x7f8d7419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 -- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 msgr2=0x7f8d741a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d741a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 -- 172.21.15.8:0/3817122004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c009d00 con 0x7f8d74075ff0 2024-02-19T16:21:11.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d7259c700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d74074750 0x7f8d741a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:11.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d72d9d700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d74112ca0 0x7f8d7419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d71d9b700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d74075ff0 0x7f8d741a4e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00b070 tx=0x7f8d6c00bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00e040 con 0x7f8d74075ff0 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7419f660 con 0x7f8d74075ff0 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c0079d0 con 0x7f8d74075ff0 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.101+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d741ab960 con 0x7f8d74075ff0 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.102+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c005040 con 0x7f8d74075ff0 2024-02-19T16:21:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.102+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d54001030 con 0x7f8d74075ff0 2024-02-19T16:21:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.104+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f8d6c004a50 con 0x7f8d74075ff0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.105+0000 7f8d637fe700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8d5c0a21e0 0x7f8d5c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.105+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8d6c0cfa30 con 0x7f8d74075ff0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.105+0000 7f8d7259c700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8d5c0a21e0 0x7f8d5c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.105+0000 7f8d637fe700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] conn(0x7f8d5c0acea0 0x7f8d5c0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.105+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 --> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d5c0af9b0 con 0x7f8d5c0acea0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.105+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6c0cfe40 con 0x7f8d74075ff0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.106+0000 7f8d72d9d700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] conn(0x7f8d5c0acea0 0x7f8d5c0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.106+0000 7f8d7259c700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8d5c0a21e0 0x7f8d5c0a46a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d7419fbc0 tx=0x7f8d64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.106+0000 7f8d72d9d700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] conn(0x7f8d5c0acea0 0x7f8d5c0af2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.107+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== osd.5 v2:172.21.15.191:6808/499228077 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8d5c0af9b0 con 0x7f8d5c0acea0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 --> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d54002dc0 con 0x7f8d5c0acea0 2024-02-19T16:21:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d637fe700 1 -- 172.21.15.8:0/3817122004 <== osd.5 v2:172.21.15.191:6808/499228077 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8d54002dc0 con 0x7f8d5c0acea0 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] conn(0x7f8d5c0acea0 msgr2=0x7f8d5c0af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] conn(0x7f8d5c0acea0 0x7f8d5c0af2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8d5c0a21e0 msgr2=0x7f8d5c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8d5c0a21e0 0x7f8d5c0a46a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d7419fbc0 tx=0x7f8d64009260 comp rx=0 tx=0).stop 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d74075ff0 msgr2=0x7f8d741a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d78a10700 1 --2- 172.21.15.8:0/3817122004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d74075ff0 0x7f8d741a4e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00b070 tx=0x7f8d6c00bdd0 comp rx=0 tx=0).stop 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.125+0000 7f8d72d9d700 1 -- 172.21.15.8:0/3817122004 reap_dead start 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.126+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 shutdown_connections 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.126+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 >> 172.21.15.8:0/3817122004 conn(0x7f8d74079070 msgr2=0x7f8d74111900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.126+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 shutdown_connections 2024-02-19T16:21:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.126+0000 7f8d78a10700 1 -- 172.21.15.8:0/3817122004 wait complete. 2024-02-19T16:21:11.963 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:11 smithi008 ceph-mon[32573]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:12.275 INFO:teuthology.orchestra.run.smithi008.stdout:55834574921 2024-02-19T16:21:12.276 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-19T16:21:12.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:11 smithi191 ceph-mon[37203]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:11 smithi156 ceph-mon[36502]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:13.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 -- 172.21.15.8:0/3190216651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 msgr2=0x7ffabc100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:13.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 --2- 172.21.15.8:0/3190216651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc100a50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009990 tx=0x7ffab0009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 -- 172.21.15.8:0/3190216651 shutdown_connections 2024-02-19T16:21:13.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 --2- 172.21.15.8:0/3190216651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffabc100f90 0x7ffabc10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 --2- 172.21.15.8:0/3190216651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc107050 0x7ffabc107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 --2- 172.21.15.8:0/3190216651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc100a50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 -- 172.21.15.8:0/3190216651 >> 172.21.15.8:0/3190216651 conn(0x7ffabc0774e0 msgr2=0x7ffabc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 -- 172.21.15.8:0/3190216651 shutdown_connections 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.770+0000 7ffac45c9700 1 -- 172.21.15.8:0/3190216651 wait complete. 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 Processor -- start 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 -- start start 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffabc100f90 0x7ffabc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc107050 0x7ffabc102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a6a70 con 0x7ffabc1005d0 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a6be0 con 0x7ffabc107050 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac45c9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1a6da0 con 0x7ffabc100f90 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:13.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35020/0 (socket says 172.21.15.8:35020) 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 -- 172.21.15.8:0/2494468944 learned_addr learned my addr 172.21.15.8:0/2494468944 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac1b64700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffabc100f90 0x7ffabc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2b66700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc107050 0x7ffabc102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 -- 172.21.15.8:0/2494468944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffabc100f90 msgr2=0x7ffabc101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffabc100f90 0x7ffabc101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 -- 172.21.15.8:0/2494468944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc107050 msgr2=0x7ffabc102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc107050 0x7ffabc102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 -- 172.21.15.8:0/2494468944 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00095f0 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffac2365700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc1017b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffab400e9a0 tx=0x7ffab400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.771+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4009790 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.772+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1a7080 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.772+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab4010430 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.772+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4018680 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.772+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1a7580 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.772+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffabc1124b0 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7ffab4010910 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffabb7fe700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ffaa40a21e0 0x7ffaa40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffac1b64700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ffaa40a21e0 0x7ffaa40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ffab4014070 con 0x7ffabc1005d0 2024-02-19T16:21:13.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffabb7fe700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] conn(0x7ffaa40acea0 0x7ffaa40af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 --> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffaa40af9b0 con 0x7ffaa40acea0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.774+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab40cec70 con 0x7ffabc1005d0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.775+0000 7ffac2b66700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] conn(0x7ffaa40acea0 0x7ffaa40af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.775+0000 7ffac1b64700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ffaa40a21e0 0x7ffaa40a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0ffb40 tx=0x7ffab000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.775+0000 7ffac2b66700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] conn(0x7ffaa40acea0 0x7ffaa40af2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.776+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== osd.9 v2:172.21.15.8:6826/1070903169 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffaa40af9b0 con 0x7ffaa40acea0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.792+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 --> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffabc068ca0 con 0x7ffaa40acea0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.792+0000 7ffabb7fe700 1 -- 172.21.15.8:0/2494468944 <== osd.9 v2:172.21.15.8:6826/1070903169 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffabc068ca0 con 0x7ffaa40acea0 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] conn(0x7ffaa40acea0 msgr2=0x7ffaa40af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] conn(0x7ffaa40acea0 0x7ffaa40af2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ffaa40a21e0 msgr2=0x7ffaa40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ffaa40a21e0 0x7ffaa40a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0ffb40 tx=0x7ffab000b260 comp rx=0 tx=0).stop 2024-02-19T16:21:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 msgr2=0x7ffabc1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 --2- 172.21.15.8:0/2494468944 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffabc1005d0 0x7ffabc1017b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffab400e9a0 tx=0x7ffab400ecb0 comp rx=0 tx=0).stop 2024-02-19T16:21:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac2b66700 1 -- 172.21.15.8:0/2494468944 reap_dead start 2024-02-19T16:21:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 shutdown_connections 2024-02-19T16:21:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 >> 172.21.15.8:0/2494468944 conn(0x7ffabc0774e0 msgr2=0x7ffabc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 shutdown_connections 2024-02-19T16:21:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:11.793+0000 7ffac45c9700 1 -- 172.21.15.8:0/2494468944 wait complete. 2024-02-19T16:21:13.657 INFO:teuthology.orchestra.run.smithi008.stdout:292057776160 2024-02-19T16:21:13.657 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-19T16:21:13.911 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:13 smithi008 ceph-mon[32573]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:13 smithi191 ceph-mon[37203]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:13 smithi156 ceph-mon[36502]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:15.639 INFO:teuthology.orchestra.run.smithi008.stdout:257698037796 2024-02-19T16:21:15.639 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-19T16:21:15.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 -- 172.21.15.8:0/3046844902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 msgr2=0x7fdc2006a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:15.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 --2- 172.21.15.8:0/3046844902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc2006a0f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc140099c0 tx=0x7fdc14009cd0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 -- 172.21.15.8:0/3046844902 shutdown_connections 2024-02-19T16:21:15.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 --2- 172.21.15.8:0/3046844902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2006a630 0x7fdc2010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 --2- 172.21.15.8:0/3046844902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc20108030 0x7fdc20108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 --2- 172.21.15.8:0/3046844902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc2006a0f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 -- 172.21.15.8:0/3046844902 >> 172.21.15.8:0/3046844902 conn(0x7fdc20077e10 msgr2=0x7fdc20078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 -- 172.21.15.8:0/3046844902 shutdown_connections 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 -- 172.21.15.8:0/3046844902 wait complete. 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.477+0000 7fdc26364700 1 Processor -- start 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 -- start start 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc201984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2006a630 0x7fdc20198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc20108030 0x7fdc2019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2019d3a0 con 0x7fdc20069c90 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2019d510 con 0x7fdc2006a630 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc26364700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2019d6d0 con 0x7fdc20108030 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc201984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:15.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc201984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35038/0 (socket says 172.21.15.8:35038) 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 -- 172.21.15.8:0/2728126512 learned_addr learned my addr 172.21.15.8:0/2728126512 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1f7fe700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2006a630 0x7fdc20198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc24901700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc20108030 0x7fdc2019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 -- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc20108030 msgr2=0x7fdc2019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc20108030 0x7fdc2019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 -- 172.21.15.8:0/2728126512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2006a630 msgr2=0x7fdc20198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:15.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2006a630 0x7fdc20198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.478+0000 7fdc1ffff700 1 -- 172.21.15.8:0/2728126512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc140095f0 con 0x7fdc20069c90 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.479+0000 7fdc1ffff700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc201984c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000e9a0 tx=0x7fdc1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.479+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10009790 con 0x7fdc20069c90 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.479+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc10010430 con 0x7fdc20069c90 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.479+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10018680 con 0x7fdc20069c90 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.479+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2019d9b0 con 0x7fdc20069c90 2024-02-19T16:21:15.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.479+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc201a5210 con 0x7fdc20069c90 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.480+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc201036e0 con 0x7fdc20069c90 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.481+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fdc10010910 con 0x7fdc20069c90 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.481+0000 7fdc1d7fa700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc080a2260 0x7fdc080a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdc10014070 con 0x7fdc20069c90 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc1f7fe700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc080a2260 0x7fdc080a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc1d7fa700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] conn(0x7fdc080acf20 0x7fdc080af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 --> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc080afa30 con 0x7fdc080acf20 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc100ceb80 con 0x7fdc20069c90 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc24901700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] conn(0x7fdc080acf20 0x7fdc080af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc1f7fe700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc080a2260 0x7fdc080a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000c00 tx=0x7fdc14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.482+0000 7fdc24901700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] conn(0x7fdc080acf20 0x7fdc080af360 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.484+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== osd.2 v2:172.21.15.191:6800/3756928102 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdc080afa30 con 0x7fdc080acf20 2024-02-19T16:21:15.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.500+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 --> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc20112320 con 0x7fdc080acf20 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.500+0000 7fdc1d7fa700 1 -- 172.21.15.8:0/2728126512 <== osd.2 v2:172.21.15.191:6800/3756928102 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdc20112320 con 0x7fdc080acf20 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] conn(0x7fdc080acf20 msgr2=0x7fdc080af360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] conn(0x7fdc080acf20 0x7fdc080af360 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc080a2260 msgr2=0x7fdc080a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdc080a2260 0x7fdc080a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000c00 tx=0x7fdc14005fb0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 msgr2=0x7fdc201984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 --2- 172.21.15.8:0/2728126512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc20069c90 0x7fdc201984c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000e9a0 tx=0x7fdc1000ecb0 comp rx=0 tx=0).stop 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc24901700 1 -- 172.21.15.8:0/2728126512 reap_dead start 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 shutdown_connections 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 >> 172.21.15.8:0/2728126512 conn(0x7fdc20077e10 msgr2=0x7fdc200ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:15.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 shutdown_connections 2024-02-19T16:21:15.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:12.501+0000 7fdc26364700 1 -- 172.21.15.8:0/2728126512 wait complete. 2024-02-19T16:21:16.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:15 smithi008 ceph-mon[32573]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:16.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:15 smithi191 ceph-mon[37203]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:15 smithi156 ceph-mon[36502]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:21:16.439 INFO:teuthology.orchestra.run.smithi008.stdout:77309411397 2024-02-19T16:21:16.439 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-19T16:21:16.850 INFO:teuthology.orchestra.run.smithi008.stdout:34359738445 2024-02-19T16:21:16.851 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-19T16:21:16.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 -- 172.21.15.8:0/3258699292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b41029f0 msgr2=0x7f43b410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:16.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/3258699292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b41029f0 0x7f43b410ee30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00b0d0 tx=0x7f43ac00b3e0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 -- 172.21.15.8:0/3258699292 shutdown_connections 2024-02-19T16:21:16.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/3258699292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b41029f0 0x7f43b410ee30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/3258699292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b4108010 0x7f43b41083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/3258699292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43b4102030 0x7f43b41024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 -- 172.21.15.8:0/3258699292 >> 172.21.15.8:0/3258699292 conn(0x7f43b4079110 msgr2=0x7f43b4079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 -- 172.21.15.8:0/3258699292 shutdown_connections 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.111+0000 7f43bbb5e700 1 -- 172.21.15.8:0/3258699292 wait complete. 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 Processor -- start 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 -- start start 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b4102030 0x7f43b41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b41029f0 0x7f43b4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43b4108010 0x7f43b419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b419d530 con 0x7f43b4102030 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b419d6a0 con 0x7f43b41029f0 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43bbb5e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b419d860 con 0x7f43b4108010 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b4102030 0x7f43b41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:16.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b4102030 0x7f43b41986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60518/0 (socket says 172.21.15.8:60518) 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 -- 172.21.15.8:0/2484054724 learned_addr learned my addr 172.21.15.8:0/2484054724 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b90f9700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b41029f0 0x7f43b4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43ba0fb700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43b4108010 0x7f43b419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 -- 172.21.15.8:0/2484054724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43b4108010 msgr2=0x7f43b419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43b4108010 0x7f43b419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 -- 172.21.15.8:0/2484054724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b41029f0 msgr2=0x7f43b4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b41029f0 0x7f43b4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43b98fa700 1 -- 172.21.15.8:0/2484054724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac009d00 con 0x7f43b4102030 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.112+0000 7f43ba0fb700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43b4108010 0x7f43b419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43b90f9700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43b41029f0 0x7f43b4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43b98fa700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b4102030 0x7f43b41986a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43a400e9a0 tx=0x7f43a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4009790 con 0x7f43b4102030 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43a4010430 con 0x7f43b4102030 2024-02-19T16:21:16.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b419db40 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4018680 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.113+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b41a53a0 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.114+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43b41122e0 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.115+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f43a4010910 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43aaffd700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f43a00a21e0 0x7f43a00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43b90f9700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f43a00a21e0 0x7f43a00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f43a4014070 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43aaffd700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] conn(0x7f43a00acea0 0x7f43a00af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 --> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43a00af9b0 con 0x7f43a00acea0 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a40cec30 con 0x7f43b4102030 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43ba0fb700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] conn(0x7f43a00acea0 0x7f43a00af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:16.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43b90f9700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f43a00a21e0 0x7f43a00a46a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43b40796f0 tx=0x7f43b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.116+0000 7f43ba0fb700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] conn(0x7f43a00acea0 0x7f43a00af2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.117+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== osd.3 v2:172.21.15.8:6810/1491647664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f43a00af9b0 con 0x7f43a00acea0 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.135+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 --> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f43b40626f0 con 0x7f43a00acea0 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.135+0000 7f43aaffd700 1 -- 172.21.15.8:0/2484054724 <== osd.3 v2:172.21.15.8:6810/1491647664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f43b40626f0 con 0x7f43a00acea0 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] conn(0x7f43a00acea0 msgr2=0x7f43a00af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] conn(0x7f43a00acea0 0x7f43a00af2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f43a00a21e0 msgr2=0x7f43a00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f43a00a21e0 0x7f43a00a46a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43b40796f0 tx=0x7f43b0009260 comp rx=0 tx=0).stop 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b4102030 msgr2=0x7f43b41986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 --2- 172.21.15.8:0/2484054724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43b4102030 0x7f43b41986a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43a400e9a0 tx=0x7f43a400ecb0 comp rx=0 tx=0).stop 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43ba0fb700 1 -- 172.21.15.8:0/2484054724 reap_dead start 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 shutdown_connections 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 >> 172.21.15.8:0/2484054724 conn(0x7f43b4079110 msgr2=0x7f43b4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 shutdown_connections 2024-02-19T16:21:16.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:16.136+0000 7f43bbb5e700 1 -- 172.21.15.8:0/2484054724 wait complete. 2024-02-19T16:21:17.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 -- 172.21.15.8:0/2205737517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 msgr2=0x7fb0a4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:17.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/2205737517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a4102510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb098009990 tx=0x7fb098009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 -- 172.21.15.8:0/2205737517 shutdown_connections 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/2205737517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0a4102a50 0x7fb0a410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/2205737517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0a4108100 0x7fb0a41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/2205737517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a4102510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 -- 172.21.15.8:0/2205737517 >> 172.21.15.8:0/2205737517 conn(0x7fb0a40790b0 msgr2=0x7fb0a40794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 -- 172.21.15.8:0/2205737517 shutdown_connections 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.692+0000 7fb0abc1e700 1 -- 172.21.15.8:0/2205737517 wait complete. 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 Processor -- start 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 -- start start 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0a4102a50 0x7fb0a4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0a4108100 0x7fb0a419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:17.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a419d460 con 0x7fb0a4102090 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a419d5d0 con 0x7fb0a4102a50 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0abc1e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a419d790 con 0x7fb0a4108100 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0a99ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0a99ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60532/0 (socket says 172.21.15.8:60532) 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.693+0000 7fb0a99ba700 1 -- 172.21.15.8:0/130556849 learned_addr learned my addr 172.21.15.8:0/130556849 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a91b9700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0a4102a50 0x7fb0a4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0aa1bb700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0a4108100 0x7fb0a419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a99ba700 1 -- 172.21.15.8:0/130556849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0a4108100 msgr2=0x7fb0a419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a99ba700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0a4108100 0x7fb0a419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a99ba700 1 -- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0a4102a50 msgr2=0x7fb0a4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a99ba700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0a4102a50 0x7fb0a4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a99ba700 1 -- 172.21.15.8:0/130556849 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0980095f0 con 0x7fb0a4102090 2024-02-19T16:21:17.697 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a99ba700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a41985d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb09400ef10 tx=0x7fb09400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:17.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400c9c0 con 0x7fb0a4102090 2024-02-19T16:21:17.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a419da10 con 0x7fb0a4102090 2024-02-19T16:21:17.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb09400f040 con 0x7fb0a4102090 2024-02-19T16:21:17.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094014480 con 0x7fb0a4102090 2024-02-19T16:21:17.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.694+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a41a52d0 con 0x7fb0a4102090 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.695+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0a4112370 con 0x7fb0a4102090 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.696+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb094007720 con 0x7fb0a4102090 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0a2ffd700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb08c0a2260 0x7fb08c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb0940cece0 con 0x7fb0a4102090 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0a91b9700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb08c0a2260 0x7fb08c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0a2ffd700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] conn(0x7fb08c0acf20 0x7fb08c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:17.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 --> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb08c0afa30 con 0x7fb08c0acf20 2024-02-19T16:21:17.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0940cf0f0 con 0x7fb0a4102090 2024-02-19T16:21:17.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.697+0000 7fb0aa1bb700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] conn(0x7fb08c0acf20 0x7fb08c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:17.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.698+0000 7fb0a91b9700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb08c0a2260 0x7fb08c0a4720 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb09800b2e0 tx=0x7fb098005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:17.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.698+0000 7fb0aa1bb700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] conn(0x7fb08c0acf20 0x7fb08c0af360 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:17.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.699+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== osd.7 v2:172.21.15.156:6816/3871421578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb08c0afa30 con 0x7fb08c0acf20 2024-02-19T16:21:17.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.717+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 --> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0a40626f0 con 0x7fb08c0acf20 2024-02-19T16:21:17.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0a2ffd700 1 -- 172.21.15.8:0/130556849 <== osd.7 v2:172.21.15.156:6816/3871421578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb0a40626f0 con 0x7fb08c0acf20 2024-02-19T16:21:17.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] conn(0x7fb08c0acf20 msgr2=0x7fb08c0af360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] conn(0x7fb08c0acf20 0x7fb08c0af360 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb08c0a2260 msgr2=0x7fb08c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb08c0a2260 0x7fb08c0a4720 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb09800b2e0 tx=0x7fb098005960 comp rx=0 tx=0).stop 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 msgr2=0x7fb0a41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 --2- 172.21.15.8:0/130556849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0a4102090 0x7fb0a41985d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb09400ef10 tx=0x7fb09400c3a0 comp rx=0 tx=0).stop 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0aa1bb700 1 -- 172.21.15.8:0/130556849 reap_dead start 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 shutdown_connections 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 >> 172.21.15.8:0/130556849 conn(0x7fb0a40790b0 msgr2=0x7fb0a4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.718+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 shutdown_connections 2024-02-19T16:21:17.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:17.719+0000 7fb0abc1e700 1 -- 172.21.15.8:0/130556849 wait complete. 2024-02-19T16:21:18.161 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:17 smithi008 ceph-mon[32573]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:21:18.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:17 smithi191 ceph-mon[37203]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:21:18.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:17 smithi156 ceph-mon[36502]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:21:18.929 INFO:teuthology.orchestra.run.smithi008.stdout:167503724596 2024-02-19T16:21:18.929 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 -- 172.21.15.8:0/848840041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc106260 msgr2=0x7fb3dc10de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 --2- 172.21.15.8:0/848840041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc106260 0x7fb3dc10de30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400b0a0 tx=0x7fb3d400b3b0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 -- 172.21.15.8:0/848840041 shutdown_connections 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 --2- 172.21.15.8:0/848840041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc106260 0x7fb3dc10de30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 --2- 172.21.15.8:0/848840041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3dc101940 0x7fb3dc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 --2- 172.21.15.8:0/848840041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3dc1022f0 0x7fb3dc105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 -- 172.21.15.8:0/848840041 >> 172.21.15.8:0/848840041 conn(0x7fb3dc0774e0 msgr2=0x7fb3dc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:19.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 -- 172.21.15.8:0/848840041 shutdown_connections 2024-02-19T16:21:19.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.129+0000 7fb3e3856700 1 -- 172.21.15.8:0/848840041 wait complete. 2024-02-19T16:21:19.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 Processor -- start 2024-02-19T16:21:19.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 -- start start 2024-02-19T16:21:19.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3dc101940 0x7fb3dc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc1022f0 0x7fb3dc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3dc106260 0x7fb3dc19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc19c470 con 0x7fb3dc1022f0 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc19c5e0 con 0x7fb3dc106260 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e3856700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc19c7a0 con 0x7fb3dc101940 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e0df1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc1022f0 0x7fb3dc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e0df1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc1022f0 0x7fb3dc197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60558/0 (socket says 172.21.15.8:60558) 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e0df1700 1 -- 172.21.15.8:0/2969138731 learned_addr learned my addr 172.21.15.8:0/2969138731 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e1df3700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3dc106260 0x7fb3dc19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.130+0000 7fb3e15f2700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3dc101940 0x7fb3dc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e0df1700 1 -- 172.21.15.8:0/2969138731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3dc101940 msgr2=0x7fb3dc197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e0df1700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3dc101940 0x7fb3dc197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e0df1700 1 -- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3dc106260 msgr2=0x7fb3dc19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e0df1700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3dc106260 0x7fb3dc19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e0df1700 1 -- 172.21.15.8:0/2969138731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4009d00 con 0x7fb3dc1022f0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e1df3700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3dc106260 0x7fb3dc19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e0df1700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc1022f0 0x7fb3dc197ad0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00ea80 tx=0x7fb3cc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc013070 con 0x7fb3dc1022f0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc19ca80 con 0x7fb3dc1022f0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3cc004830 con 0x7fb3dc1022f0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0053e0 con 0x7fb3dc1022f0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.131+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc1007a0 con 0x7fb3dc1022f0 2024-02-19T16:21:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.132+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3dc111250 con 0x7fb3dc1022f0 2024-02-19T16:21:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.133+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb3cc0049a0 con 0x7fb3dc1022f0 2024-02-19T16:21:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3d27fc700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c80a21e0 0x7fb3c80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb3cc0ce620 con 0x7fb3dc1022f0 2024-02-19T16:21:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3e15f2700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c80a21e0 0x7fb3c80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3d27fc700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] conn(0x7fb3c80acea0 0x7fb3c80af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 --> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3c80af9b0 con 0x7fb3c80acea0 2024-02-19T16:21:19.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3cc0ce990 con 0x7fb3dc1022f0 2024-02-19T16:21:19.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.134+0000 7fb3e1df3700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] conn(0x7fb3c80acea0 0x7fb3c80af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.135+0000 7fb3e15f2700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c80a21e0 0x7fb3c80a46a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009c60 tx=0x7fb3d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:19.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.135+0000 7fb3e1df3700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] conn(0x7fb3c80acea0 0x7fb3c80af2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:19.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.136+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== osd.4 v2:172.21.15.156:6808/3469489229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb3c80af9b0 con 0x7fb3c80acea0 2024-02-19T16:21:19.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.152+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 --> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3dc100b80 con 0x7fb3c80acea0 2024-02-19T16:21:19.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3d27fc700 1 -- 172.21.15.8:0/2969138731 <== osd.4 v2:172.21.15.156:6808/3469489229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb3dc100b80 con 0x7fb3c80acea0 2024-02-19T16:21:19.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] conn(0x7fb3c80acea0 msgr2=0x7fb3c80af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] conn(0x7fb3c80acea0 0x7fb3c80af2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c80a21e0 msgr2=0x7fb3c80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb3c80a21e0 0x7fb3c80a46a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009c60 tx=0x7fb3d8009260 comp rx=0 tx=0).stop 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc1022f0 msgr2=0x7fb3dc197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 --2- 172.21.15.8:0/2969138731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3dc1022f0 0x7fb3dc197ad0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00ea80 tx=0x7fb3cc00ed90 comp rx=0 tx=0).stop 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e1df3700 1 -- 172.21.15.8:0/2969138731 reap_dead start 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 shutdown_connections 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.153+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 >> 172.21.15.8:0/2969138731 conn(0x7fb3dc0774e0 msgr2=0x7fb3dc1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.154+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 shutdown_connections 2024-02-19T16:21:19.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:19.154+0000 7fb3e3856700 1 -- 172.21.15.8:0/2969138731 wait complete. 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.707+0000 7f189fe7c700 1 -- 172.21.15.8:0/1027323113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 msgr2=0x7f189806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.707+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1027323113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f189806aac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1894009990 tx=0x7f1894009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.707+0000 7f189fe7c700 1 -- 172.21.15.8:0/1027323113 shutdown_connections 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.707+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1027323113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f189810eac0 0x7f1898110eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1027323113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f189806aac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1027323113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189806b000 0x7f189810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 -- 172.21.15.8:0/1027323113 >> 172.21.15.8:0/1027323113 conn(0x7f1898076f00 msgr2=0x7f1898077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 -- 172.21.15.8:0/1027323113 shutdown_connections 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 -- 172.21.15.8:0/1027323113 wait complete. 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 Processor -- start 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.708+0000 7f189fe7c700 1 -- start start 2024-02-19T16:21:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f18981a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189806b000 0x7f18981a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f189810eac0 0x7f189819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189819ac20 con 0x7f189806a6e0 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189819ad60 con 0x7f189810eac0 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189819af20 con 0x7f189806b000 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f18981a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f18981a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60544/0 (socket says 172.21.15.8:60544) 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 -- 172.21.15.8:0/1668906767 learned_addr learned my addr 172.21.15.8:0/1668906767 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189d417700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189806b000 0x7f18981a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189e419700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f189810eac0 0x7f189819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 -- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189806b000 msgr2=0x7f18981a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189806b000 0x7f18981a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 -- 172.21.15.8:0/1668906767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f189810eac0 msgr2=0x7f189819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f189810eac0 0x7f189819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 -- 172.21.15.8:0/1668906767 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940095f0 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189dc18700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f18981a0400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189401d070 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.709+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189819b1a0 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.710+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f189400b610 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.710+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940173f0 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.710+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18981a74a0 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.710+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1898114570 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f1894022690 con 0x7f189806a6e0 2024-02-19T16:21:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f188effd700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18840a21e0 0x7f18840a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f189d417700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18840a21e0 0x7f18840a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f18940cfd40 con 0x7f189806a6e0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f188effd700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] conn(0x7f18840acea0 0x7f18840af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 --> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18840af9b0 con 0x7f18840acea0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18940d0150 con 0x7f189806a6e0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.712+0000 7f189d417700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18840a21e0 0x7f18840a46a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1898102490 tx=0x7f1888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.713+0000 7f189e419700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] conn(0x7f18840acea0 0x7f18840af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.713+0000 7f189e419700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] conn(0x7f18840acea0 0x7f18840af2e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.714+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== osd.11 v2:172.21.15.191:6824/759584656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f18840af9b0 con 0x7f18840acea0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.730+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 --> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1898062670 con 0x7f18840acea0 2024-02-19T16:21:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.730+0000 7f188effd700 1 -- 172.21.15.8:0/1668906767 <== osd.11 v2:172.21.15.191:6824/759584656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1898062670 con 0x7f18840acea0 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] conn(0x7f18840acea0 msgr2=0x7f18840af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] conn(0x7f18840acea0 0x7f18840af2e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18840a21e0 msgr2=0x7f18840a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f18840a21e0 0x7f18840a46a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1898102490 tx=0x7f1888009260 comp rx=0 tx=0).stop 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 msgr2=0x7f18981a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 --2- 172.21.15.8:0/1668906767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f189806a6e0 0x7f18981a0400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894005740 comp rx=0 tx=0).stop 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189e419700 1 -- 172.21.15.8:0/1668906767 reap_dead start 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 shutdown_connections 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 >> 172.21.15.8:0/1668906767 conn(0x7f1898076f00 msgr2=0x7f1898110110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 shutdown_connections 2024-02-19T16:21:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:18.731+0000 7f189fe7c700 1 -- 172.21.15.8:0/1668906767 wait complete. 2024-02-19T16:21:20.155 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:19 smithi008 ceph-mon[32573]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:20.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 -- 172.21.15.8:0/1615093454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 msgr2=0x7f42c010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 --2- 172.21.15.8:0/1615093454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c010ee70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f42b000b0d0 tx=0x7f42b000b3e0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 -- 172.21.15.8:0/1615093454 shutdown_connections 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 --2- 172.21.15.8:0/1615093454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c010ee70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 --2- 172.21.15.8:0/1615093454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0108030 0x7f42c0108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 --2- 172.21.15.8:0/1615093454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0069c90 0x7f42c006a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 -- 172.21.15.8:0/1615093454 >> 172.21.15.8:0/1615093454 conn(0x7f42c0077e10 msgr2=0x7f42c0078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 -- 172.21.15.8:0/1615093454 shutdown_connections 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.166+0000 7f42c7798700 1 -- 172.21.15.8:0/1615093454 wait complete. 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 Processor -- start 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 -- start start 2024-02-19T16:21:20.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0069c90 0x7f42c01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0108030 0x7f42c019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019d560 con 0x7f42c006a630 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019d6d0 con 0x7f42c0069c90 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c7798700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019d890 con 0x7f42c0108030 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60578/0 (socket says 172.21.15.8:60578) 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 -- 172.21.15.8:0/228244369 learned_addr learned my addr 172.21.15.8:0/228244369 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c5d35700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0108030 0x7f42c019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:20.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c5534700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0069c90 0x7f42c01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:20.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 -- 172.21.15.8:0/228244369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0108030 msgr2=0x7f42c019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:20.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0108030 0x7f42c019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 -- 172.21.15.8:0/228244369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0069c90 msgr2=0x7f42c01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:20.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0069c90 0x7f42c01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c4d33700 1 -- 172.21.15.8:0/228244369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0009d00 con 0x7f42c006a630 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c5534700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0069c90 0x7f42c01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.167+0000 7f42c5d35700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0108030 0x7f42c019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.168+0000 7f42c4d33700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c0198c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9a0 tx=0x7f42b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.168+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4009790 con 0x7f42c006a630 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.168+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b4010430 con 0x7f42c006a630 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.168+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c019db70 con 0x7f42c006a630 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.168+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4018680 con 0x7f42c006a630 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.168+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c01a53d0 con 0x7f42c006a630 2024-02-19T16:21:20.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.169+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42c0112290 con 0x7f42c006a630 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.170+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f42b4010910 con 0x7f42c006a630 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42be7fc700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42a80a21e0 0x7f42a80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42c5534700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42a80a21e0 0x7f42a80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f42b4014070 con 0x7f42c006a630 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42be7fc700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] conn(0x7f42a80acea0 0x7f42a80af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 --> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42a80af9b0 con 0x7f42a80acea0 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42b40cecb0 con 0x7f42c006a630 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42c5d35700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] conn(0x7f42a80acea0 0x7f42a80af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:20.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42c5534700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42a80a21e0 0x7f42a80a46a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009fd0 tx=0x7f42b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:20.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.171+0000 7f42c5d35700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] conn(0x7f42a80acea0 0x7f42a80af2e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:20.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.172+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== osd.6 v2:172.21.15.8:6818/2596853517 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42a80af9b0 con 0x7f42a80acea0 2024-02-19T16:21:20.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.190+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 --> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42c0062670 con 0x7f42a80acea0 2024-02-19T16:21:20.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.190+0000 7f42be7fc700 1 -- 172.21.15.8:0/228244369 <== osd.6 v2:172.21.15.8:6818/2596853517 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f42c0062670 con 0x7f42a80acea0 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] conn(0x7f42a80acea0 msgr2=0x7f42a80af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] conn(0x7f42a80acea0 0x7f42a80af2e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42a80a21e0 msgr2=0x7f42a80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f42a80a21e0 0x7f42a80a46a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009fd0 tx=0x7f42b8009260 comp rx=0 tx=0).stop 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 msgr2=0x7f42c0198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 --2- 172.21.15.8:0/228244369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f42c006a630 0x7f42c0198c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9a0 tx=0x7f42b400ecb0 comp rx=0 tx=0).stop 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c5d35700 1 -- 172.21.15.8:0/228244369 reap_dead start 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 shutdown_connections 2024-02-19T16:21:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 >> 172.21.15.8:0/228244369 conn(0x7f42c0077e10 msgr2=0x7f42c00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:20.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 shutdown_connections 2024-02-19T16:21:20.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:20.191+0000 7f42c7798700 1 -- 172.21.15.8:0/228244369 wait complete. 2024-02-19T16:21:20.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:19 smithi191 ceph-mon[37203]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:20.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:19 smithi156 ceph-mon[36502]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:20.769 INFO:teuthology.orchestra.run.smithi008.stdout:111669149760 2024-02-19T16:21:20.770 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-19T16:21:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-02-19T16:21:21.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-02-19T16:21:21.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:20 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-02-19T16:21:21.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-02-19T16:21:21.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:20 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-02-19T16:21:21.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-02-19T16:21:21.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-02-19T16:21:21.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-19T16:21:21.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-02-19T16:21:21.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-02-19T16:21:21.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-02-19T16:21:21.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-02-19T16:21:21.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-02-19T16:21:21.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:20 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-02-19T16:21:22.085 INFO:teuthology.orchestra.run.smithi008.stdout:146028888124 2024-02-19T16:21:22.085 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-02-19T16:21:22.168 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-02-19T16:21:22.169 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-02-19T16:21:22.169 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-02-19T16:21:22.169 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:21 smithi008 ceph-mon[32573]: osdmap e92: 12 total, 12 up, 12 in 2024-02-19T16:21:22.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-02-19T16:21:22.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-02-19T16:21:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:21 smithi191 ceph-mon[37203]: osdmap e92: 12 total, 12 up, 12 in 2024-02-19T16:21:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-02-19T16:21:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-02-19T16:21:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:21 smithi156 ceph-mon[36502]: osdmap e92: 12 total, 12 up, 12 in 2024-02-19T16:21:22.506 INFO:teuthology.orchestra.run.smithi008.stdout:201863462962 2024-02-19T16:21:22.506 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-19T16:21:22.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 -- 172.21.15.8:0/3456945382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9880069c90 msgr2=0x7f988006a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:22.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 --2- 172.21.15.8:0/3456945382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9880069c90 0x7f988006a0f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98700099c0 tx=0x7f9870009cd0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 -- 172.21.15.8:0/3456945382 shutdown_connections 2024-02-19T16:21:22.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 --2- 172.21.15.8:0/3456945382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f988006a630 0x7f988010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 --2- 172.21.15.8:0/3456945382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9880108030 0x7f9880108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 --2- 172.21.15.8:0/3456945382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9880069c90 0x7f988006a0f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 -- 172.21.15.8:0/3456945382 >> 172.21.15.8:0/3456945382 conn(0x7f9880077e10 msgr2=0x7f9880078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:22.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 -- 172.21.15.8:0/3456945382 shutdown_connections 2024-02-19T16:21:22.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.707+0000 7f9885ef8700 1 -- 172.21.15.8:0/3456945382 wait complete. 2024-02-19T16:21:22.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 Processor -- start 2024-02-19T16:21:22.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 -- start start 2024-02-19T16:21:22.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9880069c90 0x7f9880198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:22.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f988006a630 0x7f9880198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:22.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9880108030 0x7f988019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:22.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988019d3f0 con 0x7f988006a630 2024-02-19T16:21:22.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988019d560 con 0x7f9880108030 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f9885ef8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988019d720 con 0x7f9880069c90 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f987effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f988006a630 0x7f9880198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f987effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f988006a630 0x7f9880198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60602/0 (socket says 172.21.15.8:60602) 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f987effd700 1 -- 172.21.15.8:0/331401673 learned_addr learned my addr 172.21.15.8:0/331401673 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.708+0000 7f987f7fe700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9880069c90 0x7f9880198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987ffff700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9880108030 0x7f988019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987effd700 1 -- 172.21.15.8:0/331401673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9880069c90 msgr2=0x7f9880198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987effd700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9880069c90 0x7f9880198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987effd700 1 -- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9880108030 msgr2=0x7f988019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987effd700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9880108030 0x7f988019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987effd700 1 -- 172.21.15.8:0/331401673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700095f0 con 0x7f988006a630 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987effd700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f988006a630 0x7f9880198a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9870009990 tx=0x7f9870004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987001d070 con 0x7f988006a630 2024-02-19T16:21:22.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988019d9a0 con 0x7f988006a630 2024-02-19T16:21:22.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98801a5260 con 0x7f988006a630 2024-02-19T16:21:22.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9870004cc0 con 0x7f988006a630 2024-02-19T16:21:22.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.709+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870003cd0 con 0x7f988006a630 2024-02-19T16:21:22.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.710+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98801036e0 con 0x7f988006a630 2024-02-19T16:21:22.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.711+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f98700050c0 con 0x7f988006a630 2024-02-19T16:21:22.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.712+0000 7f987cff9700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98680a2260 0x7f98680a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:22.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.712+0000 7f987f7fe700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98680a2260 0x7f98680a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:22.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.712+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f98700cf820 con 0x7f988006a630 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.712+0000 7f987cff9700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] conn(0x7f98680ad340 0x7f98680af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.712+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 --> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98680afeb0 con 0x7f98680ad340 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.712+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98700cfc30 con 0x7f988006a630 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.713+0000 7f987ffff700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] conn(0x7f98680ad340 0x7f98680af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.713+0000 7f987f7fe700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98680a2260 0x7f98680a4720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9874007b60 tx=0x7f98740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.713+0000 7f987ffff700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] conn(0x7f98680ad340 0x7f98680af7e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:22.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.714+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== osd.10 v2:172.21.15.156:6824/1977244529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f98680afeb0 con 0x7f98680ad340 2024-02-19T16:21:22.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.733+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 --> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9880112370 con 0x7f98680ad340 2024-02-19T16:21:22.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.733+0000 7f987cff9700 1 -- 172.21.15.8:0/331401673 <== osd.10 v2:172.21.15.156:6824/1977244529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9880112370 con 0x7f98680ad340 2024-02-19T16:21:22.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.734+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] conn(0x7f98680ad340 msgr2=0x7f98680af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:22.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.734+0000 7f9885ef8700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] conn(0x7f98680ad340 0x7f98680af7e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.734+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98680a2260 msgr2=0x7f98680a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:22.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.734+0000 7f9885ef8700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98680a2260 0x7f98680a4720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9874007b60 tx=0x7f98740073d0 comp rx=0 tx=0).stop 2024-02-19T16:21:22.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.734+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f988006a630 msgr2=0x7f9880198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:22.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.734+0000 7f9885ef8700 1 --2- 172.21.15.8:0/331401673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f988006a630 0x7f9880198a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9870009990 tx=0x7f9870004730 comp rx=0 tx=0).stop 2024-02-19T16:21:22.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.737+0000 7f987ffff700 1 -- 172.21.15.8:0/331401673 reap_dead start 2024-02-19T16:21:22.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.737+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 shutdown_connections 2024-02-19T16:21:22.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.737+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 >> 172.21.15.8:0/331401673 conn(0x7f9880077e10 msgr2=0x7f98800ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:22.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.739+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 shutdown_connections 2024-02-19T16:21:22.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:22.739+0000 7f9885ef8700 1 -- 172.21.15.8:0/331401673 wait complete. 2024-02-19T16:21:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:22 smithi008 ceph-mon[32573]: osdmap e93: 12 total, 12 up, 12 in 2024-02-19T16:21:23.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:21:23.143 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:22 smithi191 ceph-mon[37203]: osdmap e93: 12 total, 12 up, 12 in 2024-02-19T16:21:23.143 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:21:23.175 INFO:teuthology.orchestra.run.smithi008.stdout:223338299437 2024-02-19T16:21:23.176 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-19T16:21:23.193 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:22 smithi156 ceph-mon[36502]: osdmap e93: 12 total, 12 up, 12 in 2024-02-19T16:21:23.193 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:21:23.607 INFO:teuthology.orchestra.run.smithi008.stdout:347892350999 2024-02-19T16:21:23.607 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-19T16:21:24.256 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:23 smithi008 ceph-mon[32573]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-19T16:21:24.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:23 smithi191 ceph-mon[37203]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-19T16:21:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:313532612638 2024-02-19T16:21:24.283 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-19T16:21:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:24.306 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:24.306 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:24.318 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:24.327 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:24.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:23 smithi156 ceph-mon[36502]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-19T16:21:25.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:24 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:24 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:25.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:24 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:26.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:26 smithi156 ceph-mon[36502]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-19T16:21:26.371 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:26 smithi008 ceph-mon[32573]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-19T16:21:26.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:26 smithi191 ceph-mon[37203]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-19T16:21:28.225 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:28 smithi008 ceph-mon[32573]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-02-19T16:21:28.238 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:28.245 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:28.246 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:28.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:28 smithi156 ceph-mon[36502]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-02-19T16:21:28.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:28 smithi191 ceph-mon[37203]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-02-19T16:21:29.734 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:29.736 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:30.222 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:30 smithi008 ceph-mon[32573]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-02-19T16:21:30.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:30 smithi156 ceph-mon[36502]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-02-19T16:21:30.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:30 smithi191 ceph-mon[37203]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-02-19T16:21:31.254 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:31 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1717541917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-19T16:21:31.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:31 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1717541917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-19T16:21:31.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 -- 172.21.15.8:0/2945220629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc1005d0 msgr2=0x7f72fc100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:31.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 --2- 172.21.15.8:0/2945220629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc1005d0 0x7f72fc100a50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009990 tx=0x7f72f0009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 -- 172.21.15.8:0/2945220629 shutdown_connections 2024-02-19T16:21:31.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 --2- 172.21.15.8:0/2945220629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc100f90 0x7f72fc10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 --2- 172.21.15.8:0/2945220629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc107050 0x7f72fc107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 --2- 172.21.15.8:0/2945220629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc1005d0 0x7f72fc100a50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 -- 172.21.15.8:0/2945220629 >> 172.21.15.8:0/2945220629 conn(0x7f72fc0774e0 msgr2=0x7f72fc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:31.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 -- 172.21.15.8:0/2945220629 shutdown_connections 2024-02-19T16:21:31.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.324+0000 7f7301abb700 1 -- 172.21.15.8:0/2945220629 wait complete. 2024-02-19T16:21:31.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 Processor -- start 2024-02-19T16:21:31.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 -- start start 2024-02-19T16:21:31.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc1005d0 0x7f72fc1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 0x7f72fc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc107050 0x7f72fc102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc1a6a20 con 0x7f72fc100f90 2024-02-19T16:21:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc1a6b90 con 0x7f72fc1005d0 2024-02-19T16:21:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f7301abb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc1a6d50 con 0x7f72fc107050 2024-02-19T16:21:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f72faffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 0x7f72fc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f72faffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 0x7f72fc101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35092/0 (socket says 172.21.15.8:35092) 2024-02-19T16:21:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f72faffd700 1 -- 172.21.15.8:0/1717541917 learned_addr learned my addr 172.21.15.8:0/1717541917 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f72fb7fe700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc1005d0 0x7f72fc1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:31.358 INFO:teuthology.orchestra.run.smithi008.stdout:55834574925 2024-02-19T16:21:31.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.325+0000 7f72fbfff700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc107050 0x7f72fc102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:31.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72fb7fe700 1 -- 172.21.15.8:0/1717541917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc107050 msgr2=0x7f72fc102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:31.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72fb7fe700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc107050 0x7f72fc102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72fb7fe700 1 -- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 msgr2=0x7f72fc101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:31.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72fb7fe700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 0x7f72fc101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72fb7fe700 1 -- 172.21.15.8:0/1717541917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f00095f0 con 0x7f72fc1005d0 2024-02-19T16:21:31.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72faffd700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 0x7f72fc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:31.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72fb7fe700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc1005d0 0x7f72fc1017b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e9f0 tx=0x7f72ec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:31.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00c9c0 con 0x7f72fc1005d0 2024-02-19T16:21:31.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc1a6fd0 con 0x7f72fc1005d0 2024-02-19T16:21:31.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc1a74f0 con 0x7f72fc1005d0 2024-02-19T16:21:31.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72ec010430 con 0x7f72fc1005d0 2024-02-19T16:21:31.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.326+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec018680 con 0x7f72fc1005d0 2024-02-19T16:21:31.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.327+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e8005330 con 0x7f72fc1005d0 2024-02-19T16:21:31.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.328+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f72ec00cb20 con 0x7f72fc1005d0 2024-02-19T16:21:31.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.329+0000 7f72f8ff9700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f72e40a21e0 0x7f72e40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:31.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.329+0000 7f72faffd700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f72e40a21e0 0x7f72e40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:31.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.329+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f72ec014070 con 0x7f72fc1005d0 2024-02-19T16:21:31.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.329+0000 7f72faffd700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f72e40a21e0 0x7f72e40a46a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0ffb40 tx=0x7f72f000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:31.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.333+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f72ec053230 con 0x7f72fc1005d0 2024-02-19T16:21:31.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.542+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f72e80058b0 con 0x7f72fc1005d0 2024-02-19T16:21:31.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.542+0000 7f72f8ff9700 1 -- 172.21.15.8:0/1717541917 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72ec09a720 con 0x7f72fc1005d0 2024-02-19T16:21:31.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f72e40a21e0 msgr2=0x7f72e40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:31.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f72e40a21e0 0x7f72e40a46a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0ffb40 tx=0x7f72f000b610 comp rx=0 tx=0).stop 2024-02-19T16:21:31.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc1005d0 msgr2=0x7f72fc1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:31.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc1005d0 0x7f72fc1017b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e9f0 tx=0x7f72ec00ed00 comp rx=0 tx=0).stop 2024-02-19T16:21:31.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 shutdown_connections 2024-02-19T16:21:31.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f72e40a21e0 0x7f72e40a46a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc107050 0x7f72fc102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc1005d0 0x7f72fc1017b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 --2- 172.21.15.8:0/1717541917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72fc100f90 0x7f72fc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:31.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.544+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 >> 172.21.15.8:0/1717541917 conn(0x7f72fc0774e0 msgr2=0x7f72fc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:31.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.545+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 shutdown_connections 2024-02-19T16:21:31.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:30.545+0000 7f7301abb700 1 -- 172.21.15.8:0/1717541917 wait complete. 2024-02-19T16:21:31.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:31 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1717541917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-19T16:21:32.224 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:32 smithi008 ceph-mon[32573]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 59 B/s, 1 keys/s, 0 objects/s recovering 2024-02-19T16:21:32.224 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:32 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3199576891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-19T16:21:32.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:32 smithi156 ceph-mon[36502]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 59 B/s, 1 keys/s, 0 objects/s recovering 2024-02-19T16:21:32.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:32 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3199576891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-19T16:21:32.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:32 smithi191 ceph-mon[37203]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 59 B/s, 1 keys/s, 0 objects/s recovering 2024-02-19T16:21:32.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:32 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3199576891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 -- 172.21.15.8:0/2618474224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8106f20 msgr2=0x7ff6d8107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 --2- 172.21.15.8:0/2618474224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8106f20 0x7ff6d8107300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009990 tx=0x7ff6c8009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 -- 172.21.15.8:0/2618474224 shutdown_connections 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 --2- 172.21.15.8:0/2618474224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d8101880 0x7ff6d810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 --2- 172.21.15.8:0/2618474224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8106f20 0x7ff6d8107300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 --2- 172.21.15.8:0/2618474224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 -- 172.21.15.8:0/2618474224 >> 172.21.15.8:0/2618474224 conn(0x7ff6d8077e10 msgr2=0x7ff6d8078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 -- 172.21.15.8:0/2618474224 shutdown_connections 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.357+0000 7ff6de879700 1 -- 172.21.15.8:0/2618474224 wait complete. 2024-02-19T16:21:33.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 Processor -- start 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 -- start start 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8101880 0x7ff6d8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d8106f20 0x7ff6d819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819d3e0 con 0x7ff6d8101880 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819d550 con 0x7ff6d8106f20 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.358+0000 7ff6de879700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819d710 con 0x7ff6d8100ee0 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8101880 0x7ff6d8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:38284/0 (socket says 172.21.15.8:38284) 2024-02-19T16:21:33.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 -- 172.21.15.8:0/3199576891 learned_addr learned my addr 172.21.15.8:0/3199576891 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stdout:77309411401 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6dce16700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d8106f20 0x7ff6d819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 -- 172.21.15.8:0/3199576891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d8106f20 msgr2=0x7ff6d819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d8106f20 0x7ff6d819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 -- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8101880 msgr2=0x7ff6d8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8101880 0x7ff6d8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 -- 172.21.15.8:0/3199576891 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c80095f0 con 0x7ff6d8100ee0 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d7fff700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8198500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8000c00 tx=0x7ff6c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c801d070 con 0x7ff6d8100ee0 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d819d990 con 0x7ff6d8100ee0 2024-02-19T16:21:33.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6c8022470 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8017650 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.359+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d81a5250 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.360+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d81025d0 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.361+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7ff6c80225e0 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.362+0000 7ff6d57fa700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff6c00a21e0 0x7ff6c00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.362+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7ff6c80cf550 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.362+0000 7ff6d77fe700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff6c00a21e0 0x7ff6c00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.362+0000 7ff6d77fe700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff6c00a21e0 0x7ff6c00a46a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8078380 tx=0x7ff6cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.370+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff6c80980b0 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.578+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff6d8199890 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.578+0000 7ff6d57fa700 1 -- 172.21.15.8:0/3199576891 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff6c809b770 con 0x7ff6d8100ee0 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.580+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff6c00a21e0 msgr2=0x7ff6c00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.580+0000 7ff6de879700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff6c00a21e0 0x7ff6c00a46a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8078380 tx=0x7ff6cc009260 comp rx=0 tx=0).stop 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.580+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 msgr2=0x7ff6d8198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:33.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.580+0000 7ff6de879700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8198500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8000c00 tx=0x7ff6c8004730 comp rx=0 tx=0).stop 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 shutdown_connections 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff6c00a21e0 0x7ff6c00a46a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d8106f20 0x7ff6d819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d8100ee0 0x7ff6d8198500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 --2- 172.21.15.8:0/3199576891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6d8101880 0x7ff6d8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 >> 172.21.15.8:0/3199576891 conn(0x7ff6d8077e10 msgr2=0x7ff6d8100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 shutdown_connections 2024-02-19T16:21:33.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:31.581+0000 7ff6de879700 1 -- 172.21.15.8:0/3199576891 wait complete. 2024-02-19T16:21:33.877 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411401 for osd.2 2024-02-19T16:21:33.877 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:34.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:34 smithi008 ceph-mon[32573]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 53 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:34.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574925 for osd.1 2024-02-19T16:21:34.303 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:34.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:34 smithi191 ceph-mon[37203]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 53 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:34.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:34 smithi156 ceph-mon[36502]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 53 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 -- 172.21.15.8:0/2503759687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f14108060 msgr2=0x7f4f14108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 --2- 172.21.15.8:0/2503759687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f14108060 0x7f4f14108440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4f040099c0 tx=0x7f4f04009cd0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 -- 172.21.15.8:0/2503759687 shutdown_connections 2024-02-19T16:21:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 --2- 172.21.15.8:0/2503759687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f1406b080 0x7f4f1410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 --2- 172.21.15.8:0/2503759687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f14108060 0x7f4f14108440 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 --2- 172.21.15.8:0/2503759687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406a6e0 0x7f4f1406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.143+0000 7f4f19861700 1 -- 172.21.15.8:0/2503759687 >> 172.21.15.8:0/2503759687 conn(0x7f4f14076f00 msgr2=0x7f4f14077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.144+0000 7f4f19861700 1 -- 172.21.15.8:0/2503759687 shutdown_connections 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.144+0000 7f4f19861700 1 -- 172.21.15.8:0/2503759687 wait complete. 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.144+0000 7f4f19861700 1 Processor -- start 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 -- start start 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 0x7f4f14198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406b080 0x7f4f14198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f14108060 0x7f4f1419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f12ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 0x7f4f14198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f12ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 0x7f4f14198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35136/0 (socket says 172.21.15.8:35136) 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f12ffd700 1 -- 172.21.15.8:0/3041043089 learned_addr learned my addr 172.21.15.8:0/3041043089 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f127fc700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406b080 0x7f4f14198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419d450 con 0x7f4f1406a6e0 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419d5c0 con 0x7f4f1406b080 2024-02-19T16:21:35.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f137fe700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f14108060 0x7f4f1419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:35.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.145+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419d780 con 0x7f4f14108060 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f12ffd700 1 -- 172.21.15.8:0/3041043089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f14108060 msgr2=0x7f4f1419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f12ffd700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f14108060 0x7f4f1419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f12ffd700 1 -- 172.21.15.8:0/3041043089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406b080 msgr2=0x7f4f14198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f12ffd700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406b080 0x7f4f14198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f12ffd700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f08009520 con 0x7f4f1406a6e0 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f127fc700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406b080 0x7f4f14198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.146+0000 7f4f12ffd700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 0x7f4f14198570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4f04005910 tx=0x7f4f0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.147+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0401d070 con 0x7f4f1406a6e0 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.147+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f04003680 con 0x7f4f1406a6e0 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.147+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040173f0 con 0x7f4f1406a6e0 2024-02-19T16:21:35.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.147+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f040095f0 con 0x7f4f1406a6e0 2024-02-19T16:21:35.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.147+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1419dc30 con 0x7f4f1406a6e0 2024-02-19T16:21:35.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.148+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1410a580 con 0x7f4f1406a6e0 2024-02-19T16:21:35.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.150+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f4f04022600 con 0x7f4f1406a6e0 2024-02-19T16:21:35.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.150+0000 7f4f1885f700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4efc0a21e0 0x7f4efc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:35.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.150+0000 7f4f127fc700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4efc0a21e0 0x7f4efc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:35.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.150+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f4f040ceb70 con 0x7f4f1406a6e0 2024-02-19T16:21:35.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.151+0000 7f4f127fc700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4efc0a21e0 0x7f4efc0a46a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009fd0 tx=0x7f4f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:35.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.155+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f4f04097730 con 0x7f4f1406a6e0 2024-02-19T16:21:35.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.361+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f4f14103710 con 0x7f4f1406a6e0 2024-02-19T16:21:35.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.361+0000 7f4f1885f700 1 -- 172.21.15.8:0/3041043089 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f0409adf0 con 0x7f4f1406a6e0 2024-02-19T16:21:35.362 INFO:teuthology.orchestra.run.smithi008.stdout:257698037802 2024-02-19T16:21:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4efc0a21e0 msgr2=0x7f4efc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4efc0a21e0 0x7f4efc0a46a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009fd0 tx=0x7f4f08009260 comp rx=0 tx=0).stop 2024-02-19T16:21:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 msgr2=0x7f4f14198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 0x7f4f14198570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4f04005910 tx=0x7f4f0400f690 comp rx=0 tx=0).stop 2024-02-19T16:21:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 shutdown_connections 2024-02-19T16:21:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4efc0a21e0 0x7f4efc0a46a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f14108060 0x7f4f1419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f1406a6e0 0x7f4f14198570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 --2- 172.21.15.8:0/3041043089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1406b080 0x7f4f14198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 >> 172.21.15.8:0/3041043089 conn(0x7f4f14076f00 msgr2=0x7f4f140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.363+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 shutdown_connections 2024-02-19T16:21:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:35.364+0000 7f4f19861700 1 -- 172.21.15.8:0/3041043089 wait complete. 2024-02-19T16:21:36.304 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:36 smithi008 ceph-mon[32573]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:36.304 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:36 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3041043089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-19T16:21:36.328 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037802 for osd.8 2024-02-19T16:21:36.328 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:36.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:36 smithi156 ceph-mon[36502]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:36.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:36 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3041043089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-19T16:21:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:36.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:36 smithi191 ceph-mon[37203]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:36.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:36 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3041043089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-19T16:21:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:38 smithi008 ceph-mon[32573]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:38.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:38 smithi156 ceph-mon[36502]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:38.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 -- 172.21.15.8:0/1002606782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 msgr2=0x7f7c301024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 --2- 172.21.15.8:0/1002606782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c301024e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0099c0 tx=0x7f7c2c009cd0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 -- 172.21.15.8:0/1002606782 shutdown_connections 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 --2- 172.21.15.8:0/1002606782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102a20 0x7f7c3010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 --2- 172.21.15.8:0/1002606782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 0x7f7c30108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 --2- 172.21.15.8:0/1002606782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c301024e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 -- 172.21.15.8:0/1002606782 >> 172.21.15.8:0/1002606782 conn(0x7f7c30079140 msgr2=0x7f7c30079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 -- 172.21.15.8:0/1002606782 shutdown_connections 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.404+0000 7f7c3785b700 1 -- 172.21.15.8:0/1002606782 wait complete. 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 Processor -- start 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 -- start start 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c30198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:38.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102a20 0x7f7c301989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 0x7f7c3019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019d340 con 0x7f7c30102060 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019d4b0 con 0x7f7c30102a20 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c3785b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019d670 con 0x7f7c30108070 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c355f7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c30198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c355f7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c30198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51698/0 (socket says 172.21.15.8:51698) 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c355f7700 1 -- 172.21.15.8:0/4275561053 learned_addr learned my addr 172.21.15.8:0/4275561053 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.405+0000 7f7c35df8700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 0x7f7c3019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c34df6700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102a20 0x7f7c301989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c35df8700 1 -- 172.21.15.8:0/4275561053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102a20 msgr2=0x7f7c301989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c35df8700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102a20 0x7f7c301989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c35df8700 1 -- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 msgr2=0x7f7c30198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c35df8700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c30198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c35df8700 1 -- 172.21.15.8:0/4275561053 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0095f0 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c355f7700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c30198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c35df8700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 0x7f7c3019cd30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800efc0 tx=0x7f7c2800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28018070 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3019d8f0 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c2800f040 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800cd20 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.406+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c300743e0 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.407+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3010a550 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.408+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f7c28003a00 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.409+0000 7f7c267fc700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c1c0a21e0 0x7f7c1c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.409+0000 7f7c355f7700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c1c0a21e0 0x7f7c1c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.409+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f7c280ce280 con 0x7f7c30108070 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.409+0000 7f7c355f7700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c1c0a21e0 0x7f7c1c0a46a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097f0 tx=0x7f7c20012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:38.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.417+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7c28096de0 con 0x7f7c30108070 2024-02-19T16:21:38.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:38 smithi191 ceph-mon[37203]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-02-19T16:21:38.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.625+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7c3019dae0 con 0x7f7c30108070 2024-02-19T16:21:38.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.625+0000 7f7c267fc700 1 -- 172.21.15.8:0/4275561053 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c28003cb0 con 0x7f7c30108070 2024-02-19T16:21:38.626 INFO:teuthology.orchestra.run.smithi008.stdout:34359738451 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c1c0a21e0 msgr2=0x7f7c1c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c1c0a21e0 0x7f7c1c0a46a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097f0 tx=0x7f7c20012040 comp rx=0 tx=0).stop 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 msgr2=0x7f7c3019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 0x7f7c3019cd30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800efc0 tx=0x7f7c2800c3a0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 shutdown_connections 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7c1c0a21e0 0x7f7c1c0a46a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30108070 0x7f7c3019cd30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c30102060 0x7f7c30198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 --2- 172.21.15.8:0/4275561053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c30102a20 0x7f7c301989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:38.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 >> 172.21.15.8:0/4275561053 conn(0x7f7c30079140 msgr2=0x7f7c30100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:38.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 shutdown_connections 2024-02-19T16:21:38.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:38.628+0000 7f7c3785b700 1 -- 172.21.15.8:0/4275561053 wait complete. 2024-02-19T16:21:39.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4275561053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-19T16:21:39.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4275561053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-19T16:21:39.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4275561053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-19T16:21:40.238 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:40 smithi008 ceph-mon[32573]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:40.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:40 smithi156 ceph-mon[36502]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:40.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:40 smithi191 ceph-mon[37203]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 -- 172.21.15.8:0/812685906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c102030 msgr2=0x7f399c1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 --2- 172.21.15.8:0/812685906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c102030 0x7f399c1024b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f398c0099c0 tx=0x7f398c009cd0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 -- 172.21.15.8:0/812685906 shutdown_connections 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 --2- 172.21.15.8:0/812685906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f399c1029f0 0x7f399c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 --2- 172.21.15.8:0/812685906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f399c108010 0x7f399c1083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 --2- 172.21.15.8:0/812685906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c102030 0x7f399c1024b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 -- 172.21.15.8:0/812685906 >> 172.21.15.8:0/812685906 conn(0x7f399c079110 msgr2=0x7f399c079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 -- 172.21.15.8:0/812685906 shutdown_connections 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 -- 172.21.15.8:0/812685906 wait complete. 2024-02-19T16:21:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 Processor -- start 2024-02-19T16:21:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.551+0000 7f39a3d39700 1 -- start start 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f399c102030 0x7f399c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 0x7f399c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f399c108010 0x7f399c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c19d400 con 0x7f399c1029f0 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c19d570 con 0x7f399c102030 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c19d730 con 0x7f399c108010 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 0x7f399c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 0x7f399c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51722/0 (socket says 172.21.15.8:51722) 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 -- 172.21.15.8:0/720012276 learned_addr learned my addr 172.21.15.8:0/720012276 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a1ad5700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f399c102030 0x7f399c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a22d6700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f399c108010 0x7f399c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 -- 172.21.15.8:0/720012276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f399c108010 msgr2=0x7f399c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f399c108010 0x7f399c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 -- 172.21.15.8:0/720012276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f399c102030 msgr2=0x7f399c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f399c102030 0x7f399c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 -- 172.21.15.8:0/720012276 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0095f0 con 0x7f399c1029f0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a12d4700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 0x7f399c198ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f398c00f690 tx=0x7f398c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c01d070 con 0x7f399c1029f0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f398c003680 con 0x7f399c1029f0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c19d9b0 con 0x7f399c1029f0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.552+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c022e90 con 0x7f399c1029f0 2024-02-19T16:21:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.553+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c1a5270 con 0x7f399c1029f0 2024-02-19T16:21:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.553+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c10a4f0 con 0x7f399c1029f0 2024-02-19T16:21:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.555+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f398c022590 con 0x7f399c1029f0 2024-02-19T16:21:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.555+0000 7f3992ffd700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f39880a21e0 0x7f39880a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:40.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.555+0000 7f39a1ad5700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f39880a21e0 0x7f39880a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:40.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.555+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f398c0cf4c0 con 0x7f399c1029f0 2024-02-19T16:21:40.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.556+0000 7f39a1ad5700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f39880a21e0 0x7f39880a46a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3998009be0 tx=0x7f3998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:40.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.559+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f398c098080 con 0x7f399c1029f0 2024-02-19T16:21:40.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.766+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f399c199880 con 0x7f399c1029f0 2024-02-19T16:21:40.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.766+0000 7f3992ffd700 1 -- 172.21.15.8:0/720012276 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f398c031020 con 0x7f399c1029f0 2024-02-19T16:21:40.767 INFO:teuthology.orchestra.run.smithi008.stdout:201863462966 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f39880a21e0 msgr2=0x7f39880a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f39880a21e0 0x7f39880a46a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3998009be0 tx=0x7f3998009260 comp rx=0 tx=0).stop 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 msgr2=0x7f399c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 0x7f399c198ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f398c00f690 tx=0x7f398c00bfd0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 shutdown_connections 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f39880a21e0 0x7f39880a46a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f399c108010 0x7f399c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f399c102030 0x7f399c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 --2- 172.21.15.8:0/720012276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f399c1029f0 0x7f399c198ab0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 >> 172.21.15.8:0/720012276 conn(0x7f399c079110 msgr2=0x7f399c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:40.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 shutdown_connections 2024-02-19T16:21:40.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:40.768+0000 7f39a3d39700 1 -- 172.21.15.8:0/720012276 wait complete. 2024-02-19T16:21:41.174 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/720012276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 -- 172.21.15.8:0/541172854 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae18102030 msgr2=0x7fae181024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/541172854 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae18102030 0x7fae181024b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00b0d0 tx=0x7fae0c00b3e0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 -- 172.21.15.8:0/541172854 shutdown_connections 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/541172854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae181029f0 0x7fae1810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/541172854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae18108010 0x7fae181083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/541172854 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae18102030 0x7fae181024b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 -- 172.21.15.8:0/541172854 >> 172.21.15.8:0/541172854 conn(0x7fae18079110 msgr2=0x7fae18079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:41.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 -- 172.21.15.8:0/541172854 shutdown_connections 2024-02-19T16:21:41.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 -- 172.21.15.8:0/541172854 wait complete. 2024-02-19T16:21:41.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.382+0000 7fae1f3e8700 1 Processor -- start 2024-02-19T16:21:41.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 -- start start 2024-02-19T16:21:41.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae18102030 0x7fae180742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:41.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 0x7fae18074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:41.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae18108010 0x7fae18074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:41.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a69e0 con 0x7fae181029f0 2024-02-19T16:21:41.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a6b50 con 0x7fae18102030 2024-02-19T16:21:41.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1f3e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a6d10 con 0x7fae18108010 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 0x7fae18074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 0x7fae18074820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51742/0 (socket says 172.21.15.8:51742) 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 -- 172.21.15.8:0/2525964209 learned_addr learned my addr 172.21.15.8:0/2525964209 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1d985700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae18108010 0x7fae18074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1d184700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae18102030 0x7fae180742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 -- 172.21.15.8:0/2525964209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae18108010 msgr2=0x7fae18074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae18108010 0x7fae18074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 -- 172.21.15.8:0/2525964209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae18102030 msgr2=0x7fae180742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae18102030 0x7fae180742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 -- 172.21.15.8:0/2525964209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c009d00 con 0x7fae181029f0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.383+0000 7fae1c983700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 0x7fae18074820 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00bd50 tx=0x7fae0c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.384+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00e040 con 0x7fae181029f0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.384+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae0c0079d0 con 0x7fae181029f0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.384+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae181a6f90 con 0x7fae181029f0 2024-02-19T16:21:41.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.384+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01b9b0 con 0x7fae181029f0 2024-02-19T16:21:41.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.384+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae181a73b0 con 0x7fae181029f0 2024-02-19T16:21:41.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.385+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1810a4f0 con 0x7fae181029f0 2024-02-19T16:21:41.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.387+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fae0c01dc10 con 0x7fae181029f0 2024-02-19T16:21:41.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.387+0000 7fae167fc700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fae000a2260 0x7fae000a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:41.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.388+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fae0c0cfbf0 con 0x7fae181029f0 2024-02-19T16:21:41.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.388+0000 7fae1d184700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fae000a2260 0x7fae000a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:41.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.390+0000 7fae1d184700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fae000a2260 0x7fae000a4720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae10009be0 tx=0x7fae10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:41.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.392+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fae0c098830 con 0x7fae181029f0 2024-02-19T16:21:41.394 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:41.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/720012276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-19T16:21:41.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/720012276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-19T16:21:41.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.609+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fae18112010 con 0x7fae181029f0 2024-02-19T16:21:41.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.609+0000 7fae167fc700 1 -- 172.21.15.8:0/2525964209 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae0c021090 con 0x7fae181029f0 2024-02-19T16:21:41.610 INFO:teuthology.orchestra.run.smithi008.stdout:146028888128 2024-02-19T16:21:41.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fae000a2260 msgr2=0x7fae000a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:41.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fae000a2260 0x7fae000a4720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae10009be0 tx=0x7fae10009260 comp rx=0 tx=0).stop 2024-02-19T16:21:41.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 msgr2=0x7fae18074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:41.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 0x7fae18074820 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00bd50 tx=0x7fae0c00bd80 comp rx=0 tx=0).stop 2024-02-19T16:21:41.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 shutdown_connections 2024-02-19T16:21:41.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fae000a2260 0x7fae000a4720 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae18108010 0x7fae18074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae18102030 0x7fae180742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 --2- 172.21.15.8:0/2525964209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae181029f0 0x7fae18074820 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:41.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.611+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 >> 172.21.15.8:0/2525964209 conn(0x7fae18079110 msgr2=0x7fae18100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:41.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.612+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 shutdown_connections 2024-02-19T16:21:41.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:41.612+0000 7fae1f3e8700 1 -- 172.21.15.8:0/2525964209 wait complete. 2024-02-19T16:21:42.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738451 for osd.0 2024-02-19T16:21:42.072 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:42.313 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:42 smithi008 ceph-mon[32573]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:42.314 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:42 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2525964209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-19T16:21:42.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:42 smithi191 ceph-mon[37203]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:42.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:42 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2525964209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-19T16:21:42.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:42 smithi156 ceph-mon[36502]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:42.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:42 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2525964209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-19T16:21:44.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:44 smithi008 ceph-mon[32573]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:21:44.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2786420941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-19T16:21:44.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:44 smithi191 ceph-mon[37203]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:21:44.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2786420941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-19T16:21:44.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:44 smithi156 ceph-mon[36502]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:21:44.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2786420941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-19T16:21:44.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888124 got 146028888128 for osd.4 2024-02-19T16:21:44.584 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 -- 172.21.15.8:0/944409946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320107360 msgr2=0x7fb32010ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 --2- 172.21.15.8:0/944409946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320107360 0x7fb32010ef30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb31800b0a0 tx=0x7fb31800b3b0 comp rx=0 tx=0).stop 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 -- 172.21.15.8:0/944409946 shutdown_connections 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 --2- 172.21.15.8:0/944409946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320107360 0x7fb32010ef30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 --2- 172.21.15.8:0/944409946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb320102290 0x7fb320102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 --2- 172.21.15.8:0/944409946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb320102c40 0x7fb320106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 -- 172.21.15.8:0/944409946 >> 172.21.15.8:0/944409946 conn(0x7fb320079140 msgr2=0x7fb320079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 -- 172.21.15.8:0/944409946 shutdown_connections 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 -- 172.21.15.8:0/944409946 wait complete. 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 Processor -- start 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.883+0000 7fb3278a5700 1 -- start start 2024-02-19T16:21:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 0x7fb3201986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb320102c40 0x7fb320198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb320107360 0x7fb32019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32019d5a0 con 0x7fb320102290 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32019d710 con 0x7fb320102c40 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32019d8d0 con 0x7fb320107360 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 0x7fb3201986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 0x7fb3201986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51782/0 (socket says 172.21.15.8:51782) 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 -- 172.21.15.8:0/1618123180 learned_addr learned my addr 172.21.15.8:0/1618123180 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb324e40700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb320102c40 0x7fb320198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325e42700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb320107360 0x7fb32019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 -- 172.21.15.8:0/1618123180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb320107360 msgr2=0x7fb32019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb320107360 0x7fb32019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 -- 172.21.15.8:0/1618123180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb320102c40 msgr2=0x7fb320198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb320102c40 0x7fb320198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 -- 172.21.15.8:0/1618123180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb318009d00 con 0x7fb320102290 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb325641700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 0x7fb3201986c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb31000e9a0 tx=0x7fb31000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310009790 con 0x7fb320102290 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb310010430 con 0x7fb320102290 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32019dbb0 con 0x7fb320102290 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.884+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310018680 con 0x7fb320102290 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.885+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3201a5410 con 0x7fb320102290 2024-02-19T16:21:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.886+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32010a680 con 0x7fb320102290 2024-02-19T16:21:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.887+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb310010910 con 0x7fb320102290 2024-02-19T16:21:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.887+0000 7fb3167fc700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb30c0a2260 0x7fb30c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.888+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fb310014070 con 0x7fb320102290 2024-02-19T16:21:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.888+0000 7fb324e40700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb30c0a2260 0x7fb30c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.888+0000 7fb324e40700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb30c0a2260 0x7fb30c0a4720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb320108830 tx=0x7fb31c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:44.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:44.891+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb310097330 con 0x7fb320102290 2024-02-19T16:21:45.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.099+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb320199a20 con 0x7fb320102290 2024-02-19T16:21:45.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.100+0000 7fb3167fc700 1 -- 172.21.15.8:0/1618123180 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb310026090 con 0x7fb320102290 2024-02-19T16:21:45.101 INFO:teuthology.orchestra.run.smithi008.stdout:167503724602 2024-02-19T16:21:45.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb30c0a2260 msgr2=0x7fb30c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb30c0a2260 0x7fb30c0a4720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb320108830 tx=0x7fb31c009260 comp rx=0 tx=0).stop 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 msgr2=0x7fb3201986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 0x7fb3201986c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb31000e9a0 tx=0x7fb31000ecb0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 shutdown_connections 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb30c0a2260 0x7fb30c0a4720 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb320107360 0x7fb32019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb320102290 0x7fb3201986c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 --2- 172.21.15.8:0/1618123180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb320102c40 0x7fb320198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 >> 172.21.15.8:0/1618123180 conn(0x7fb320079140 msgr2=0x7fb320100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 shutdown_connections 2024-02-19T16:21:45.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.102+0000 7fb3278a5700 1 -- 172.21.15.8:0/1618123180 wait complete. 2024-02-19T16:21:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 -- 172.21.15.8:0/1237739568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 msgr2=0x7f70601024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 --2- 172.21.15.8:0/1237739568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601024e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f704c009990 tx=0x7f704c009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 -- 172.21.15.8:0/1237739568 shutdown_connections 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 --2- 172.21.15.8:0/1237739568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7060102a20 0x7f706010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 --2- 172.21.15.8:0/1237739568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70601080b0 0x7f7060108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 --2- 172.21.15.8:0/1237739568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601024e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 -- 172.21.15.8:0/1237739568 >> 172.21.15.8:0/1237739568 conn(0x7f7060079130 msgr2=0x7f7060079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 -- 172.21.15.8:0/1237739568 shutdown_connections 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.559+0000 7f706721f700 1 -- 172.21.15.8:0/1237739568 wait complete. 2024-02-19T16:21:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 Processor -- start 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 -- start start 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7060102a20 0x7f70601a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70601080b0 0x7f706019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706019ac00 con 0x7f7060102060 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706019ad40 con 0x7f7060102a20 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f706721f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706019af00 con 0x7f70601080b0 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f7064fbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f7064fbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57380/0 (socket says 172.21.15.8:57380) 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f7064fbb700 1 -- 172.21.15.8:0/458610811 learned_addr learned my addr 172.21.15.8:0/458610811 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f705ffff700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7060102a20 0x7f70601a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f70657bc700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70601080b0 0x7f706019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f7064fbb700 1 -- 172.21.15.8:0/458610811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70601080b0 msgr2=0x7f706019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.560+0000 7f7064fbb700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70601080b0 0x7f706019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f7064fbb700 1 -- 172.21.15.8:0/458610811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7060102a20 msgr2=0x7f70601a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f7064fbb700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7060102a20 0x7f70601a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f7064fbb700 1 -- 172.21.15.8:0/458610811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0095f0 con 0x7f7060102060 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f7064fbb700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601a03e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f705400e9a0 tx=0x7f705400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054009790 con 0x7f7060102060 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7054010430 con 0x7f7060102060 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706019b1e0 con 0x7f7060102060 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054018680 con 0x7f7060102060 2024-02-19T16:21:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.561+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70601a7480 con 0x7f7060102060 2024-02-19T16:21:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.562+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70601053b0 con 0x7f7060102060 2024-02-19T16:21:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.563+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f7054010910 con 0x7f7060102060 2024-02-19T16:21:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.564+0000 7f705dffb700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70500a21e0 0x7f70500a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.564+0000 7f705ffff700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70500a21e0 0x7f70500a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.564+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f7054014070 con 0x7f7060102060 2024-02-19T16:21:45.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.564+0000 7f705ffff700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70500a21e0 0x7f70500a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70600799d0 tx=0x7f704c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:45.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.571+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7054097310 con 0x7f7060102060 2024-02-19T16:21:45.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.781+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f706019bad0 con 0x7f7060102060 2024-02-19T16:21:45.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.782+0000 7f705dffb700 1 -- 172.21.15.8:0/458610811 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70540105a0 con 0x7f7060102060 2024-02-19T16:21:45.783 INFO:teuthology.orchestra.run.smithi008.stdout:223338299442 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70500a21e0 msgr2=0x7f70500a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70500a21e0 0x7f70500a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70600799d0 tx=0x7f704c00b280 comp rx=0 tx=0).stop 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 msgr2=0x7f70601a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601a03e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f705400e9a0 tx=0x7f705400ecb0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 shutdown_connections 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f70500a21e0 0x7f70500a46a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70601080b0 0x7f706019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7060102060 0x7f70601a03e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 --2- 172.21.15.8:0/458610811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7060102a20 0x7f70601a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:45.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 >> 172.21.15.8:0/458610811 conn(0x7f7060079130 msgr2=0x7f7060074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:45.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 shutdown_connections 2024-02-19T16:21:45.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:45.785+0000 7f706721f700 1 -- 172.21.15.8:0/458610811 wait complete. 2024-02-19T16:21:46.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724602 for osd.5 2024-02-19T16:21:46.047 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:46.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/3635460524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4102220 msgr2=0x7fbdf4102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:46.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/3635460524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4102220 0x7fbdf4102600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbde400b0a0 tx=0x7fbde400b3b0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/3635460524 shutdown_connections 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/3635460524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbdf4107300 0x7fbdf410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/3635460524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4102220 0x7fbdf4102600 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/3635460524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdf4102bd0 0x7fbdf4106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/3635460524 >> 172.21.15.8:0/3635460524 conn(0x7fbdf40fda90 msgr2=0x7fbdf40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/3635460524 shutdown_connections 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.410+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/3635460524 wait complete. 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 Processor -- start 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 -- start start 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdf4102220 0x7fbdf41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbdf4102bd0 0x7fbdf4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 0x7fbdf419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf419d530 con 0x7fbdf4107300 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf419d6a0 con 0x7fbdf4102220 2024-02-19T16:21:46.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf8c7b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf419d860 con 0x7fbdf4102bd0 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 0x7fbdf419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 0x7fbdf419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51766/0 (socket says 172.21.15.8:51766) 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 -- 172.21.15.8:0/2786420941 learned_addr learned my addr 172.21.15.8:0/2786420941 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf1d9b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbdf4102bd0 0x7fbdf4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf259c700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdf4102220 0x7fbdf41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 -- 172.21.15.8:0/2786420941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbdf4102bd0 msgr2=0x7fbdf4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbdf4102bd0 0x7fbdf4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 -- 172.21.15.8:0/2786420941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdf4102220 msgr2=0x7fbdf41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdf4102220 0x7fbdf41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.411+0000 7fbdf2d9d700 1 -- 172.21.15.8:0/2786420941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009d00 con 0x7fbdf4107300 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.412+0000 7fbdf2d9d700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 0x7fbdf419cf20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00e940 tx=0x7fbdec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:46.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.412+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0097b0 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.412+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf419db40 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.412+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdec005020 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.412+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00ccf0 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.412+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf41a53a0 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.413+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf410a5c0 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.414+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fbdec005490 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.415+0000 7fbdeb7fe700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbde00a21e0 0x7fbde00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.415+0000 7fbdf259c700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbde00a21e0 0x7fbde00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.415+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fbdec014070 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.415+0000 7fbdf259c700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbde00a21e0 0x7fbde00a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009fd0 tx=0x7fbde400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.418+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbdec097040 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.626+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fbdf41999b0 con 0x7fbdf4107300 2024-02-19T16:21:46.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.627+0000 7fbdeb7fe700 1 -- 172.21.15.8:0/2786420941 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbdec09a700 con 0x7fbdf4107300 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stdout:292057776167 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbde00a21e0 msgr2=0x7fbde00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbde00a21e0 0x7fbde00a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009fd0 tx=0x7fbde400b720 comp rx=0 tx=0).stop 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 msgr2=0x7fbdf419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 0x7fbdf419cf20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00e940 tx=0x7fbdec00ec50 comp rx=0 tx=0).stop 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 shutdown_connections 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbde00a21e0 0x7fbde00a46a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdf4107300 0x7fbdf419cf20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdf4102220 0x7fbdf41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 --2- 172.21.15.8:0/2786420941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbdf4102bd0 0x7fbdf4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:46.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 >> 172.21.15.8:0/2786420941 conn(0x7fbdf40fda90 msgr2=0x7fbdf40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:46.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 shutdown_connections 2024-02-19T16:21:46.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:43.629+0000 7fbdf8c7b700 1 -- 172.21.15.8:0/2786420941 wait complete. 2024-02-19T16:21:46.325 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:46 smithi008 ceph-mon[32573]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:46.325 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:46 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1618123180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-19T16:21:46.325 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:46 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/458610811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-19T16:21:46.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:46 smithi191 ceph-mon[37203]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:46.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:46 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1618123180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-19T16:21:46.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:46 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/458610811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-19T16:21:46.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:46 smithi156 ceph-mon[36502]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:46.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:46 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1618123180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-19T16:21:46.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:46 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/458610811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-19T16:21:46.988 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776167 for osd.9 2024-02-19T16:21:46.988 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1305265161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8108010 msgr2=0x7f1ae81083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1305265161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8108010 0x7f1ae81083f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0099c0 tx=0x7f1adc009cd0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1305265161 shutdown_connections 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1305265161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae81029f0 0x7f1ae810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1305265161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8108010 0x7f1ae81083f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1305265161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8102030 0x7f1ae81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1305265161 >> 172.21.15.8:0/1305265161 conn(0x7f1ae8079110 msgr2=0x7f1ae8079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1305265161 shutdown_connections 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1305265161 wait complete. 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 Processor -- start 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.315+0000 7f1aeecf2700 1 -- start start 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 0x7f1ae80742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae81029f0 0x7f1ae8074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8108010 0x7f1ae8074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81a6900 con 0x7f1ae8102030 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81a6a70 con 0x7f1ae8108010 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81a6c30 con 0x7f1ae81029f0 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 0x7f1ae80742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 0x7f1ae80742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57406/0 (socket says 172.21.15.8:57406) 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 -- 172.21.15.8:0/1561025238 learned_addr learned my addr 172.21.15.8:0/1561025238 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1ae7fff700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae81029f0 0x7f1ae8074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aed28f700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8108010 0x7f1ae8074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 -- 172.21.15.8:0/1561025238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae81029f0 msgr2=0x7f1ae8074820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae81029f0 0x7f1ae8074820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 -- 172.21.15.8:0/1561025238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8108010 msgr2=0x7f1ae8074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8108010 0x7f1ae8074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 -- 172.21.15.8:0/1561025238 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0095f0 con 0x7f1ae8102030 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeca8e700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 0x7f1ae80742e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1adc006010 tx=0x7f1adc005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc01d070 con 0x7f1ae8102030 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1adc0056c0 con 0x7f1ae8102030 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae81a6eb0 con 0x7f1ae8102030 2024-02-19T16:21:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.316+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc00fc20 con 0x7f1ae8102030 2024-02-19T16:21:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.317+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae81a7260 con 0x7f1ae8102030 2024-02-19T16:21:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.317+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae810a4f0 con 0x7f1ae8102030 2024-02-19T16:21:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.319+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f1adc00b790 con 0x7f1ae8102030 2024-02-19T16:21:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.319+0000 7f1ae5ffb700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ad80a21e0 0x7f1ad80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:47.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.319+0000 7f1ae7fff700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ad80a21e0 0x7f1ad80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:47.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.319+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f1adc0cf140 con 0x7f1ae8102030 2024-02-19T16:21:47.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.320+0000 7f1ae7fff700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ad80a21e0 0x7f1ad80a46a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80796f0 tx=0x7f1ad40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:47.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.323+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1adc097d00 con 0x7f1ae8102030 2024-02-19T16:21:47.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.533+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ae8112010 con 0x7f1ae8102030 2024-02-19T16:21:47.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.534+0000 7f1ae5ffb700 1 -- 172.21.15.8:0/1561025238 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1adc09b3c0 con 0x7f1ae8102030 2024-02-19T16:21:47.535 INFO:teuthology.orchestra.run.smithi008.stdout:111669149767 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ad80a21e0 msgr2=0x7f1ad80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ad80a21e0 0x7f1ad80a46a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80796f0 tx=0x7f1ad40073d0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 msgr2=0x7f1ae80742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 0x7f1ae80742e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1adc006010 tx=0x7f1adc005280 comp rx=0 tx=0).stop 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 shutdown_connections 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1ad80a21e0 0x7f1ad80a46a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8108010 0x7f1ae8074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ae8102030 0x7f1ae80742e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 --2- 172.21.15.8:0/1561025238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae81029f0 0x7f1ae8074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:47.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 >> 172.21.15.8:0/1561025238 conn(0x7f1ae8079110 msgr2=0x7f1ae8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:47.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 shutdown_connections 2024-02-19T16:21:47.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:47.536+0000 7f1aeecf2700 1 -- 172.21.15.8:0/1561025238 wait complete. 2024-02-19T16:21:47.788 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462966 for osd.6 2024-02-19T16:21:47.789 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:48.341 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:48 smithi008 ceph-mon[32573]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:21:48.341 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:48 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1561025238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-19T16:21:48.343 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149760 got 111669149767 for osd.3 2024-02-19T16:21:48.343 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:48.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:48 smithi191 ceph-mon[37203]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:21:48.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:48 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1561025238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-19T16:21:48.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:48 smithi156 ceph-mon[36502]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:21:48.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:48 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1561025238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-19T16:21:48.766 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299437 got 223338299442 for osd.7 2024-02-19T16:21:48.766 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:50.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 -- 172.21.15.8:0/610406317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac105f20 msgr2=0x7fb5ac106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/610406317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac105f20 0x7fb5ac106300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009990 tx=0x7fb5a0009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 -- 172.21.15.8:0/610406317 shutdown_connections 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/610406317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac10ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/610406317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac105f20 0x7fb5ac106300 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/610406317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ac101a30 0x7fb5ac101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 -- 172.21.15.8:0/610406317 >> 172.21.15.8:0/610406317 conn(0x7fb5ac076220 msgr2=0x7fb5ac076630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 -- 172.21.15.8:0/610406317 shutdown_connections 2024-02-19T16:21:50.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 -- 172.21.15.8:0/610406317 wait complete. 2024-02-19T16:21:50.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 Processor -- start 2024-02-19T16:21:50.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.297+0000 7fb5b20ab700 1 -- start start 2024-02-19T16:21:50.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5b20ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac101a30 0x7fb5ac198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5b20ab700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5b20ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ac105f20 0x7fb5ac19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5b20ab700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac19d3f0 con 0x7fb5ac1023f0 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5b20ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac19d560 con 0x7fb5ac101a30 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5b20ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac19d720 con 0x7fb5ac105f20 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac101a30 0x7fb5ac198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57426/0 (socket says 172.21.15.8:57426) 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5aaffd700 1 -- 172.21.15.8:0/532992728 learned_addr learned my addr 172.21.15.8:0/532992728 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5abfff700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ac105f20 0x7fb5ac19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5ab7fe700 1 -- 172.21.15.8:0/532992728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ac105f20 msgr2=0x7fb5ac19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5ab7fe700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ac105f20 0x7fb5ac19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5ab7fe700 1 -- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 msgr2=0x7fb5ac198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5ab7fe700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5ab7fe700 1 -- 172.21.15.8:0/532992728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a00095f0 con 0x7fb5ac101a30 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.298+0000 7fb5aaffd700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:50.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.299+0000 7fb5ab7fe700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac101a30 0x7fb5ac198510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00098e0 tx=0x7fb5a0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:50.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.299+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a001d070 con 0x7fb5ac101a30 2024-02-19T16:21:50.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.299+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5a000b610 con 0x7fb5ac101a30 2024-02-19T16:21:50.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.299+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a00173f0 con 0x7fb5ac101a30 2024-02-19T16:21:50.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.299+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac19d9a0 con 0x7fb5ac101a30 2024-02-19T16:21:50.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.299+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac1a5260 con 0x7fb5ac101a30 2024-02-19T16:21:50.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.300+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac10a4b0 con 0x7fb5ac101a30 2024-02-19T16:21:50.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.302+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fb5a0022600 con 0x7fb5ac101a30 2024-02-19T16:21:50.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.302+0000 7fb5a8ff9700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb5940a21e0 0x7fb5940a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.303+0000 7fb5aaffd700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb5940a21e0 0x7fb5940a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.303+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fb5a00cf700 con 0x7fb5ac101a30 2024-02-19T16:21:50.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.303+0000 7fb5aaffd700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb5940a21e0 0x7fb5940a46a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac076900 tx=0x7fb59c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:50.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.305+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb5a0098260 con 0x7fb5ac101a30 2024-02-19T16:21:50.398 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:50 smithi008 ceph-mon[32573]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 -- 172.21.15.8:0/2858231753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102240 msgr2=0x7fac88102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 --2- 172.21.15.8:0/2858231753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102240 0x7fac88102620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fac78009990 tx=0x7fac78009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 -- 172.21.15.8:0/2858231753 shutdown_connections 2024-02-19T16:21:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 --2- 172.21.15.8:0/2858231753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 0x7fac8810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 --2- 172.21.15.8:0/2858231753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102240 0x7fac88102620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 --2- 172.21.15.8:0/2858231753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102bf0 0x7fac88106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 -- 172.21.15.8:0/2858231753 >> 172.21.15.8:0/2858231753 conn(0x7fac880fdab0 msgr2=0x7fac880ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 -- 172.21.15.8:0/2858231753 shutdown_connections 2024-02-19T16:21:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.499+0000 7fac8e54d700 1 -- 172.21.15.8:0/2858231753 wait complete. 2024-02-19T16:21:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 Processor -- start 2024-02-19T16:21:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 -- start start 2024-02-19T16:21:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 0x7fac88198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102bf0 0x7fac88198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 0x7fac8819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819d470 con 0x7fac88102bf0 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819d5e0 con 0x7fac88107320 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.500+0000 7fac8e54d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819d7a0 con 0x7fac88102240 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac87fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 0x7fac88198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac877fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102bf0 0x7fac88198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac87fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 0x7fac88198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:43788/0 (socket says 172.21.15.8:43788) 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac877fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102bf0 0x7fac88198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57446/0 (socket says 172.21.15.8:57446) 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac87fff700 1 -- 172.21.15.8:0/1216282690 learned_addr learned my addr 172.21.15.8:0/1216282690 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 0x7fac8819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 -- 172.21.15.8:0/1216282690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 msgr2=0x7fac88198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 0x7fac88198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 -- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102bf0 msgr2=0x7fac88198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102bf0 0x7fac88198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 -- 172.21.15.8:0/1216282690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780095f0 con 0x7fac88107320 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac87fff700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 0x7fac88198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8caea700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 0x7fac8819ce60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac8000efc0 tx=0x7fac8000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80018070 con 0x7fac88107320 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8819da20 con 0x7fac88107320 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac8000f040 con 0x7fac88107320 2024-02-19T16:21:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8000cd10 con 0x7fac88107320 2024-02-19T16:21:50.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.501+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac881a52e0 con 0x7fac88107320 2024-02-19T16:21:50.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.502+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8810a5e0 con 0x7fac88107320 2024-02-19T16:21:50.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.503+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fac80004ad0 con 0x7fac88107320 2024-02-19T16:21:50.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.504+0000 7fac857fa700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fac700a21e0 0x7fac700a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:50.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.504+0000 7fac87fff700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fac700a21e0 0x7fac700a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:50.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.504+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fac800cddf0 con 0x7fac88107320 2024-02-19T16:21:50.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.505+0000 7fac87fff700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fac700a21e0 0x7fac700a46a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac7800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:50.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.507+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fac80096950 con 0x7fac88107320 2024-02-19T16:21:50.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.518+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fb5ac068cd0 con 0x7fb5ac101a30 2024-02-19T16:21:50.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.519+0000 7fb5a8ff9700 1 -- 172.21.15.8:0/532992728 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb5a0027020 con 0x7fb5ac101a30 2024-02-19T16:21:50.520 INFO:teuthology.orchestra.run.smithi008.stdout:313532612643 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb5940a21e0 msgr2=0x7fb5940a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb5940a21e0 0x7fb5940a46a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac076900 tx=0x7fb59c009260 comp rx=0 tx=0).stop 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac101a30 msgr2=0x7fb5ac198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac101a30 0x7fb5ac198510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00098e0 tx=0x7fb5a0005740 comp rx=0 tx=0).stop 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 shutdown_connections 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb5940a21e0 0x7fb5940a46a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ac105f20 0x7fb5ac19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac101a30 0x7fb5ac198510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 --2- 172.21.15.8:0/532992728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5ac1023f0 0x7fb5ac198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 >> 172.21.15.8:0/532992728 conn(0x7fb5ac076220 msgr2=0x7fb5ac100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:50.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 shutdown_connections 2024-02-19T16:21:50.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.521+0000 7fb5b20ab700 1 -- 172.21.15.8:0/532992728 wait complete. 2024-02-19T16:21:50.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:50 smithi191 ceph-mon[37203]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:50.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:50 smithi156 ceph-mon[36502]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:21:50.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.737+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fac88068ca0 con 0x7fac88107320 2024-02-19T16:21:50.740 INFO:teuthology.orchestra.run.smithi008.stdout:347892351005 2024-02-19T16:21:50.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.739+0000 7fac857fa700 1 -- 172.21.15.8:0/1216282690 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fac8009a010 con 0x7fac88107320 2024-02-19T16:21:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.745+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fac700a21e0 msgr2=0x7fac700a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.745+0000 7fac8e54d700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fac700a21e0 0x7fac700a46a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac7800b260 comp rx=0 tx=0).stop 2024-02-19T16:21:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.745+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 msgr2=0x7fac8819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.745+0000 7fac8e54d700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 0x7fac8819ce60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac8000efc0 tx=0x7fac8000c3a0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.746+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 shutdown_connections 2024-02-19T16:21:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.746+0000 7fac8e54d700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fac700a21e0 0x7fac700a46a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.746+0000 7fac8e54d700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac88107320 0x7fac8819ce60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.747+0000 7fac8e54d700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac88102240 0x7fac88198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.747+0000 7fac8e54d700 1 --2- 172.21.15.8:0/1216282690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac88102bf0 0x7fac88198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:50.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.747+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 >> 172.21.15.8:0/1216282690 conn(0x7fac880fdab0 msgr2=0x7fac880ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:50.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.750+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 shutdown_connections 2024-02-19T16:21:50.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:50.751+0000 7fac8e54d700 1 -- 172.21.15.8:0/1216282690 wait complete. 2024-02-19T16:21:51.210 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612638 got 313532612643 for osd.10 2024-02-19T16:21:51.211 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:51.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:51 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/532992728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-19T16:21:51.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:51 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1216282690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-19T16:21:51.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:51 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/532992728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-19T16:21:51.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:51 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1216282690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-19T16:21:51.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:51 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/532992728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-19T16:21:51.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:51 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1216282690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-19T16:21:52.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350999 got 347892351005 for osd.11 2024-02-19T16:21:52.009 DEBUG:teuthology.parallel:result is None 2024-02-19T16:21:52.009 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-19T16:21:52.009 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph pg dump --format=json 2024-02-19T16:21:52.291 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:52 smithi008 ceph-mon[32573]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:21:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:21:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:52 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:52 smithi191 ceph-mon[37203]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:52.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:21:52.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:21:52.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:52 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:52 smithi156 ceph-mon[36502]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:21:52.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:52.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:21:52.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:21:52.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:52 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' 2024-02-19T16:21:53.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:53 smithi008 ceph-mon[32573]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-19T16:21:53.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:53 smithi191 ceph-mon[37203]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-19T16:21:53.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:53 smithi156 ceph-mon[36502]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-19T16:21:54.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.998+0000 7f114fe4a700 1 -- 172.21.15.8:0/3547773443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1148074210 msgr2=0x7f11480745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:54.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.998+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3547773443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1148074210 0x7f11480745f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1144009990 tx=0x7f1144009ca0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 -- 172.21.15.8:0/3547773443 shutdown_connections 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3547773443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114810e9e0 0x7f1148110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3547773443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1148074210 0x7f11480745f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3547773443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1148074b30 0x7f114810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 -- 172.21.15.8:0/3547773443 >> 172.21.15.8:0/3547773443 conn(0x7f1148079110 msgr2=0x7f1148079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 -- 172.21.15.8:0/3547773443 shutdown_connections 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 -- 172.21.15.8:0/3547773443 wait complete. 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 Processor -- start 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 -- start start 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:53.999+0000 7f114fe4a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1148074210 0x7f11481a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1148074b30 0x7f11481a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 0x7f114819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:54.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114819ac70 con 0x7f114810e9e0 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114819adb0 con 0x7f1148074b30 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114819af70 con 0x7f1148074210 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 0x7f114819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 0x7f114819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57468/0 (socket says 172.21.15.8:57468) 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 -- 172.21.15.8:0/3018789675 learned_addr learned my addr 172.21.15.8:0/3018789675 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114dbe6700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1148074210 0x7f11481a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114d3e5700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1148074b30 0x7f11481a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 -- 172.21.15.8:0/3018789675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1148074210 msgr2=0x7f11481a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1148074210 0x7f11481a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 -- 172.21.15.8:0/3018789675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1148074b30 msgr2=0x7f11481a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1148074b30 0x7f11481a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 -- 172.21.15.8:0/3018789675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140009c40 con 0x7f114810e9e0 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114e3e7700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 0x7f114819a610 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f114000c370 tx=0x7f114000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:54.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140017070 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11440095f0 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114000f040 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000e050 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.000+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11481a72d0 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.001+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1148109bf0 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.003+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f114000cb60 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.004+0000 7f113effd700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f11340a21d0 0x7f11340a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.004+0000 7f114dbe6700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f11340a21d0 0x7f11340a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.004+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f11400cd960 con 0x7f114810e9e0 2024-02-19T16:21:54.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.004+0000 7f114dbe6700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f11340a21d0 0x7f11340a4690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f11440098e0 tx=0x7f1144005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:54.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.008+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1140096590 con 0x7f114810e9e0 2024-02-19T16:21:54.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.204+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f114819baf0 con 0x7f11340a21d0 2024-02-19T16:21:54.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.220+0000 7f113effd700 1 -- 172.21.15.8:0/3018789675 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428465 (secure 0 0 0) 0x7f114819baf0 con 0x7f11340a21d0 2024-02-19T16:21:54.227 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.224+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f11340a21d0 msgr2=0x7f11340a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.224+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f11340a21d0 0x7f11340a4690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f11440098e0 tx=0x7f1144005960 comp rx=0 tx=0).stop 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.224+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 msgr2=0x7f114819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.224+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 0x7f114819a610 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f114000c370 tx=0x7f114000c680 comp rx=0 tx=0).stop 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 shutdown_connections 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f11340a21d0 0x7f11340a4690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f114810e9e0 0x7f114819a610 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1148074210 0x7f11481a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 --2- 172.21.15.8:0/3018789675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1148074b30 0x7f11481a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 >> 172.21.15.8:0/3018789675 conn(0x7f1148079110 msgr2=0x7f1148100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 shutdown_connections 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:54.225+0000 7f114fe4a700 1 -- 172.21.15.8:0/3018789675 wait complete. 2024-02-19T16:21:54.231 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:21:54.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:54 smithi008 ceph-mon[32573]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-19T16:21:54.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:54 smithi191 ceph-mon[37203]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-19T16:21:54.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:54 smithi156 ceph-mon[36502]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-19T16:21:54.921 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-02-19T16:21:52.735926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656519,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":574,"num_read_kb":407,"num_write":348,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":556,"ondisk_log_size":556,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378204,"kb_used_data":12380,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563620,"statfs":{"total":1151940427776,"available":1151553146880,"internally_reserved":0,"allocated":12677120,"data_stored":8104115,"data_compressed":396768,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883784+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:21:21.883784+0000","last_peered":"2024-02-19T16:21:21.883784+0000","last_clean":"2024-02-19T16:21:21.883784+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:21:21.883784+0000","last_undegraded":"2024-02-19T16:21:21.883784+0000","last_fullsized":"2024-02-19T16:21:21.883784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877401+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:21:21.877401+0000","last_peered":"2024-02-19T16:21:21.877401+0000","last_clean":"2024-02-19T16:21:21.877401+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:21:21.877401+0000","last_undegraded":"2024-02-19T16:21:21.877401+0000","last_fullsized":"2024-02-19T16:21:21.877401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877198+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:21:21.877198+0000","last_peered":"2024-02-19T16:21:21.877198+0000","last_clean":"2024-02-19T16:21:21.877198+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:21:21.877198+0000","last_undegraded":"2024-02-19T16:21:21.877198+0000","last_fullsized":"2024-02-19T16:21:21.877198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.749334+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:21:50.749334+0000","last_peered":"2024-02-19T16:21:50.749334+0000","last_clean":"2024-02-19T16:21:50.749334+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:21:50.749334+0000","last_undegraded":"2024-02-19T16:21:50.749334+0000","last_fullsized":"2024-02-19T16:21:50.749334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.740400+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:21:50.740400+0000","last_peered":"2024-02-19T16:21:50.740400+0000","last_clean":"2024-02-19T16:21:50.740400+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:21:50.740400+0000","last_undegraded":"2024-02-19T16:21:50.740400+0000","last_fullsized":"2024-02-19T16:21:50.740400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877871+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:21:21.877871+0000","last_peered":"2024-02-19T16:21:21.877871+0000","last_clean":"2024-02-19T16:21:21.877871+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:21:21.877871+0000","last_undegraded":"2024-02-19T16:21:21.877871+0000","last_fullsized":"2024-02-19T16:21:21.877871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750666+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:21:50.750666+0000","last_peered":"2024-02-19T16:21:50.750666+0000","last_clean":"2024-02-19T16:21:50.750666+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:21:50.750666+0000","last_undegraded":"2024-02-19T16:21:50.750666+0000","last_fullsized":"2024-02-19T16:21:50.750666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762122+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:21:50.762122+0000","last_peered":"2024-02-19T16:21:50.762122+0000","last_clean":"2024-02-19T16:21:50.762122+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:21:50.762122+0000","last_undegraded":"2024-02-19T16:21:50.762122+0000","last_fullsized":"2024-02-19T16:21:50.762122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756040+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:21:50.756040+0000","last_peered":"2024-02-19T16:21:50.756040+0000","last_clean":"2024-02-19T16:21:50.756040+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:21:50.756040+0000","last_undegraded":"2024-02-19T16:21:50.756040+0000","last_fullsized":"2024-02-19T16:21:50.756040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878105+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:21:21.878105+0000","last_peered":"2024-02-19T16:21:21.878105+0000","last_clean":"2024-02-19T16:21:21.878105+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:21:21.878105+0000","last_undegraded":"2024-02-19T16:21:21.878105+0000","last_fullsized":"2024-02-19T16:21:21.878105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756737+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:21:50.756737+0000","last_peered":"2024-02-19T16:21:50.756737+0000","last_clean":"2024-02-19T16:21:50.756737+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:21:50.756737+0000","last_undegraded":"2024-02-19T16:21:50.756737+0000","last_fullsized":"2024-02-19T16:21:50.756737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.740107+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:21:50.740107+0000","last_peered":"2024-02-19T16:21:50.740107+0000","last_clean":"2024-02-19T16:21:50.740107+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:21:50.740107+0000","last_undegraded":"2024-02-19T16:21:50.740107+0000","last_fullsized":"2024-02-19T16:21:50.740107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755017+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:21:50.755017+0000","last_peered":"2024-02-19T16:21:50.755017+0000","last_clean":"2024-02-19T16:21:50.755017+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:21:50.755017+0000","last_undegraded":"2024-02-19T16:21:50.755017+0000","last_fullsized":"2024-02-19T16:21:50.755017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877021+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:21:21.877021+0000","last_peered":"2024-02-19T16:21:21.877021+0000","last_clean":"2024-02-19T16:21:21.877021+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:21:21.877021+0000","last_undegraded":"2024-02-19T16:21:21.877021+0000","last_fullsized":"2024-02-19T16:21:21.877021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877206+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:21:21.877206+0000","last_peered":"2024-02-19T16:21:21.877206+0000","last_clean":"2024-02-19T16:21:21.877206+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:21:21.877206+0000","last_undegraded":"2024-02-19T16:21:21.877206+0000","last_fullsized":"2024-02-19T16:21:21.877206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756193+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:21:50.756193+0000","last_peered":"2024-02-19T16:21:50.756193+0000","last_clean":"2024-02-19T16:21:50.756193+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:21:50.756193+0000","last_undegraded":"2024-02-19T16:21:50.756193+0000","last_fullsized":"2024-02-19T16:21:50.756193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883677+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:21:21.883677+0000","last_peered":"2024-02-19T16:21:21.883677+0000","last_clean":"2024-02-19T16:21:21.883677+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:21:21.883677+0000","last_undegraded":"2024-02-19T16:21:21.883677+0000","last_fullsized":"2024-02-19T16:21:21.883677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.748335+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:21:50.748335+0000","last_peered":"2024-02-19T16:21:50.748335+0000","last_clean":"2024-02-19T16:21:50.748335+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:21:50.748335+0000","last_undegraded":"2024-02-19T16:21:50.748335+0000","last_fullsized":"2024-02-19T16:21:50.748335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878135+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:21:21.878135+0000","last_peered":"2024-02-19T16:21:21.878135+0000","last_clean":"2024-02-19T16:21:21.878135+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:21:21.878135+0000","last_undegraded":"2024-02-19T16:21:21.878135+0000","last_fullsized":"2024-02-19T16:21:21.878135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877804+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:21:21.877804+0000","last_peered":"2024-02-19T16:21:21.877804+0000","last_clean":"2024-02-19T16:21:21.877804+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:21:21.877804+0000","last_undegraded":"2024-02-19T16:21:21.877804+0000","last_fullsized":"2024-02-19T16:21:21.877804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.892098+0000","last_change":"2024-02-19T16:21:21.892098+0000","last_active":"2024-02-19T16:21:21.892098+0000","last_peered":"2024-02-19T16:21:21.892098+0000","last_clean":"2024-02-19T16:21:21.892098+0000","last_became_active":"2024-02-19T16:21:21.884753+0000","last_became_peered":"2024-02-19T16:21:21.884753+0000","last_unstale":"2024-02-19T16:21:21.892098+0000","last_undegraded":"2024-02-19T16:21:21.892098+0000","last_fullsized":"2024-02-19T16:21:21.892098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:24.360019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.760365+0000","last_change":"2024-02-19T16:20:21.662054+0000","last_active":"2024-02-19T16:21:50.760365+0000","last_peered":"2024-02-19T16:21:50.760365+0000","last_clean":"2024-02-19T16:21:50.760365+0000","last_became_active":"2024-02-19T16:20:21.661721+0000","last_became_peered":"2024-02-19T16:20:21.661721+0000","last_unstale":"2024-02-19T16:21:50.760365+0000","last_undegraded":"2024-02-19T16:21:50.760365+0000","last_fullsized":"2024-02-19T16:21:50.760365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:27:13.322445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875332+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:21:21.875332+0000","last_peered":"2024-02-19T16:21:21.875332+0000","last_clean":"2024-02-19T16:21:21.875332+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:21:21.875332+0000","last_undegraded":"2024-02-19T16:21:21.875332+0000","last_fullsized":"2024-02-19T16:21:21.875332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214984+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:21:22.214984+0000","last_peered":"2024-02-19T16:21:22.214984+0000","last_clean":"2024-02-19T16:21:22.214984+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:21:22.214984+0000","last_undegraded":"2024-02-19T16:21:22.214984+0000","last_fullsized":"2024-02-19T16:21:22.214984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877371+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:21:21.877371+0000","last_peered":"2024-02-19T16:21:21.877371+0000","last_clean":"2024-02-19T16:21:21.877371+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:21:21.877371+0000","last_undegraded":"2024-02-19T16:21:21.877371+0000","last_fullsized":"2024-02-19T16:21:21.877371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875468+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:21:21.875468+0000","last_peered":"2024-02-19T16:21:21.875468+0000","last_clean":"2024-02-19T16:21:21.875468+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:21:21.875468+0000","last_undegraded":"2024-02-19T16:21:21.875468+0000","last_fullsized":"2024-02-19T16:21:21.875468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.889760+0000","last_change":"2024-02-19T16:21:21.889760+0000","last_active":"2024-02-19T16:21:21.889760+0000","last_peered":"2024-02-19T16:21:21.889760+0000","last_clean":"2024-02-19T16:21:21.889760+0000","last_became_active":"2024-02-19T16:21:21.884230+0000","last_became_peered":"2024-02-19T16:21:21.884230+0000","last_unstale":"2024-02-19T16:21:21.889760+0000","last_undegraded":"2024-02-19T16:21:21.889760+0000","last_fullsized":"2024-02-19T16:21:21.889760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:48:45.580976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.747225+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:21:50.747225+0000","last_peered":"2024-02-19T16:21:50.747225+0000","last_clean":"2024-02-19T16:21:50.747225+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:21:50.747225+0000","last_undegraded":"2024-02-19T16:21:50.747225+0000","last_fullsized":"2024-02-19T16:21:50.747225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875427+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:21:21.875427+0000","last_peered":"2024-02-19T16:21:21.875427+0000","last_clean":"2024-02-19T16:21:21.875427+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:21:21.875427+0000","last_undegraded":"2024-02-19T16:21:21.875427+0000","last_fullsized":"2024-02-19T16:21:21.875427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881420+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:21:21.881420+0000","last_peered":"2024-02-19T16:21:21.881420+0000","last_clean":"2024-02-19T16:21:21.881420+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:21:21.881420+0000","last_undegraded":"2024-02-19T16:21:21.881420+0000","last_fullsized":"2024-02-19T16:21:21.881420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881196+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:21:21.881196+0000","last_peered":"2024-02-19T16:21:21.881196+0000","last_clean":"2024-02-19T16:21:21.881196+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:21:21.881196+0000","last_undegraded":"2024-02-19T16:21:21.881196+0000","last_fullsized":"2024-02-19T16:21:21.881196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878027+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:21:21.878027+0000","last_peered":"2024-02-19T16:21:21.878027+0000","last_clean":"2024-02-19T16:21:21.878027+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:21:21.878027+0000","last_undegraded":"2024-02-19T16:21:21.878027+0000","last_fullsized":"2024-02-19T16:21:21.878027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:48.392380+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:21:48.392380+0000","last_peered":"2024-02-19T16:21:48.392380+0000","last_clean":"2024-02-19T16:21:48.392380+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:21:48.392380+0000","last_undegraded":"2024-02-19T16:21:48.392380+0000","last_fullsized":"2024-02-19T16:21:48.392380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.737572+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:21:50.737572+0000","last_peered":"2024-02-19T16:21:50.737572+0000","last_clean":"2024-02-19T16:21:50.737572+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:21:50.737572+0000","last_undegraded":"2024-02-19T16:21:50.737572+0000","last_fullsized":"2024-02-19T16:21:50.737572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.885536+0000","last_change":"2024-02-19T16:21:21.885536+0000","last_active":"2024-02-19T16:21:21.885536+0000","last_peered":"2024-02-19T16:21:21.885536+0000","last_clean":"2024-02-19T16:21:21.885536+0000","last_became_active":"2024-02-19T16:21:21.885125+0000","last_became_peered":"2024-02-19T16:21:21.885125+0000","last_unstale":"2024-02-19T16:21:21.885536+0000","last_undegraded":"2024-02-19T16:21:21.885536+0000","last_fullsized":"2024-02-19T16:21:21.885536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:23:23.180070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.344872+0000","last_change":"2024-02-19T16:21:22.344872+0000","last_active":"2024-02-19T16:21:22.344872+0000","last_peered":"2024-02-19T16:21:22.344872+0000","last_clean":"2024-02-19T16:21:22.344872+0000","last_became_active":"2024-02-19T16:21:22.342317+0000","last_became_peered":"2024-02-19T16:21:22.342317+0000","last_unstale":"2024-02-19T16:21:22.344872+0000","last_undegraded":"2024-02-19T16:21:22.344872+0000","last_fullsized":"2024-02-19T16:21:22.344872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:53:45.080091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882622+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:21:21.882622+0000","last_peered":"2024-02-19T16:21:21.882622+0000","last_clean":"2024-02-19T16:21:21.882622+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:21:21.882622+0000","last_undegraded":"2024-02-19T16:21:21.882622+0000","last_fullsized":"2024-02-19T16:21:21.882622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883562+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:21:21.883562+0000","last_peered":"2024-02-19T16:21:21.883562+0000","last_clean":"2024-02-19T16:21:21.883562+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:21:21.883562+0000","last_undegraded":"2024-02-19T16:21:21.883562+0000","last_fullsized":"2024-02-19T16:21:21.883562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875280+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:21:21.875280+0000","last_peered":"2024-02-19T16:21:21.875280+0000","last_clean":"2024-02-19T16:21:21.875280+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:21:21.875280+0000","last_undegraded":"2024-02-19T16:21:21.875280+0000","last_fullsized":"2024-02-19T16:21:21.875280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762361+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:21:50.762361+0000","last_peered":"2024-02-19T16:21:50.762361+0000","last_clean":"2024-02-19T16:21:50.762361+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:21:50.762361+0000","last_undegraded":"2024-02-19T16:21:50.762361+0000","last_fullsized":"2024-02-19T16:21:50.762361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877516+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:21:21.877516+0000","last_peered":"2024-02-19T16:21:21.877516+0000","last_clean":"2024-02-19T16:21:21.877516+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:21:21.877516+0000","last_undegraded":"2024-02-19T16:21:21.877516+0000","last_fullsized":"2024-02-19T16:21:21.877516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879769+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:21:21.879769+0000","last_peered":"2024-02-19T16:21:21.879769+0000","last_clean":"2024-02-19T16:21:21.879769+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:21:21.879769+0000","last_undegraded":"2024-02-19T16:21:21.879769+0000","last_fullsized":"2024-02-19T16:21:21.879769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762502+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:21:50.762502+0000","last_peered":"2024-02-19T16:21:50.762502+0000","last_clean":"2024-02-19T16:21:50.762502+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:21:50.762502+0000","last_undegraded":"2024-02-19T16:21:50.762502+0000","last_fullsized":"2024-02-19T16:21:50.762502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214481+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:21:22.214481+0000","last_peered":"2024-02-19T16:21:22.214481+0000","last_clean":"2024-02-19T16:21:22.214481+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:21:22.214481+0000","last_undegraded":"2024-02-19T16:21:22.214481+0000","last_fullsized":"2024-02-19T16:21:22.214481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":99,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:47.391050+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:21:47.391050+0000","last_peered":"2024-02-19T16:21:47.391050+0000","last_clean":"2024-02-19T16:21:47.391050+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:21:47.391050+0000","last_undegraded":"2024-02-19T16:21:47.391050+0000","last_fullsized":"2024-02-19T16:21:47.391050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751486+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:21:50.751486+0000","last_peered":"2024-02-19T16:21:50.751486+0000","last_clean":"2024-02-19T16:21:50.751486+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:21:50.751486+0000","last_undegraded":"2024-02-19T16:21:50.751486+0000","last_fullsized":"2024-02-19T16:21:50.751486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878692+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:21:21.878692+0000","last_peered":"2024-02-19T16:21:21.878692+0000","last_clean":"2024-02-19T16:21:21.878692+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:21:21.878692+0000","last_undegraded":"2024-02-19T16:21:21.878692+0000","last_fullsized":"2024-02-19T16:21:21.878692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882239+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:21:21.882239+0000","last_peered":"2024-02-19T16:21:21.882239+0000","last_clean":"2024-02-19T16:21:21.882239+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:21:21.882239+0000","last_undegraded":"2024-02-19T16:21:21.882239+0000","last_fullsized":"2024-02-19T16:21:21.882239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877429+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:21:21.877429+0000","last_peered":"2024-02-19T16:21:21.877429+0000","last_clean":"2024-02-19T16:21:21.877429+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:21:21.877429+0000","last_undegraded":"2024-02-19T16:21:21.877429+0000","last_fullsized":"2024-02-19T16:21:21.877429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875579+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:21:21.875579+0000","last_peered":"2024-02-19T16:21:21.875579+0000","last_clean":"2024-02-19T16:21:21.875579+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:21:21.875579+0000","last_undegraded":"2024-02-19T16:21:21.875579+0000","last_fullsized":"2024-02-19T16:21:21.875579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881088+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:21:21.881088+0000","last_peered":"2024-02-19T16:21:21.881088+0000","last_clean":"2024-02-19T16:21:21.881088+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:21:21.881088+0000","last_undegraded":"2024-02-19T16:21:21.881088+0000","last_fullsized":"2024-02-19T16:21:21.881088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.744260+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:21:50.744260+0000","last_peered":"2024-02-19T16:21:50.744260+0000","last_clean":"2024-02-19T16:21:50.744260+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:21:50.744260+0000","last_undegraded":"2024-02-19T16:21:50.744260+0000","last_fullsized":"2024-02-19T16:21:50.744260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.885348+0000","last_change":"2024-02-19T16:21:21.885002+0000","last_active":"2024-02-19T16:21:21.885348+0000","last_peered":"2024-02-19T16:21:21.885348+0000","last_clean":"2024-02-19T16:21:21.885348+0000","last_became_active":"2024-02-19T16:21:21.884557+0000","last_became_peered":"2024-02-19T16:21:21.884557+0000","last_unstale":"2024-02-19T16:21:21.885348+0000","last_undegraded":"2024-02-19T16:21:21.885348+0000","last_fullsized":"2024-02-19T16:21:21.885348+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:08:31.859577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":202,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:48.393824+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:21:48.393824+0000","last_peered":"2024-02-19T16:21:48.393824+0000","last_clean":"2024-02-19T16:21:48.393824+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:21:48.393824+0000","last_undegraded":"2024-02-19T16:21:48.393824+0000","last_fullsized":"2024-02-19T16:21:48.393824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":143,"num_read_kb":132,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.743519+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:21:50.743519+0000","last_peered":"2024-02-19T16:21:50.743519+0000","last_clean":"2024-02-19T16:21:50.743519+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:21:50.743519+0000","last_undegraded":"2024-02-19T16:21:50.743519+0000","last_fullsized":"2024-02-19T16:21:50.743519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881519+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:21:21.881519+0000","last_peered":"2024-02-19T16:21:21.881519+0000","last_clean":"2024-02-19T16:21:21.881519+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:21:21.881519+0000","last_undegraded":"2024-02-19T16:21:21.881519+0000","last_fullsized":"2024-02-19T16:21:21.881519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877900+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:21:21.877900+0000","last_peered":"2024-02-19T16:21:21.877900+0000","last_clean":"2024-02-19T16:21:21.877900+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:21:21.877900+0000","last_undegraded":"2024-02-19T16:21:21.877900+0000","last_fullsized":"2024-02-19T16:21:21.877900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.763093+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:21:50.763093+0000","last_peered":"2024-02-19T16:21:50.763093+0000","last_clean":"2024-02-19T16:21:50.763093+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:21:50.763093+0000","last_undegraded":"2024-02-19T16:21:50.763093+0000","last_fullsized":"2024-02-19T16:21:50.763093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882811+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:21:21.882811+0000","last_peered":"2024-02-19T16:21:21.882811+0000","last_clean":"2024-02-19T16:21:21.882811+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:21:21.882811+0000","last_undegraded":"2024-02-19T16:21:21.882811+0000","last_fullsized":"2024-02-19T16:21:21.882811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881375+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:21:21.881375+0000","last_peered":"2024-02-19T16:21:21.881375+0000","last_clean":"2024-02-19T16:21:21.881375+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:21:21.881375+0000","last_undegraded":"2024-02-19T16:21:21.881375+0000","last_fullsized":"2024-02-19T16:21:21.881375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755871+0000","last_change":"2024-02-19T16:20:21.660468+0000","last_active":"2024-02-19T16:21:50.755871+0000","last_peered":"2024-02-19T16:21:50.755871+0000","last_clean":"2024-02-19T16:21:50.755871+0000","last_became_active":"2024-02-19T16:20:21.659988+0000","last_became_peered":"2024-02-19T16:20:21.659988+0000","last_unstale":"2024-02-19T16:21:50.755871+0000","last_undegraded":"2024-02-19T16:21:50.755871+0000","last_fullsized":"2024-02-19T16:21:50.755871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:57:08.316143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.216357+0000","last_change":"2024-02-19T16:21:22.216357+0000","last_active":"2024-02-19T16:21:22.216357+0000","last_peered":"2024-02-19T16:21:22.216357+0000","last_clean":"2024-02-19T16:21:22.216357+0000","last_became_active":"2024-02-19T16:21:22.215883+0000","last_became_peered":"2024-02-19T16:21:22.215883+0000","last_unstale":"2024-02-19T16:21:22.216357+0000","last_undegraded":"2024-02-19T16:21:22.216357+0000","last_fullsized":"2024-02-19T16:21:22.216357+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:32:43.235515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":107,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.407791+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:21:50.407791+0000","last_peered":"2024-02-19T16:21:50.407791+0000","last_clean":"2024-02-19T16:21:50.407791+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:21:50.407791+0000","last_undegraded":"2024-02-19T16:21:50.407791+0000","last_fullsized":"2024-02-19T16:21:50.407791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750819+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:21:50.750819+0000","last_peered":"2024-02-19T16:21:50.750819+0000","last_clean":"2024-02-19T16:21:50.750819+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:21:50.750819+0000","last_undegraded":"2024-02-19T16:21:50.750819+0000","last_fullsized":"2024-02-19T16:21:50.750819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875693+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:21:21.875693+0000","last_peered":"2024-02-19T16:21:21.875693+0000","last_clean":"2024-02-19T16:21:21.875693+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:21:21.875693+0000","last_undegraded":"2024-02-19T16:21:21.875693+0000","last_fullsized":"2024-02-19T16:21:21.875693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879364+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:21:21.879364+0000","last_peered":"2024-02-19T16:21:21.879364+0000","last_clean":"2024-02-19T16:21:21.879364+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:21:21.879364+0000","last_undegraded":"2024-02-19T16:21:21.879364+0000","last_fullsized":"2024-02-19T16:21:21.879364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879543+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:21:21.879543+0000","last_peered":"2024-02-19T16:21:21.879543+0000","last_clean":"2024-02-19T16:21:21.879543+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:21:21.879543+0000","last_undegraded":"2024-02-19T16:21:21.879543+0000","last_fullsized":"2024-02-19T16:21:21.879543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879449+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:21:21.879449+0000","last_peered":"2024-02-19T16:21:21.879449+0000","last_clean":"2024-02-19T16:21:21.879449+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:21:21.879449+0000","last_undegraded":"2024-02-19T16:21:21.879449+0000","last_fullsized":"2024-02-19T16:21:21.879449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881148+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:21:21.881148+0000","last_peered":"2024-02-19T16:21:21.881148+0000","last_clean":"2024-02-19T16:21:21.881148+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:21:21.881148+0000","last_undegraded":"2024-02-19T16:21:21.881148+0000","last_fullsized":"2024-02-19T16:21:21.881148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883488+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:21:21.883488+0000","last_peered":"2024-02-19T16:21:21.883488+0000","last_clean":"2024-02-19T16:21:21.883488+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:21:21.883488+0000","last_undegraded":"2024-02-19T16:21:21.883488+0000","last_fullsized":"2024-02-19T16:21:21.883488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879749+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:21:21.879749+0000","last_peered":"2024-02-19T16:21:21.879749+0000","last_clean":"2024-02-19T16:21:21.879749+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:21:21.879749+0000","last_undegraded":"2024-02-19T16:21:21.879749+0000","last_fullsized":"2024-02-19T16:21:21.879749+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879182+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:21:21.879182+0000","last_peered":"2024-02-19T16:21:21.879182+0000","last_clean":"2024-02-19T16:21:21.879182+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:21:21.879182+0000","last_undegraded":"2024-02-19T16:21:21.879182+0000","last_fullsized":"2024-02-19T16:21:21.879182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877849+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:21:21.877849+0000","last_peered":"2024-02-19T16:21:21.877849+0000","last_clean":"2024-02-19T16:21:21.877849+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:21:21.877849+0000","last_undegraded":"2024-02-19T16:21:21.877849+0000","last_fullsized":"2024-02-19T16:21:21.877849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875519+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:21:21.875519+0000","last_peered":"2024-02-19T16:21:21.875519+0000","last_clean":"2024-02-19T16:21:21.875519+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:21:21.875519+0000","last_undegraded":"2024-02-19T16:21:21.875519+0000","last_fullsized":"2024-02-19T16:21:21.875519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881611+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:21:21.881611+0000","last_peered":"2024-02-19T16:21:21.881611+0000","last_clean":"2024-02-19T16:21:21.881611+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:21:21.881611+0000","last_undegraded":"2024-02-19T16:21:21.881611+0000","last_fullsized":"2024-02-19T16:21:21.881611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877537+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:21:21.877537+0000","last_peered":"2024-02-19T16:21:21.877537+0000","last_clean":"2024-02-19T16:21:21.877537+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:21:21.877537+0000","last_undegraded":"2024-02-19T16:21:21.877537+0000","last_fullsized":"2024-02-19T16:21:21.877537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875704+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:21:21.875704+0000","last_peered":"2024-02-19T16:21:21.875704+0000","last_clean":"2024-02-19T16:21:21.875704+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:21:21.875704+0000","last_undegraded":"2024-02-19T16:21:21.875704+0000","last_fullsized":"2024-02-19T16:21:21.875704+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879963+0000","last_change":"2024-02-19T16:20:21.671091+0000","last_active":"2024-02-19T16:21:21.879963+0000","last_peered":"2024-02-19T16:21:21.879963+0000","last_clean":"2024-02-19T16:21:21.879963+0000","last_became_active":"2024-02-19T16:20:21.670762+0000","last_became_peered":"2024-02-19T16:20:21.670762+0000","last_unstale":"2024-02-19T16:21:21.879963+0000","last_undegraded":"2024-02-19T16:21:21.879963+0000","last_fullsized":"2024-02-19T16:21:21.879963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:14:35.707812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.708433+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:21:50.708433+0000","last_peered":"2024-02-19T16:21:50.708433+0000","last_clean":"2024-02-19T16:21:50.708433+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:21:50.708433+0000","last_undegraded":"2024-02-19T16:21:50.708433+0000","last_fullsized":"2024-02-19T16:21:50.708433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882228+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:21:21.882228+0000","last_peered":"2024-02-19T16:21:21.882228+0000","last_clean":"2024-02-19T16:21:21.882228+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:21:21.882228+0000","last_undegraded":"2024-02-19T16:21:21.882228+0000","last_fullsized":"2024-02-19T16:21:21.882228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.760249+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:21:50.760249+0000","last_peered":"2024-02-19T16:21:50.760249+0000","last_clean":"2024-02-19T16:21:50.760249+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:21:50.760249+0000","last_undegraded":"2024-02-19T16:21:50.760249+0000","last_fullsized":"2024-02-19T16:21:50.760249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875557+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:21:21.875557+0000","last_peered":"2024-02-19T16:21:21.875557+0000","last_clean":"2024-02-19T16:21:21.875557+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:21:21.875557+0000","last_undegraded":"2024-02-19T16:21:21.875557+0000","last_fullsized":"2024-02-19T16:21:21.875557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881457+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:21:21.881457+0000","last_peered":"2024-02-19T16:21:21.881457+0000","last_clean":"2024-02-19T16:21:21.881457+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:21:21.881457+0000","last_undegraded":"2024-02-19T16:21:21.881457+0000","last_fullsized":"2024-02-19T16:21:21.881457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881761+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:21:21.881761+0000","last_peered":"2024-02-19T16:21:21.881761+0000","last_clean":"2024-02-19T16:21:21.881761+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:21:21.881761+0000","last_undegraded":"2024-02-19T16:21:21.881761+0000","last_fullsized":"2024-02-19T16:21:21.881761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.745051+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:21:50.745051+0000","last_peered":"2024-02-19T16:21:50.745051+0000","last_clean":"2024-02-19T16:21:50.745051+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:21:50.745051+0000","last_undegraded":"2024-02-19T16:21:50.745051+0000","last_fullsized":"2024-02-19T16:21:50.745051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881524+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:21:21.881524+0000","last_peered":"2024-02-19T16:21:21.881524+0000","last_clean":"2024-02-19T16:21:21.881524+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:21:21.881524+0000","last_undegraded":"2024-02-19T16:21:21.881524+0000","last_fullsized":"2024-02-19T16:21:21.881524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883638+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:21:21.883638+0000","last_peered":"2024-02-19T16:21:21.883638+0000","last_clean":"2024-02-19T16:21:21.883638+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:21:21.883638+0000","last_undegraded":"2024-02-19T16:21:21.883638+0000","last_fullsized":"2024-02-19T16:21:21.883638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.880727+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:21:21.880727+0000","last_peered":"2024-02-19T16:21:21.880727+0000","last_clean":"2024-02-19T16:21:21.880727+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:21:21.880727+0000","last_undegraded":"2024-02-19T16:21:21.880727+0000","last_fullsized":"2024-02-19T16:21:21.880727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750393+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:21:50.750393+0000","last_peered":"2024-02-19T16:21:50.750393+0000","last_clean":"2024-02-19T16:21:50.750393+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:21:50.750393+0000","last_undegraded":"2024-02-19T16:21:50.750393+0000","last_fullsized":"2024-02-19T16:21:50.750393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214300+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:21:22.214300+0000","last_peered":"2024-02-19T16:21:22.214300+0000","last_clean":"2024-02-19T16:21:22.214300+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:21:22.214300+0000","last_undegraded":"2024-02-19T16:21:22.214300+0000","last_fullsized":"2024-02-19T16:21:22.214300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879525+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:21:21.879525+0000","last_peered":"2024-02-19T16:21:21.879525+0000","last_clean":"2024-02-19T16:21:21.879525+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:21:21.879525+0000","last_undegraded":"2024-02-19T16:21:21.879525+0000","last_fullsized":"2024-02-19T16:21:21.879525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.761716+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:21:50.761716+0000","last_peered":"2024-02-19T16:21:50.761716+0000","last_clean":"2024-02-19T16:21:50.761716+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:21:50.761716+0000","last_undegraded":"2024-02-19T16:21:50.761716+0000","last_fullsized":"2024-02-19T16:21:50.761716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877528+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:21:21.877528+0000","last_peered":"2024-02-19T16:21:21.877528+0000","last_clean":"2024-02-19T16:21:21.877528+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:21:21.877528+0000","last_undegraded":"2024-02-19T16:21:21.877528+0000","last_fullsized":"2024-02-19T16:21:21.877528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875857+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:21:21.875857+0000","last_peered":"2024-02-19T16:21:21.875857+0000","last_clean":"2024-02-19T16:21:21.875857+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:21:21.875857+0000","last_undegraded":"2024-02-19T16:21:21.875857+0000","last_fullsized":"2024-02-19T16:21:21.875857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877503+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:21:21.877503+0000","last_peered":"2024-02-19T16:21:21.877503+0000","last_clean":"2024-02-19T16:21:21.877503+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:21:21.877503+0000","last_undegraded":"2024-02-19T16:21:21.877503+0000","last_fullsized":"2024-02-19T16:21:21.877503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875847+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:21:21.875847+0000","last_peered":"2024-02-19T16:21:21.875847+0000","last_clean":"2024-02-19T16:21:21.875847+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:21:21.875847+0000","last_undegraded":"2024-02-19T16:21:21.875847+0000","last_fullsized":"2024-02-19T16:21:21.875847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.341280+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:21:22.341280+0000","last_peered":"2024-02-19T16:21:22.341280+0000","last_clean":"2024-02-19T16:21:22.341280+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:21:22.341280+0000","last_undegraded":"2024-02-19T16:21:22.341280+0000","last_fullsized":"2024-02-19T16:21:22.341280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.711824+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:21:50.711824+0000","last_peered":"2024-02-19T16:21:50.711824+0000","last_clean":"2024-02-19T16:21:50.711824+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:21:50.711824+0000","last_undegraded":"2024-02-19T16:21:50.711824+0000","last_fullsized":"2024-02-19T16:21:50.711824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.216182+0000","last_change":"2024-02-19T16:21:22.216182+0000","last_active":"2024-02-19T16:21:22.216182+0000","last_peered":"2024-02-19T16:21:22.216182+0000","last_clean":"2024-02-19T16:21:22.216182+0000","last_became_active":"2024-02-19T16:21:22.215705+0000","last_became_peered":"2024-02-19T16:21:22.215705+0000","last_unstale":"2024-02-19T16:21:22.216182+0000","last_undegraded":"2024-02-19T16:21:22.216182+0000","last_fullsized":"2024-02-19T16:21:22.216182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:29:29.574422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214589+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:21:22.214589+0000","last_peered":"2024-02-19T16:21:22.214589+0000","last_clean":"2024-02-19T16:21:22.214589+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:21:22.214589+0000","last_undegraded":"2024-02-19T16:21:22.214589+0000","last_fullsized":"2024-02-19T16:21:22.214589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881641+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:21:21.881641+0000","last_peered":"2024-02-19T16:21:21.881641+0000","last_clean":"2024-02-19T16:21:21.881641+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:21:21.881641+0000","last_undegraded":"2024-02-19T16:21:21.881641+0000","last_fullsized":"2024-02-19T16:21:21.881641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875710+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:21:21.875710+0000","last_peered":"2024-02-19T16:21:21.875710+0000","last_clean":"2024-02-19T16:21:21.875710+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:21:21.875710+0000","last_undegraded":"2024-02-19T16:21:21.875710+0000","last_fullsized":"2024-02-19T16:21:21.875710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877637+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:21:21.877637+0000","last_peered":"2024-02-19T16:21:21.877637+0000","last_clean":"2024-02-19T16:21:21.877637+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:21:21.877637+0000","last_undegraded":"2024-02-19T16:21:21.877637+0000","last_fullsized":"2024-02-19T16:21:21.877637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762059+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:21:50.762059+0000","last_peered":"2024-02-19T16:21:50.762059+0000","last_clean":"2024-02-19T16:21:50.762059+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:21:50.762059+0000","last_undegraded":"2024-02-19T16:21:50.762059+0000","last_fullsized":"2024-02-19T16:21:50.762059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875771+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:21:21.875771+0000","last_peered":"2024-02-19T16:21:21.875771+0000","last_clean":"2024-02-19T16:21:21.875771+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:21:21.875771+0000","last_undegraded":"2024-02-19T16:21:21.875771+0000","last_fullsized":"2024-02-19T16:21:21.875771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.894667+0000","last_change":"2024-02-19T16:21:21.894342+0000","last_active":"2024-02-19T16:21:21.894667+0000","last_peered":"2024-02-19T16:21:21.894667+0000","last_clean":"2024-02-19T16:21:21.894667+0000","last_became_active":"2024-02-19T16:21:21.885142+0000","last_became_peered":"2024-02-19T16:21:21.885142+0000","last_unstale":"2024-02-19T16:21:21.894667+0000","last_undegraded":"2024-02-19T16:21:21.894667+0000","last_fullsized":"2024-02-19T16:21:21.894667+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:19:30.846573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755779+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:21:50.755779+0000","last_peered":"2024-02-19T16:21:50.755779+0000","last_clean":"2024-02-19T16:21:50.755779+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:21:50.755779+0000","last_undegraded":"2024-02-19T16:21:50.755779+0000","last_fullsized":"2024-02-19T16:21:50.755779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.341453+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:21:22.341453+0000","last_peered":"2024-02-19T16:21:22.341453+0000","last_clean":"2024-02-19T16:21:22.341453+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:21:22.341453+0000","last_undegraded":"2024-02-19T16:21:22.341453+0000","last_fullsized":"2024-02-19T16:21:22.341453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878166+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:21:21.878166+0000","last_peered":"2024-02-19T16:21:21.878166+0000","last_clean":"2024-02-19T16:21:21.878166+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:21:21.878166+0000","last_undegraded":"2024-02-19T16:21:21.878166+0000","last_fullsized":"2024-02-19T16:21:21.878166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756738+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:21:50.756738+0000","last_peered":"2024-02-19T16:21:50.756738+0000","last_clean":"2024-02-19T16:21:50.756738+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:21:50.756738+0000","last_undegraded":"2024-02-19T16:21:50.756738+0000","last_fullsized":"2024-02-19T16:21:50.756738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877327+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:21:21.877327+0000","last_peered":"2024-02-19T16:21:21.877327+0000","last_clean":"2024-02-19T16:21:21.877327+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:21:21.877327+0000","last_undegraded":"2024-02-19T16:21:21.877327+0000","last_fullsized":"2024-02-19T16:21:21.877327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881292+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:21:21.881292+0000","last_peered":"2024-02-19T16:21:21.881292+0000","last_clean":"2024-02-19T16:21:21.881292+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:21:21.881292+0000","last_undegraded":"2024-02-19T16:21:21.881292+0000","last_fullsized":"2024-02-19T16:21:21.881292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751124+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:21:50.751124+0000","last_peered":"2024-02-19T16:21:50.751124+0000","last_clean":"2024-02-19T16:21:50.751124+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:21:50.751124+0000","last_undegraded":"2024-02-19T16:21:50.751124+0000","last_fullsized":"2024-02-19T16:21:50.751124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878995+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:21:21.878995+0000","last_peered":"2024-02-19T16:21:21.878995+0000","last_clean":"2024-02-19T16:21:21.878995+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:21:21.878995+0000","last_undegraded":"2024-02-19T16:21:21.878995+0000","last_fullsized":"2024-02-19T16:21:21.878995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875705+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:21:21.875705+0000","last_peered":"2024-02-19T16:21:21.875705+0000","last_clean":"2024-02-19T16:21:21.875705+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:21:21.875705+0000","last_undegraded":"2024-02-19T16:21:21.875705+0000","last_fullsized":"2024-02-19T16:21:21.875705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751658+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:21:50.751658+0000","last_peered":"2024-02-19T16:21:50.751658+0000","last_clean":"2024-02-19T16:21:50.751658+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:21:50.751658+0000","last_undegraded":"2024-02-19T16:21:50.751658+0000","last_fullsized":"2024-02-19T16:21:50.751658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881732+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:21:21.881732+0000","last_peered":"2024-02-19T16:21:21.881732+0000","last_clean":"2024-02-19T16:21:21.881732+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:21:21.881732+0000","last_undegraded":"2024-02-19T16:21:21.881732+0000","last_fullsized":"2024-02-19T16:21:21.881732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877918+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:21:21.877918+0000","last_peered":"2024-02-19T16:21:21.877918+0000","last_clean":"2024-02-19T16:21:21.877918+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:21:21.877918+0000","last_undegraded":"2024-02-19T16:21:21.877918+0000","last_fullsized":"2024-02-19T16:21:21.877918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877523+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:21:21.877523+0000","last_peered":"2024-02-19T16:21:21.877523+0000","last_clean":"2024-02-19T16:21:21.877523+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:21:21.877523+0000","last_undegraded":"2024-02-19T16:21:21.877523+0000","last_fullsized":"2024-02-19T16:21:21.877523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.884814+0000","last_change":"2024-02-19T16:21:21.884814+0000","last_active":"2024-02-19T16:21:21.884814+0000","last_peered":"2024-02-19T16:21:21.884814+0000","last_clean":"2024-02-19T16:21:21.884814+0000","last_became_active":"2024-02-19T16:21:21.884410+0000","last_became_peered":"2024-02-19T16:21:21.884410+0000","last_unstale":"2024-02-19T16:21:21.884814+0000","last_undegraded":"2024-02-19T16:21:21.884814+0000","last_fullsized":"2024-02-19T16:21:21.884814+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:39:44.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755441+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:21:50.755441+0000","last_peered":"2024-02-19T16:21:50.755441+0000","last_clean":"2024-02-19T16:21:50.755441+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:21:50.755441+0000","last_undegraded":"2024-02-19T16:21:50.755441+0000","last_fullsized":"2024-02-19T16:21:50.755441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883311+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:21:21.883311+0000","last_peered":"2024-02-19T16:21:21.883311+0000","last_clean":"2024-02-19T16:21:21.883311+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:21:21.883311+0000","last_undegraded":"2024-02-19T16:21:21.883311+0000","last_fullsized":"2024-02-19T16:21:21.883311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878159+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:21:21.878159+0000","last_peered":"2024-02-19T16:21:21.878159+0000","last_clean":"2024-02-19T16:21:21.878159+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:21:21.878159+0000","last_undegraded":"2024-02-19T16:21:21.878159+0000","last_fullsized":"2024-02-19T16:21:21.878159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.758202+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:21:50.758202+0000","last_peered":"2024-02-19T16:21:50.758202+0000","last_clean":"2024-02-19T16:21:50.758202+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:21:50.758202+0000","last_undegraded":"2024-02-19T16:21:50.758202+0000","last_fullsized":"2024-02-19T16:21:50.758202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879873+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:21:21.879873+0000","last_peered":"2024-02-19T16:21:21.879873+0000","last_clean":"2024-02-19T16:21:21.879873+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:21:21.879873+0000","last_undegraded":"2024-02-19T16:21:21.879873+0000","last_fullsized":"2024-02-19T16:21:21.879873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.752750+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:21:50.752750+0000","last_peered":"2024-02-19T16:21:50.752750+0000","last_clean":"2024-02-19T16:21:50.752750+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:21:50.752750+0000","last_undegraded":"2024-02-19T16:21:50.752750+0000","last_fullsized":"2024-02-19T16:21:50.752750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751742+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:21:50.751742+0000","last_peered":"2024-02-19T16:21:50.751742+0000","last_clean":"2024-02-19T16:21:50.751742+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:21:50.751742+0000","last_undegraded":"2024-02-19T16:21:50.751742+0000","last_fullsized":"2024-02-19T16:21:50.751742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.758043+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:21:50.758043+0000","last_peered":"2024-02-19T16:21:50.758043+0000","last_clean":"2024-02-19T16:21:50.758043+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:21:50.758043+0000","last_undegraded":"2024-02-19T16:21:50.758043+0000","last_fullsized":"2024-02-19T16:21:50.758043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882974+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:21:21.882974+0000","last_peered":"2024-02-19T16:21:21.882974+0000","last_clean":"2024-02-19T16:21:21.882974+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:21:21.882974+0000","last_undegraded":"2024-02-19T16:21:21.882974+0000","last_fullsized":"2024-02-19T16:21:21.882974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65917,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":438,"num_read_kb":288,"num_write":165,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":197859,"data_compressed":5154,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":513792,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.56599999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.53500000000000003,"15min":0.55400000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52800000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.371},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.53200000000000003,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56499999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.55900000000000005,"15min":0.57599999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.54200000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.57899999999999996,"15min":0.60499999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.54300000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.40300000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.50600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.32000000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.66900000000000004,"5min":0.998,"15min":0.998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.63100000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.66300000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68799999999999994,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.55700000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.497,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.67700000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.88700000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59899999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.72199999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.73299999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996}]}]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1162644,"data_compressed":61171,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.315}]},{"osd":2,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71499999999999997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.438}]},{"osd":3,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55300000000000005}]},{"osd":4,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.55000000000000004,"15min":0.56499999999999995},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.73599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80700000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.35399999999999998}]},{"osd":5,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.82199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.62,"15min":0.63600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66700000000000004,"15min":0.68000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97399999999999998,"5min":0.999,"15min":0.999},"last":0.629},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":1.028,"15min":1.028},"last":0.56899999999999995}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.72799999999999998,"15min":0.75},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91600000000000004,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.90000000000000002,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54300000000000004}]},{"osd":9,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.59899999999999998}]},{"osd":11,"last update":"Mon Feb 19 16:21:37 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.56399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53200000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":517542,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71599999999999997}]},{"osd":1,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.626,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.55600000000000005,"15min":0.58099999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.35899999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.627},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66000000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72699999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.66400000000000003,"15min":0.69399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.749,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45300000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82099999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.73299999999999998,"15min":0.78000000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76000000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.38900000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67700000000000005,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":1.169,"15min":1.169},"last":0.57099999999999995}]},{"osd":10,"last update":"Mon Feb 19 16:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.77600000000000002,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.86099999999999999,"5min":1.278,"15min":1.278},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.75600000000000001,"15min":0.77400000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81000000000000005,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55000000000000004}]},{"osd":11,"last update":"Mon Feb 19 16:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58599999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":513366,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63800000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55700000000000005,"15min":0.57899999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.875,"15min":0.875},"last":0.505},{"interface":"front","average":{"1min":0.498,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.64800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41799999999999998,"15min":0.42899999999999999},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57999999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.55600000000000005,"15min":0.57399999999999995},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60699999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.39300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.53300000000000003,"15min":0.55800000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.56599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.56399999999999995,"15min":0.58899999999999997},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.443}]},{"osd":6,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.62,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.68100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.49199999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.876,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93400000000000005,"5min":1.393,"15min":1.393},"last":0.52000000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.52200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.73599999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58499999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63600000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.623},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65300000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1108834,"data_compressed":59453,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.315},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.27800000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39700000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75800000000000001,"5min":0.87,"15min":0.87},"last":0.42999999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71399999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.379},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.45600000000000002},"min":{"1min":0.39500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.29899999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91000000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.50900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49099999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67400000000000004,"5min":0.997,"15min":0.997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70099999999999996,"15min":0.71499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.70599999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55200000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60499999999999998}]},{"osd":11,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.62},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.66800000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":508622,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41599999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.37},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.504}]},{"osd":3,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.442},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.56299999999999994}]},{"osd":7,"last update":"Mon Feb 19 16:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.626}]},{"osd":8,"last update":"Mon Feb 19 16:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92900000000000005,"5min":1.014,"15min":1.014},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.59199999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96999999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69699999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":570556,"data_compressed":25413,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.51300000000000001},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32600000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.66500000000000004,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.309,"15min":0.309},"max":{"1min":14.067,"5min":14.067,"15min":14.067},"last":0.435}]},{"osd":2,"last update":"Mon Feb 19 16:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.311},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54000000000000004,"15min":0.496},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72999999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.51800000000000002,"15min":0.49299999999999999},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41599999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.41399999999999998,"15min":0.376},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59999999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44400000000000001,"15min":0.40500000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38300000000000001}]},{"osd":5,"last update":"Mon Feb 19 16:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.50800000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.373,"5min":0.316,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.373},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.496}]},{"osd":8,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66300000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66100000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53300000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.82099999999999995,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":14.105,"5min":14.105,"15min":14.105},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71099999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61399999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.84699999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59899999999999998}]},{"osd":11,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.623}]}]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":510010,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79100000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63200000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.311}]},{"osd":2,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72599999999999998,"5min":0.874,"15min":0.874},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.38,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.377}]},{"osd":5,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.497},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85099999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Feb 19 16:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57099999999999995}]},{"osd":8,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89500000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53800000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.87,"15min":0.87},"last":0.622},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85199999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.63300000000000001}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95499999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48399999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.65000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":571657,"data_compressed":25413,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Feb 19 16:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.48599999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.5},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.53900000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Mon Feb 19 16:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.52800000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48999999999999999,"15min":0.47799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60299999999999998,"5min":1.026,"15min":1.026},"last":0.39400000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.629,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71199999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60599999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.504}]},{"osd":8,"last update":"Mon Feb 19 16:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.623,"15min":0.64400000000000002},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97299999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69399999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.755,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.63700000000000001}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.56899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.625}]},{"osd":11,"last update":"Mon Feb 19 16:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":508816,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.35999999999999999},"min":{"1min":0.29099999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.40400000000000003,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69299999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.76000000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75700000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82399999999999995,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77800000000000002,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57599999999999996}]},{"osd":5,"last update":"Mon Feb 19 16:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61899999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67500000000000004}]},{"osd":6,"last update":"Mon Feb 19 16:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90800000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.624},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.87,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72499999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91400000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95299999999999996,"5min":1.018,"15min":1.018},"last":0.71399999999999997}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.05,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.60999999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.77300000000000002,"15min":0.78000000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":1.03,"15min":1.03},"last":0.74099999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77900000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888131,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":513804,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.26300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.34899999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.373,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.51500000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.377},{"interface":"front","average":{"1min":0.379,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":1.03,"15min":1.03},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.60499999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.313},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.39100000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79700000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56399999999999995}]},{"osd":8,"last update":"Mon Feb 19 16:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.624},"min":{"1min":0.47999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.502},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85199999999999998,"5min":1.073,"15min":1.073},"last":0.46999999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":0.64100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.70299999999999996,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.65300000000000002,"5min":1.044,"15min":1.044},"last":0.51500000000000001}]},{"osd":10,"last update":"Mon Feb 19 16:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.442},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.53200000000000003}]},{"osd":11,"last update":"Mon Feb 19 16:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.623}]}]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1104472,"data_compressed":59453,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32300000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.79300000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.51000000000000001,"15min":0.51700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73299999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53400000000000003,"15min":0.52100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64600000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.55000000000000004,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46899999999999997}]},{"osd":2,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.44800000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.372},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.498,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78000000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73399999999999999,"5min":0.871,"15min":0.871},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.437}]},{"osd":6,"last update":"Mon Feb 19 16:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.65600000000000003,"15min":0.59399999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.495}]},{"osd":7,"last update":"Mon Feb 19 16:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78400000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55200000000000005}]},{"osd":8,"last update":"Mon Feb 19 16:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63100000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56200000000000006}]},{"osd":9,"last update":"Mon Feb 19 16:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66300000000000003,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.74199999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.626}]},{"osd":10,"last update":"Mon Feb 19 16:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.873,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.71399999999999997,"15min":0.73199999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":62024,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63125,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63744,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:21:54.929 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph pg dump --format=json 2024-02-19T16:21:55.186 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:55.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:55 smithi191 ceph-mon[37203]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:21:55.561 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:55 smithi008 ceph-mon[32573]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:21:55.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:55 smithi156 ceph-mon[36502]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:21:56.340 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:56 smithi008 ceph-mon[32573]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:21:56.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:56 smithi191 ceph-mon[37203]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:21:56.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:56 smithi156 ceph-mon[36502]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:21:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 -- 172.21.15.8:0/2326482898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d6406b080 msgr2=0x7f9d6410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2326482898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d6406b080 0x7f9d6410eea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b0d0 tx=0x7f9d5c00b3e0 comp rx=0 tx=0).stop 2024-02-19T16:21:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 -- 172.21.15.8:0/2326482898 shutdown_connections 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2326482898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d6406b080 0x7f9d6410eea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2326482898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d64108060 0x7f9d64108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2326482898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 0x7f9d6406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 -- 172.21.15.8:0/2326482898 >> 172.21.15.8:0/2326482898 conn(0x7f9d64076f00 msgr2=0x7f9d64077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 -- 172.21.15.8:0/2326482898 shutdown_connections 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 -- 172.21.15.8:0/2326482898 wait complete. 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 Processor -- start 2024-02-19T16:21:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.891+0000 7f9d6a353700 1 -- start start 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d6a353700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 0x7f9d641a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d6a353700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d6406b080 0x7f9d641a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d6a353700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 0x7f9d6419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d6a353700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419ae20 con 0x7f9d64108060 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d6a353700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419af90 con 0x7f9d6406a6e0 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d6a353700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419b150 con 0x7f9d6406b080 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d688f0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 0x7f9d6419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d688f0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 0x7f9d6419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44766/0 (socket says 172.21.15.8:44766) 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d63fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 0x7f9d641a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d688f0700 1 -- 172.21.15.8:0/2557422580 learned_addr learned my addr 172.21.15.8:0/2557422580 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.892+0000 7f9d637fe700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d6406b080 0x7f9d641a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d688f0700 1 -- 172.21.15.8:0/2557422580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d6406b080 msgr2=0x7f9d641a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d688f0700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d6406b080 0x7f9d641a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d688f0700 1 -- 172.21.15.8:0/2557422580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 msgr2=0x7f9d641a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d688f0700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 0x7f9d641a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d688f0700 1 -- 172.21.15.8:0/2557422580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c009d00 con 0x7f9d64108060 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d63fff700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 0x7f9d641a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d688f0700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 0x7f9d6419a670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00d010 tx=0x7f9d5c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c00e040 con 0x7f9d64108060 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d5c0079d0 con 0x7f9d64108060 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6419b3d0 con 0x7f9d64108060 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c01d6e0 con 0x7f9d64108060 2024-02-19T16:21:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.893+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d641a7520 con 0x7f9d64108060 2024-02-19T16:21:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.894+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d64103710 con 0x7f9d64108060 2024-02-19T16:21:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.895+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9d5c003a00 con 0x7f9d64108060 2024-02-19T16:21:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.896+0000 7f9d617fa700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d4c0a21d0 0x7f9d4c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:21:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.896+0000 7f9d63fff700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d4c0a21d0 0x7f9d4c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:21:56.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.896+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f9d5c0cec70 con 0x7f9d64108060 2024-02-19T16:21:56.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.896+0000 7f9d63fff700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d4c0a21d0 0x7f9d4c0a4690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009fd0 tx=0x7f9d54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:21:56.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:56.900+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9d5c097aa0 con 0x7f9d64108060 2024-02-19T16:21:57.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.099+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d641120d0 con 0x7f9d4c0a21d0 2024-02-19T16:21:57.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.112+0000 7f9d617fa700 1 -- 172.21.15.8:0/2557422580 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428633 (secure 0 0 0) 0x7f9d641120d0 con 0x7f9d4c0a21d0 2024-02-19T16:21:57.118 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d4c0a21d0 msgr2=0x7f9d4c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d4c0a21d0 0x7f9d4c0a4690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009fd0 tx=0x7f9d54009260 comp rx=0 tx=0).stop 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 msgr2=0x7f9d6419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 0x7f9d6419a670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00d010 tx=0x7f9d5c00bd50 comp rx=0 tx=0).stop 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 shutdown_connections 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d4c0a21d0 0x7f9d4c0a4690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:57.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d64108060 0x7f9d6419a670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:57.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6406a6e0 0x7f9d641a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:57.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 --2- 172.21.15.8:0/2557422580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d6406b080 0x7f9d641a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:21:57.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 >> 172.21.15.8:0/2557422580 conn(0x7f9d64076f00 msgr2=0x7f9d6410d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:21:57.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 shutdown_connections 2024-02-19T16:21:57.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:21:57.115+0000 7f9d6a353700 1 -- 172.21.15.8:0/2557422580 wait complete. 2024-02-19T16:21:57.123 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:21:58.049 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-02-19T16:21:56.736929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656768,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":606,"num_read_kb":424,"num_write":351,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378172,"kb_used_data":12380,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563652,"statfs":{"total":1151940427776,"available":1151553179648,"internally_reserved":0,"allocated":12677120,"data_stored":8104754,"data_compressed":396768,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":37,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750365+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:21:50.750365+0000","last_peered":"2024-02-19T16:21:50.750365+0000","last_clean":"2024-02-19T16:21:50.750365+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:21:50.750365+0000","last_undegraded":"2024-02-19T16:21:50.750365+0000","last_fullsized":"2024-02-19T16:21:50.750365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.746983+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:21:50.746983+0000","last_peered":"2024-02-19T16:21:50.746983+0000","last_clean":"2024-02-19T16:21:50.746983+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:21:50.746983+0000","last_undegraded":"2024-02-19T16:21:50.746983+0000","last_fullsized":"2024-02-19T16:21:50.746983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.757745+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:21:50.757745+0000","last_peered":"2024-02-19T16:21:50.757745+0000","last_clean":"2024-02-19T16:21:50.757745+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:21:50.757745+0000","last_undegraded":"2024-02-19T16:21:50.757745+0000","last_fullsized":"2024-02-19T16:21:50.757745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.749334+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:21:50.749334+0000","last_peered":"2024-02-19T16:21:50.749334+0000","last_clean":"2024-02-19T16:21:50.749334+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:21:50.749334+0000","last_undegraded":"2024-02-19T16:21:50.749334+0000","last_fullsized":"2024-02-19T16:21:50.749334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.740400+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:21:50.740400+0000","last_peered":"2024-02-19T16:21:50.740400+0000","last_clean":"2024-02-19T16:21:50.740400+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:21:50.740400+0000","last_undegraded":"2024-02-19T16:21:50.740400+0000","last_fullsized":"2024-02-19T16:21:50.740400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.743477+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:21:50.743477+0000","last_peered":"2024-02-19T16:21:50.743477+0000","last_clean":"2024-02-19T16:21:50.743477+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:21:50.743477+0000","last_undegraded":"2024-02-19T16:21:50.743477+0000","last_fullsized":"2024-02-19T16:21:50.743477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750666+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:21:50.750666+0000","last_peered":"2024-02-19T16:21:50.750666+0000","last_clean":"2024-02-19T16:21:50.750666+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:21:50.750666+0000","last_undegraded":"2024-02-19T16:21:50.750666+0000","last_fullsized":"2024-02-19T16:21:50.750666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762122+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:21:50.762122+0000","last_peered":"2024-02-19T16:21:50.762122+0000","last_clean":"2024-02-19T16:21:50.762122+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:21:50.762122+0000","last_undegraded":"2024-02-19T16:21:50.762122+0000","last_fullsized":"2024-02-19T16:21:50.762122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756040+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:21:50.756040+0000","last_peered":"2024-02-19T16:21:50.756040+0000","last_clean":"2024-02-19T16:21:50.756040+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:21:50.756040+0000","last_undegraded":"2024-02-19T16:21:50.756040+0000","last_fullsized":"2024-02-19T16:21:50.756040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.752269+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:21:50.752269+0000","last_peered":"2024-02-19T16:21:50.752269+0000","last_clean":"2024-02-19T16:21:50.752269+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:21:50.752269+0000","last_undegraded":"2024-02-19T16:21:50.752269+0000","last_fullsized":"2024-02-19T16:21:50.752269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756737+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:21:50.756737+0000","last_peered":"2024-02-19T16:21:50.756737+0000","last_clean":"2024-02-19T16:21:50.756737+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:21:50.756737+0000","last_undegraded":"2024-02-19T16:21:50.756737+0000","last_fullsized":"2024-02-19T16:21:50.756737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.740107+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:21:50.740107+0000","last_peered":"2024-02-19T16:21:50.740107+0000","last_clean":"2024-02-19T16:21:50.740107+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:21:50.740107+0000","last_undegraded":"2024-02-19T16:21:50.740107+0000","last_fullsized":"2024-02-19T16:21:50.740107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755017+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:21:50.755017+0000","last_peered":"2024-02-19T16:21:50.755017+0000","last_clean":"2024-02-19T16:21:50.755017+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:21:50.755017+0000","last_undegraded":"2024-02-19T16:21:50.755017+0000","last_fullsized":"2024-02-19T16:21:50.755017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750660+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:21:50.750660+0000","last_peered":"2024-02-19T16:21:50.750660+0000","last_clean":"2024-02-19T16:21:50.750660+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:21:50.750660+0000","last_undegraded":"2024-02-19T16:21:50.750660+0000","last_fullsized":"2024-02-19T16:21:50.750660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751048+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:21:50.751048+0000","last_peered":"2024-02-19T16:21:50.751048+0000","last_clean":"2024-02-19T16:21:50.751048+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:21:50.751048+0000","last_undegraded":"2024-02-19T16:21:50.751048+0000","last_fullsized":"2024-02-19T16:21:50.751048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756193+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:21:50.756193+0000","last_peered":"2024-02-19T16:21:50.756193+0000","last_clean":"2024-02-19T16:21:50.756193+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:21:50.756193+0000","last_undegraded":"2024-02-19T16:21:50.756193+0000","last_fullsized":"2024-02-19T16:21:50.756193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750853+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:21:50.750853+0000","last_peered":"2024-02-19T16:21:50.750853+0000","last_clean":"2024-02-19T16:21:50.750853+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:21:50.750853+0000","last_undegraded":"2024-02-19T16:21:50.750853+0000","last_fullsized":"2024-02-19T16:21:50.750853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.748335+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:21:50.748335+0000","last_peered":"2024-02-19T16:21:50.748335+0000","last_clean":"2024-02-19T16:21:50.748335+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:21:50.748335+0000","last_undegraded":"2024-02-19T16:21:50.748335+0000","last_fullsized":"2024-02-19T16:21:50.748335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.752699+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:21:50.752699+0000","last_peered":"2024-02-19T16:21:50.752699+0000","last_clean":"2024-02-19T16:21:50.752699+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:21:50.752699+0000","last_undegraded":"2024-02-19T16:21:50.752699+0000","last_fullsized":"2024-02-19T16:21:50.752699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877804+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:21:21.877804+0000","last_peered":"2024-02-19T16:21:21.877804+0000","last_clean":"2024-02-19T16:21:21.877804+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:21:21.877804+0000","last_undegraded":"2024-02-19T16:21:21.877804+0000","last_fullsized":"2024-02-19T16:21:21.877804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.892098+0000","last_change":"2024-02-19T16:21:21.892098+0000","last_active":"2024-02-19T16:21:21.892098+0000","last_peered":"2024-02-19T16:21:21.892098+0000","last_clean":"2024-02-19T16:21:21.892098+0000","last_became_active":"2024-02-19T16:21:21.884753+0000","last_became_peered":"2024-02-19T16:21:21.884753+0000","last_unstale":"2024-02-19T16:21:21.892098+0000","last_undegraded":"2024-02-19T16:21:21.892098+0000","last_fullsized":"2024-02-19T16:21:21.892098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:24.360019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.760365+0000","last_change":"2024-02-19T16:20:21.662054+0000","last_active":"2024-02-19T16:21:50.760365+0000","last_peered":"2024-02-19T16:21:50.760365+0000","last_clean":"2024-02-19T16:21:50.760365+0000","last_became_active":"2024-02-19T16:20:21.661721+0000","last_became_peered":"2024-02-19T16:20:21.661721+0000","last_unstale":"2024-02-19T16:21:50.760365+0000","last_undegraded":"2024-02-19T16:21:50.760365+0000","last_fullsized":"2024-02-19T16:21:50.760365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:27:13.322445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875332+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:21:21.875332+0000","last_peered":"2024-02-19T16:21:21.875332+0000","last_clean":"2024-02-19T16:21:21.875332+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:21:21.875332+0000","last_undegraded":"2024-02-19T16:21:21.875332+0000","last_fullsized":"2024-02-19T16:21:21.875332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214984+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:21:22.214984+0000","last_peered":"2024-02-19T16:21:22.214984+0000","last_clean":"2024-02-19T16:21:22.214984+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:21:22.214984+0000","last_undegraded":"2024-02-19T16:21:22.214984+0000","last_fullsized":"2024-02-19T16:21:22.214984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.757535+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:21:50.757535+0000","last_peered":"2024-02-19T16:21:50.757535+0000","last_clean":"2024-02-19T16:21:50.757535+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:21:50.757535+0000","last_undegraded":"2024-02-19T16:21:50.757535+0000","last_fullsized":"2024-02-19T16:21:50.757535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875468+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:21:21.875468+0000","last_peered":"2024-02-19T16:21:21.875468+0000","last_clean":"2024-02-19T16:21:21.875468+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:21:21.875468+0000","last_undegraded":"2024-02-19T16:21:21.875468+0000","last_fullsized":"2024-02-19T16:21:21.875468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.889760+0000","last_change":"2024-02-19T16:21:21.889760+0000","last_active":"2024-02-19T16:21:21.889760+0000","last_peered":"2024-02-19T16:21:21.889760+0000","last_clean":"2024-02-19T16:21:21.889760+0000","last_became_active":"2024-02-19T16:21:21.884230+0000","last_became_peered":"2024-02-19T16:21:21.884230+0000","last_unstale":"2024-02-19T16:21:21.889760+0000","last_undegraded":"2024-02-19T16:21:21.889760+0000","last_fullsized":"2024-02-19T16:21:21.889760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:48:45.580976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.747225+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:21:50.747225+0000","last_peered":"2024-02-19T16:21:50.747225+0000","last_clean":"2024-02-19T16:21:50.747225+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:21:50.747225+0000","last_undegraded":"2024-02-19T16:21:50.747225+0000","last_fullsized":"2024-02-19T16:21:50.747225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875427+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:21:21.875427+0000","last_peered":"2024-02-19T16:21:21.875427+0000","last_clean":"2024-02-19T16:21:21.875427+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:21:21.875427+0000","last_undegraded":"2024-02-19T16:21:21.875427+0000","last_fullsized":"2024-02-19T16:21:21.875427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881420+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:21:21.881420+0000","last_peered":"2024-02-19T16:21:21.881420+0000","last_clean":"2024-02-19T16:21:21.881420+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:21:21.881420+0000","last_undegraded":"2024-02-19T16:21:21.881420+0000","last_fullsized":"2024-02-19T16:21:21.881420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.757057+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:21:50.757057+0000","last_peered":"2024-02-19T16:21:50.757057+0000","last_clean":"2024-02-19T16:21:50.757057+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:21:50.757057+0000","last_undegraded":"2024-02-19T16:21:50.757057+0000","last_fullsized":"2024-02-19T16:21:50.757057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878027+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:21:21.878027+0000","last_peered":"2024-02-19T16:21:21.878027+0000","last_clean":"2024-02-19T16:21:21.878027+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:21:21.878027+0000","last_undegraded":"2024-02-19T16:21:21.878027+0000","last_fullsized":"2024-02-19T16:21:21.878027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:53.397716+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:21:53.397716+0000","last_peered":"2024-02-19T16:21:53.397716+0000","last_clean":"2024-02-19T16:21:53.397716+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:21:53.397716+0000","last_undegraded":"2024-02-19T16:21:53.397716+0000","last_fullsized":"2024-02-19T16:21:53.397716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.737572+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:21:50.737572+0000","last_peered":"2024-02-19T16:21:50.737572+0000","last_clean":"2024-02-19T16:21:50.737572+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:21:50.737572+0000","last_undegraded":"2024-02-19T16:21:50.737572+0000","last_fullsized":"2024-02-19T16:21:50.737572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.885536+0000","last_change":"2024-02-19T16:21:21.885536+0000","last_active":"2024-02-19T16:21:21.885536+0000","last_peered":"2024-02-19T16:21:21.885536+0000","last_clean":"2024-02-19T16:21:21.885536+0000","last_became_active":"2024-02-19T16:21:21.885125+0000","last_became_peered":"2024-02-19T16:21:21.885125+0000","last_unstale":"2024-02-19T16:21:21.885536+0000","last_undegraded":"2024-02-19T16:21:21.885536+0000","last_fullsized":"2024-02-19T16:21:21.885536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:23:23.180070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.344872+0000","last_change":"2024-02-19T16:21:22.344872+0000","last_active":"2024-02-19T16:21:22.344872+0000","last_peered":"2024-02-19T16:21:22.344872+0000","last_clean":"2024-02-19T16:21:22.344872+0000","last_became_active":"2024-02-19T16:21:22.342317+0000","last_became_peered":"2024-02-19T16:21:22.342317+0000","last_unstale":"2024-02-19T16:21:22.344872+0000","last_undegraded":"2024-02-19T16:21:22.344872+0000","last_fullsized":"2024-02-19T16:21:22.344872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:53:45.080091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.760808+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:21:50.760808+0000","last_peered":"2024-02-19T16:21:50.760808+0000","last_clean":"2024-02-19T16:21:50.760808+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:21:50.760808+0000","last_undegraded":"2024-02-19T16:21:50.760808+0000","last_fullsized":"2024-02-19T16:21:50.760808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883562+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:21:21.883562+0000","last_peered":"2024-02-19T16:21:21.883562+0000","last_clean":"2024-02-19T16:21:21.883562+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:21:21.883562+0000","last_undegraded":"2024-02-19T16:21:21.883562+0000","last_fullsized":"2024-02-19T16:21:21.883562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875280+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:21:21.875280+0000","last_peered":"2024-02-19T16:21:21.875280+0000","last_clean":"2024-02-19T16:21:21.875280+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:21:21.875280+0000","last_undegraded":"2024-02-19T16:21:21.875280+0000","last_fullsized":"2024-02-19T16:21:21.875280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762361+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:21:50.762361+0000","last_peered":"2024-02-19T16:21:50.762361+0000","last_clean":"2024-02-19T16:21:50.762361+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:21:50.762361+0000","last_undegraded":"2024-02-19T16:21:50.762361+0000","last_fullsized":"2024-02-19T16:21:50.762361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877516+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:21:21.877516+0000","last_peered":"2024-02-19T16:21:21.877516+0000","last_clean":"2024-02-19T16:21:21.877516+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:21:21.877516+0000","last_undegraded":"2024-02-19T16:21:21.877516+0000","last_fullsized":"2024-02-19T16:21:21.877516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879769+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:21:21.879769+0000","last_peered":"2024-02-19T16:21:21.879769+0000","last_clean":"2024-02-19T16:21:21.879769+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:21:21.879769+0000","last_undegraded":"2024-02-19T16:21:21.879769+0000","last_fullsized":"2024-02-19T16:21:21.879769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762502+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:21:50.762502+0000","last_peered":"2024-02-19T16:21:50.762502+0000","last_clean":"2024-02-19T16:21:50.762502+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:21:50.762502+0000","last_undegraded":"2024-02-19T16:21:50.762502+0000","last_fullsized":"2024-02-19T16:21:50.762502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214481+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:21:22.214481+0000","last_peered":"2024-02-19T16:21:22.214481+0000","last_clean":"2024-02-19T16:21:22.214481+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:21:22.214481+0000","last_undegraded":"2024-02-19T16:21:22.214481+0000","last_fullsized":"2024-02-19T16:21:22.214481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:52.396119+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:21:52.396119+0000","last_peered":"2024-02-19T16:21:52.396119+0000","last_clean":"2024-02-19T16:21:52.396119+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:21:52.396119+0000","last_undegraded":"2024-02-19T16:21:52.396119+0000","last_fullsized":"2024-02-19T16:21:52.396119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751486+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:21:50.751486+0000","last_peered":"2024-02-19T16:21:50.751486+0000","last_clean":"2024-02-19T16:21:50.751486+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:21:50.751486+0000","last_undegraded":"2024-02-19T16:21:50.751486+0000","last_fullsized":"2024-02-19T16:21:50.751486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878692+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:21:21.878692+0000","last_peered":"2024-02-19T16:21:21.878692+0000","last_clean":"2024-02-19T16:21:21.878692+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:21:21.878692+0000","last_undegraded":"2024-02-19T16:21:21.878692+0000","last_fullsized":"2024-02-19T16:21:21.878692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882239+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:21:21.882239+0000","last_peered":"2024-02-19T16:21:21.882239+0000","last_clean":"2024-02-19T16:21:21.882239+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:21:21.882239+0000","last_undegraded":"2024-02-19T16:21:21.882239+0000","last_fullsized":"2024-02-19T16:21:21.882239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756960+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:21:50.756960+0000","last_peered":"2024-02-19T16:21:50.756960+0000","last_clean":"2024-02-19T16:21:50.756960+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:21:50.756960+0000","last_undegraded":"2024-02-19T16:21:50.756960+0000","last_fullsized":"2024-02-19T16:21:50.756960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875579+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:21:21.875579+0000","last_peered":"2024-02-19T16:21:21.875579+0000","last_clean":"2024-02-19T16:21:21.875579+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:21:21.875579+0000","last_undegraded":"2024-02-19T16:21:21.875579+0000","last_fullsized":"2024-02-19T16:21:21.875579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881088+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:21:21.881088+0000","last_peered":"2024-02-19T16:21:21.881088+0000","last_clean":"2024-02-19T16:21:21.881088+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:21:21.881088+0000","last_undegraded":"2024-02-19T16:21:21.881088+0000","last_fullsized":"2024-02-19T16:21:21.881088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.744260+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:21:50.744260+0000","last_peered":"2024-02-19T16:21:50.744260+0000","last_clean":"2024-02-19T16:21:50.744260+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:21:50.744260+0000","last_undegraded":"2024-02-19T16:21:50.744260+0000","last_fullsized":"2024-02-19T16:21:50.744260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.885348+0000","last_change":"2024-02-19T16:21:21.885002+0000","last_active":"2024-02-19T16:21:21.885348+0000","last_peered":"2024-02-19T16:21:21.885348+0000","last_clean":"2024-02-19T16:21:21.885348+0000","last_became_active":"2024-02-19T16:21:21.884557+0000","last_became_peered":"2024-02-19T16:21:21.884557+0000","last_unstale":"2024-02-19T16:21:21.885348+0000","last_undegraded":"2024-02-19T16:21:21.885348+0000","last_fullsized":"2024-02-19T16:21:21.885348+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:08:31.859577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'34","reported_seq":216,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:53.398864+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:21:53.398864+0000","last_peered":"2024-02-19T16:21:53.398864+0000","last_clean":"2024-02-19T16:21:53.398864+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:21:53.398864+0000","last_undegraded":"2024-02-19T16:21:53.398864+0000","last_fullsized":"2024-02-19T16:21:53.398864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":154,"num_read_kb":138,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.743519+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:21:50.743519+0000","last_peered":"2024-02-19T16:21:50.743519+0000","last_clean":"2024-02-19T16:21:50.743519+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:21:50.743519+0000","last_undegraded":"2024-02-19T16:21:50.743519+0000","last_fullsized":"2024-02-19T16:21:50.743519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881519+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:21:21.881519+0000","last_peered":"2024-02-19T16:21:21.881519+0000","last_clean":"2024-02-19T16:21:21.881519+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:21:21.881519+0000","last_undegraded":"2024-02-19T16:21:21.881519+0000","last_fullsized":"2024-02-19T16:21:21.881519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877900+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:21:21.877900+0000","last_peered":"2024-02-19T16:21:21.877900+0000","last_clean":"2024-02-19T16:21:21.877900+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:21:21.877900+0000","last_undegraded":"2024-02-19T16:21:21.877900+0000","last_fullsized":"2024-02-19T16:21:21.877900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.763093+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:21:50.763093+0000","last_peered":"2024-02-19T16:21:50.763093+0000","last_clean":"2024-02-19T16:21:50.763093+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:21:50.763093+0000","last_undegraded":"2024-02-19T16:21:50.763093+0000","last_fullsized":"2024-02-19T16:21:50.763093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882811+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:21:21.882811+0000","last_peered":"2024-02-19T16:21:21.882811+0000","last_clean":"2024-02-19T16:21:21.882811+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:21:21.882811+0000","last_undegraded":"2024-02-19T16:21:21.882811+0000","last_fullsized":"2024-02-19T16:21:21.882811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881375+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:21:21.881375+0000","last_peered":"2024-02-19T16:21:21.881375+0000","last_clean":"2024-02-19T16:21:21.881375+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:21:21.881375+0000","last_undegraded":"2024-02-19T16:21:21.881375+0000","last_fullsized":"2024-02-19T16:21:21.881375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755871+0000","last_change":"2024-02-19T16:20:21.660468+0000","last_active":"2024-02-19T16:21:50.755871+0000","last_peered":"2024-02-19T16:21:50.755871+0000","last_clean":"2024-02-19T16:21:50.755871+0000","last_became_active":"2024-02-19T16:20:21.659988+0000","last_became_peered":"2024-02-19T16:20:21.659988+0000","last_unstale":"2024-02-19T16:21:50.755871+0000","last_undegraded":"2024-02-19T16:21:50.755871+0000","last_fullsized":"2024-02-19T16:21:50.755871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:57:08.316143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.216357+0000","last_change":"2024-02-19T16:21:22.216357+0000","last_active":"2024-02-19T16:21:22.216357+0000","last_peered":"2024-02-19T16:21:22.216357+0000","last_clean":"2024-02-19T16:21:22.216357+0000","last_became_active":"2024-02-19T16:21:22.215883+0000","last_became_peered":"2024-02-19T16:21:22.215883+0000","last_unstale":"2024-02-19T16:21:22.216357+0000","last_undegraded":"2024-02-19T16:21:22.216357+0000","last_fullsized":"2024-02-19T16:21:22.216357+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:32:43.235515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:54.413416+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:21:54.413416+0000","last_peered":"2024-02-19T16:21:54.413416+0000","last_clean":"2024-02-19T16:21:54.413416+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:21:54.413416+0000","last_undegraded":"2024-02-19T16:21:54.413416+0000","last_fullsized":"2024-02-19T16:21:54.413416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750819+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:21:50.750819+0000","last_peered":"2024-02-19T16:21:50.750819+0000","last_clean":"2024-02-19T16:21:50.750819+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:21:50.750819+0000","last_undegraded":"2024-02-19T16:21:50.750819+0000","last_fullsized":"2024-02-19T16:21:50.750819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875693+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:21:21.875693+0000","last_peered":"2024-02-19T16:21:21.875693+0000","last_clean":"2024-02-19T16:21:21.875693+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:21:21.875693+0000","last_undegraded":"2024-02-19T16:21:21.875693+0000","last_fullsized":"2024-02-19T16:21:21.875693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879364+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:21:21.879364+0000","last_peered":"2024-02-19T16:21:21.879364+0000","last_clean":"2024-02-19T16:21:21.879364+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:21:21.879364+0000","last_undegraded":"2024-02-19T16:21:21.879364+0000","last_fullsized":"2024-02-19T16:21:21.879364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756151+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:21:50.756151+0000","last_peered":"2024-02-19T16:21:50.756151+0000","last_clean":"2024-02-19T16:21:50.756151+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:21:50.756151+0000","last_undegraded":"2024-02-19T16:21:50.756151+0000","last_fullsized":"2024-02-19T16:21:50.756151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879449+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:21:21.879449+0000","last_peered":"2024-02-19T16:21:21.879449+0000","last_clean":"2024-02-19T16:21:21.879449+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:21:21.879449+0000","last_undegraded":"2024-02-19T16:21:21.879449+0000","last_fullsized":"2024-02-19T16:21:21.879449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881148+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:21:21.881148+0000","last_peered":"2024-02-19T16:21:21.881148+0000","last_clean":"2024-02-19T16:21:21.881148+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:21:21.881148+0000","last_undegraded":"2024-02-19T16:21:21.881148+0000","last_fullsized":"2024-02-19T16:21:21.881148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.742102+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:21:50.742102+0000","last_peered":"2024-02-19T16:21:50.742102+0000","last_clean":"2024-02-19T16:21:50.742102+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:21:50.742102+0000","last_undegraded":"2024-02-19T16:21:50.742102+0000","last_fullsized":"2024-02-19T16:21:50.742102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879749+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:21:21.879749+0000","last_peered":"2024-02-19T16:21:21.879749+0000","last_clean":"2024-02-19T16:21:21.879749+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:21:21.879749+0000","last_undegraded":"2024-02-19T16:21:21.879749+0000","last_fullsized":"2024-02-19T16:21:21.879749+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879182+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:21:21.879182+0000","last_peered":"2024-02-19T16:21:21.879182+0000","last_clean":"2024-02-19T16:21:21.879182+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:21:21.879182+0000","last_undegraded":"2024-02-19T16:21:21.879182+0000","last_fullsized":"2024-02-19T16:21:21.879182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.748308+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:21:50.748308+0000","last_peered":"2024-02-19T16:21:50.748308+0000","last_clean":"2024-02-19T16:21:50.748308+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:21:50.748308+0000","last_undegraded":"2024-02-19T16:21:50.748308+0000","last_fullsized":"2024-02-19T16:21:50.748308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875519+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:21:21.875519+0000","last_peered":"2024-02-19T16:21:21.875519+0000","last_clean":"2024-02-19T16:21:21.875519+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:21:21.875519+0000","last_undegraded":"2024-02-19T16:21:21.875519+0000","last_fullsized":"2024-02-19T16:21:21.875519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881611+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:21:21.881611+0000","last_peered":"2024-02-19T16:21:21.881611+0000","last_clean":"2024-02-19T16:21:21.881611+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:21:21.881611+0000","last_undegraded":"2024-02-19T16:21:21.881611+0000","last_fullsized":"2024-02-19T16:21:21.881611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.738580+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:21:50.738580+0000","last_peered":"2024-02-19T16:21:50.738580+0000","last_clean":"2024-02-19T16:21:50.738580+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:21:50.738580+0000","last_undegraded":"2024-02-19T16:21:50.738580+0000","last_fullsized":"2024-02-19T16:21:50.738580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875704+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:21:21.875704+0000","last_peered":"2024-02-19T16:21:21.875704+0000","last_clean":"2024-02-19T16:21:21.875704+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:21:21.875704+0000","last_undegraded":"2024-02-19T16:21:21.875704+0000","last_fullsized":"2024-02-19T16:21:21.875704+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755868+0000","last_change":"2024-02-19T16:20:21.671091+0000","last_active":"2024-02-19T16:21:50.755868+0000","last_peered":"2024-02-19T16:21:50.755868+0000","last_clean":"2024-02-19T16:21:50.755868+0000","last_became_active":"2024-02-19T16:20:21.670762+0000","last_became_peered":"2024-02-19T16:20:21.670762+0000","last_unstale":"2024-02-19T16:21:50.755868+0000","last_undegraded":"2024-02-19T16:21:50.755868+0000","last_fullsized":"2024-02-19T16:21:50.755868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:14:35.707812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:55.414134+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:21:55.414134+0000","last_peered":"2024-02-19T16:21:55.414134+0000","last_clean":"2024-02-19T16:21:55.414134+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:21:55.414134+0000","last_undegraded":"2024-02-19T16:21:55.414134+0000","last_fullsized":"2024-02-19T16:21:55.414134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.882228+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:21:21.882228+0000","last_peered":"2024-02-19T16:21:21.882228+0000","last_clean":"2024-02-19T16:21:21.882228+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:21:21.882228+0000","last_undegraded":"2024-02-19T16:21:21.882228+0000","last_fullsized":"2024-02-19T16:21:21.882228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.760249+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:21:50.760249+0000","last_peered":"2024-02-19T16:21:50.760249+0000","last_clean":"2024-02-19T16:21:50.760249+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:21:50.760249+0000","last_undegraded":"2024-02-19T16:21:50.760249+0000","last_fullsized":"2024-02-19T16:21:50.760249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875557+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:21:21.875557+0000","last_peered":"2024-02-19T16:21:21.875557+0000","last_clean":"2024-02-19T16:21:21.875557+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:21:21.875557+0000","last_undegraded":"2024-02-19T16:21:21.875557+0000","last_fullsized":"2024-02-19T16:21:21.875557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881457+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:21:21.881457+0000","last_peered":"2024-02-19T16:21:21.881457+0000","last_clean":"2024-02-19T16:21:21.881457+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:21:21.881457+0000","last_undegraded":"2024-02-19T16:21:21.881457+0000","last_fullsized":"2024-02-19T16:21:21.881457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881761+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:21:21.881761+0000","last_peered":"2024-02-19T16:21:21.881761+0000","last_clean":"2024-02-19T16:21:21.881761+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:21:21.881761+0000","last_undegraded":"2024-02-19T16:21:21.881761+0000","last_fullsized":"2024-02-19T16:21:21.881761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.745051+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:21:50.745051+0000","last_peered":"2024-02-19T16:21:50.745051+0000","last_clean":"2024-02-19T16:21:50.745051+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:21:50.745051+0000","last_undegraded":"2024-02-19T16:21:50.745051+0000","last_fullsized":"2024-02-19T16:21:50.745051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881524+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:21:21.881524+0000","last_peered":"2024-02-19T16:21:21.881524+0000","last_clean":"2024-02-19T16:21:21.881524+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:21:21.881524+0000","last_undegraded":"2024-02-19T16:21:21.881524+0000","last_fullsized":"2024-02-19T16:21:21.881524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.883638+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:21:21.883638+0000","last_peered":"2024-02-19T16:21:21.883638+0000","last_clean":"2024-02-19T16:21:21.883638+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:21:21.883638+0000","last_undegraded":"2024-02-19T16:21:21.883638+0000","last_fullsized":"2024-02-19T16:21:21.883638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.880727+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:21:21.880727+0000","last_peered":"2024-02-19T16:21:21.880727+0000","last_clean":"2024-02-19T16:21:21.880727+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:21:21.880727+0000","last_undegraded":"2024-02-19T16:21:21.880727+0000","last_fullsized":"2024-02-19T16:21:21.880727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.750393+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:21:50.750393+0000","last_peered":"2024-02-19T16:21:50.750393+0000","last_clean":"2024-02-19T16:21:50.750393+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:21:50.750393+0000","last_undegraded":"2024-02-19T16:21:50.750393+0000","last_fullsized":"2024-02-19T16:21:50.750393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214300+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:21:22.214300+0000","last_peered":"2024-02-19T16:21:22.214300+0000","last_clean":"2024-02-19T16:21:22.214300+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:21:22.214300+0000","last_undegraded":"2024-02-19T16:21:22.214300+0000","last_fullsized":"2024-02-19T16:21:22.214300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.879525+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:21:21.879525+0000","last_peered":"2024-02-19T16:21:21.879525+0000","last_clean":"2024-02-19T16:21:21.879525+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:21:21.879525+0000","last_undegraded":"2024-02-19T16:21:21.879525+0000","last_fullsized":"2024-02-19T16:21:21.879525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.761716+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:21:50.761716+0000","last_peered":"2024-02-19T16:21:50.761716+0000","last_clean":"2024-02-19T16:21:50.761716+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:21:50.761716+0000","last_undegraded":"2024-02-19T16:21:50.761716+0000","last_fullsized":"2024-02-19T16:21:50.761716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877528+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:21:21.877528+0000","last_peered":"2024-02-19T16:21:21.877528+0000","last_clean":"2024-02-19T16:21:21.877528+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:21:21.877528+0000","last_undegraded":"2024-02-19T16:21:21.877528+0000","last_fullsized":"2024-02-19T16:21:21.877528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875857+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:21:21.875857+0000","last_peered":"2024-02-19T16:21:21.875857+0000","last_clean":"2024-02-19T16:21:21.875857+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:21:21.875857+0000","last_undegraded":"2024-02-19T16:21:21.875857+0000","last_fullsized":"2024-02-19T16:21:21.875857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762971+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:21:50.762971+0000","last_peered":"2024-02-19T16:21:50.762971+0000","last_clean":"2024-02-19T16:21:50.762971+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:21:50.762971+0000","last_undegraded":"2024-02-19T16:21:50.762971+0000","last_fullsized":"2024-02-19T16:21:50.762971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875847+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:21:21.875847+0000","last_peered":"2024-02-19T16:21:21.875847+0000","last_clean":"2024-02-19T16:21:21.875847+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:21:21.875847+0000","last_undegraded":"2024-02-19T16:21:21.875847+0000","last_fullsized":"2024-02-19T16:21:21.875847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.341280+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:21:22.341280+0000","last_peered":"2024-02-19T16:21:22.341280+0000","last_clean":"2024-02-19T16:21:22.341280+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:21:22.341280+0000","last_undegraded":"2024-02-19T16:21:22.341280+0000","last_fullsized":"2024-02-19T16:21:22.341280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.711824+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:21:50.711824+0000","last_peered":"2024-02-19T16:21:50.711824+0000","last_clean":"2024-02-19T16:21:50.711824+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:21:50.711824+0000","last_undegraded":"2024-02-19T16:21:50.711824+0000","last_fullsized":"2024-02-19T16:21:50.711824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.216182+0000","last_change":"2024-02-19T16:21:22.216182+0000","last_active":"2024-02-19T16:21:22.216182+0000","last_peered":"2024-02-19T16:21:22.216182+0000","last_clean":"2024-02-19T16:21:22.216182+0000","last_became_active":"2024-02-19T16:21:22.215705+0000","last_became_peered":"2024-02-19T16:21:22.215705+0000","last_unstale":"2024-02-19T16:21:22.216182+0000","last_undegraded":"2024-02-19T16:21:22.216182+0000","last_fullsized":"2024-02-19T16:21:22.216182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:29:29.574422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.214589+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:21:22.214589+0000","last_peered":"2024-02-19T16:21:22.214589+0000","last_clean":"2024-02-19T16:21:22.214589+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:21:22.214589+0000","last_undegraded":"2024-02-19T16:21:22.214589+0000","last_fullsized":"2024-02-19T16:21:22.214589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751081+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:21:50.751081+0000","last_peered":"2024-02-19T16:21:50.751081+0000","last_clean":"2024-02-19T16:21:50.751081+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:21:50.751081+0000","last_undegraded":"2024-02-19T16:21:50.751081+0000","last_fullsized":"2024-02-19T16:21:50.751081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875710+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:21:21.875710+0000","last_peered":"2024-02-19T16:21:21.875710+0000","last_clean":"2024-02-19T16:21:21.875710+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:21:21.875710+0000","last_undegraded":"2024-02-19T16:21:21.875710+0000","last_fullsized":"2024-02-19T16:21:21.875710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877637+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:21:21.877637+0000","last_peered":"2024-02-19T16:21:21.877637+0000","last_clean":"2024-02-19T16:21:21.877637+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:21:21.877637+0000","last_undegraded":"2024-02-19T16:21:21.877637+0000","last_fullsized":"2024-02-19T16:21:21.877637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762059+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:21:50.762059+0000","last_peered":"2024-02-19T16:21:50.762059+0000","last_clean":"2024-02-19T16:21:50.762059+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:21:50.762059+0000","last_undegraded":"2024-02-19T16:21:50.762059+0000","last_fullsized":"2024-02-19T16:21:50.762059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875771+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:21:21.875771+0000","last_peered":"2024-02-19T16:21:21.875771+0000","last_clean":"2024-02-19T16:21:21.875771+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:21:21.875771+0000","last_undegraded":"2024-02-19T16:21:21.875771+0000","last_fullsized":"2024-02-19T16:21:21.875771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.894667+0000","last_change":"2024-02-19T16:21:21.894342+0000","last_active":"2024-02-19T16:21:21.894667+0000","last_peered":"2024-02-19T16:21:21.894667+0000","last_clean":"2024-02-19T16:21:21.894667+0000","last_became_active":"2024-02-19T16:21:21.885142+0000","last_became_peered":"2024-02-19T16:21:21.885142+0000","last_unstale":"2024-02-19T16:21:21.894667+0000","last_undegraded":"2024-02-19T16:21:21.894667+0000","last_fullsized":"2024-02-19T16:21:21.894667+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:19:30.846573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755779+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:21:50.755779+0000","last_peered":"2024-02-19T16:21:50.755779+0000","last_clean":"2024-02-19T16:21:50.755779+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:21:50.755779+0000","last_undegraded":"2024-02-19T16:21:50.755779+0000","last_fullsized":"2024-02-19T16:21:50.755779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:22.341453+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:21:22.341453+0000","last_peered":"2024-02-19T16:21:22.341453+0000","last_clean":"2024-02-19T16:21:22.341453+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:21:22.341453+0000","last_undegraded":"2024-02-19T16:21:22.341453+0000","last_fullsized":"2024-02-19T16:21:22.341453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878166+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:21:21.878166+0000","last_peered":"2024-02-19T16:21:21.878166+0000","last_clean":"2024-02-19T16:21:21.878166+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:21:21.878166+0000","last_undegraded":"2024-02-19T16:21:21.878166+0000","last_fullsized":"2024-02-19T16:21:21.878166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756738+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:21:50.756738+0000","last_peered":"2024-02-19T16:21:50.756738+0000","last_clean":"2024-02-19T16:21:50.756738+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:21:50.756738+0000","last_undegraded":"2024-02-19T16:21:50.756738+0000","last_fullsized":"2024-02-19T16:21:50.756738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.877327+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:21:21.877327+0000","last_peered":"2024-02-19T16:21:21.877327+0000","last_clean":"2024-02-19T16:21:21.877327+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:21:21.877327+0000","last_undegraded":"2024-02-19T16:21:21.877327+0000","last_fullsized":"2024-02-19T16:21:21.877327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.881292+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:21:21.881292+0000","last_peered":"2024-02-19T16:21:21.881292+0000","last_clean":"2024-02-19T16:21:21.881292+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:21:21.881292+0000","last_undegraded":"2024-02-19T16:21:21.881292+0000","last_fullsized":"2024-02-19T16:21:21.881292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751124+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:21:50.751124+0000","last_peered":"2024-02-19T16:21:50.751124+0000","last_clean":"2024-02-19T16:21:50.751124+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:21:50.751124+0000","last_undegraded":"2024-02-19T16:21:50.751124+0000","last_fullsized":"2024-02-19T16:21:50.751124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.878995+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:21:21.878995+0000","last_peered":"2024-02-19T16:21:21.878995+0000","last_clean":"2024-02-19T16:21:21.878995+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:21:21.878995+0000","last_undegraded":"2024-02-19T16:21:21.878995+0000","last_fullsized":"2024-02-19T16:21:21.878995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:21.875705+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:21:21.875705+0000","last_peered":"2024-02-19T16:21:21.875705+0000","last_clean":"2024-02-19T16:21:21.875705+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:21:21.875705+0000","last_undegraded":"2024-02-19T16:21:21.875705+0000","last_fullsized":"2024-02-19T16:21:21.875705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751658+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:21:50.751658+0000","last_peered":"2024-02-19T16:21:50.751658+0000","last_clean":"2024-02-19T16:21:50.751658+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:21:50.751658+0000","last_undegraded":"2024-02-19T16:21:50.751658+0000","last_fullsized":"2024-02-19T16:21:50.751658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751409+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:21:50.751409+0000","last_peered":"2024-02-19T16:21:50.751409+0000","last_clean":"2024-02-19T16:21:50.751409+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:21:50.751409+0000","last_undegraded":"2024-02-19T16:21:50.751409+0000","last_fullsized":"2024-02-19T16:21:50.751409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.754896+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:21:50.754896+0000","last_peered":"2024-02-19T16:21:50.754896+0000","last_clean":"2024-02-19T16:21:50.754896+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:21:50.754896+0000","last_undegraded":"2024-02-19T16:21:50.754896+0000","last_fullsized":"2024-02-19T16:21:50.754896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.740252+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:21:50.740252+0000","last_peered":"2024-02-19T16:21:50.740252+0000","last_clean":"2024-02-19T16:21:50.740252+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:21:50.740252+0000","last_undegraded":"2024-02-19T16:21:50.740252+0000","last_fullsized":"2024-02-19T16:21:50.740252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.756230+0000","last_change":"2024-02-19T16:21:21.884814+0000","last_active":"2024-02-19T16:21:50.756230+0000","last_peered":"2024-02-19T16:21:50.756230+0000","last_clean":"2024-02-19T16:21:50.756230+0000","last_became_active":"2024-02-19T16:21:21.884410+0000","last_became_peered":"2024-02-19T16:21:21.884410+0000","last_unstale":"2024-02-19T16:21:50.756230+0000","last_undegraded":"2024-02-19T16:21:50.756230+0000","last_fullsized":"2024-02-19T16:21:50.756230+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:39:44.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.755441+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:21:50.755441+0000","last_peered":"2024-02-19T16:21:50.755441+0000","last_clean":"2024-02-19T16:21:50.755441+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:21:50.755441+0000","last_undegraded":"2024-02-19T16:21:50.755441+0000","last_fullsized":"2024-02-19T16:21:50.755441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.740675+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:21:50.740675+0000","last_peered":"2024-02-19T16:21:50.740675+0000","last_clean":"2024-02-19T16:21:50.740675+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:21:50.740675+0000","last_undegraded":"2024-02-19T16:21:50.740675+0000","last_fullsized":"2024-02-19T16:21:50.740675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.754139+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:21:50.754139+0000","last_peered":"2024-02-19T16:21:50.754139+0000","last_clean":"2024-02-19T16:21:50.754139+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:21:50.754139+0000","last_undegraded":"2024-02-19T16:21:50.754139+0000","last_fullsized":"2024-02-19T16:21:50.754139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.758202+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:21:50.758202+0000","last_peered":"2024-02-19T16:21:50.758202+0000","last_clean":"2024-02-19T16:21:50.758202+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:21:50.758202+0000","last_undegraded":"2024-02-19T16:21:50.758202+0000","last_fullsized":"2024-02-19T16:21:50.758202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.762497+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:21:50.762497+0000","last_peered":"2024-02-19T16:21:50.762497+0000","last_clean":"2024-02-19T16:21:50.762497+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:21:50.762497+0000","last_undegraded":"2024-02-19T16:21:50.762497+0000","last_fullsized":"2024-02-19T16:21:50.762497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.752750+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:21:50.752750+0000","last_peered":"2024-02-19T16:21:50.752750+0000","last_clean":"2024-02-19T16:21:50.752750+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:21:50.752750+0000","last_undegraded":"2024-02-19T16:21:50.752750+0000","last_fullsized":"2024-02-19T16:21:50.752750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.751742+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:21:50.751742+0000","last_peered":"2024-02-19T16:21:50.751742+0000","last_clean":"2024-02-19T16:21:50.751742+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:21:50.751742+0000","last_undegraded":"2024-02-19T16:21:50.751742+0000","last_fullsized":"2024-02-19T16:21:50.751742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.758043+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:21:50.758043+0000","last_peered":"2024-02-19T16:21:50.758043+0000","last_clean":"2024-02-19T16:21:50.758043+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:21:50.758043+0000","last_undegraded":"2024-02-19T16:21:50.758043+0000","last_fullsized":"2024-02-19T16:21:50.758043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-19T16:21:50.737802+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:21:50.737802+0000","last_peered":"2024-02-19T16:21:50.737802+0000","last_clean":"2024-02-19T16:21:50.737802+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:21:50.737802+0000","last_undegraded":"2024-02-19T16:21:50.737802+0000","last_fullsized":"2024-02-19T16:21:50.737802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66166,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":470,"num_read_kb":305,"num_write":168,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":198498,"data_compressed":5154,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":513792,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60099999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.56599999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.53500000000000003,"15min":0.55400000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52800000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.53200000000000003,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.316},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56499999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.55900000000000005,"15min":0.57599999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.54200000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.57899999999999996,"15min":0.60499999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.54300000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.34200000000000003}]},{"osd":5,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.50600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.66900000000000004,"5min":0.998,"15min":0.998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.63100000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.66300000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68799999999999994,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.44400000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47999999999999998}]},{"osd":9,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.497,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.67700000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.88700000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Feb 19 16:21:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.72199999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.73299999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.51400000000000001}]}]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1155072,"data_stored":1163034,"data_compressed":61171,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.502}]},{"osd":1,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71499999999999997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34699999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.55000000000000004,"15min":0.56499999999999995},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.73599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80700000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.313}]},{"osd":5,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.82199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.62,"15min":0.63600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89800000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38200000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66700000000000004,"15min":0.68000000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97399999999999998,"5min":0.999,"15min":0.999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":1.028,"15min":1.028},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.72799999999999998,"15min":0.75},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91600000000000004,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.90000000000000002,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.40300000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:21:37 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.56399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56999999999999995}]}]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":517542,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71599999999999997}]},{"osd":1,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.626,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.55600000000000005,"15min":0.58099999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.35899999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.627},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66000000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72699999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.66400000000000003,"15min":0.69399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.749,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45300000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82099999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.73299999999999998,"15min":0.78000000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76000000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.38900000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67700000000000005,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":1.169,"15min":1.169},"last":0.57099999999999995}]},{"osd":10,"last update":"Mon Feb 19 16:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.77600000000000002,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.86099999999999999,"5min":1.278,"15min":1.278},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.75600000000000001,"15min":0.77400000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81000000000000005,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55000000000000004}]},{"osd":11,"last update":"Mon Feb 19 16:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58599999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":513366,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63800000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.495}]},{"osd":1,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.55700000000000005,"15min":0.57899999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.875,"15min":0.875},"last":0.373},{"interface":"front","average":{"1min":0.498,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.64800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.505}]},{"osd":2,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41799999999999998,"15min":0.42899999999999999},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.309},{"interface":"front","average":{"1min":0.376,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57999999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.45700000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.55600000000000005,"15min":0.57399999999999995},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60699999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.61799999999999999},"min":{"1min":0.39300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.40500000000000003}]},{"osd":5,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.53300000000000003,"15min":0.55800000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.56599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.56399999999999995,"15min":0.58899999999999997},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.436}]},{"osd":6,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.62,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.68100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:21:18 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.876,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93400000000000005,"5min":1.393,"15min":1.393},"last":0.59599999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.52200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.62},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.73599999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66900000000000004}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55000000000000004}]},{"osd":11,"last update":"Mon Feb 19 16:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52000000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1108834,"data_compressed":59453,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.315},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.27800000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39700000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75800000000000001,"5min":0.87,"15min":0.87},"last":0.42999999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71399999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.379},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.45600000000000002},"min":{"1min":0.39500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.29899999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91000000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.50900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49099999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67400000000000004,"5min":0.997,"15min":0.997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70099999999999996,"15min":0.71499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.70599999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55200000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60499999999999998}]},{"osd":11,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.62},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.66800000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":508622,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.61799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58699999999999997}]},{"osd":7,"last update":"Mon Feb 19 16:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.73899999999999999}]},{"osd":8,"last update":"Mon Feb 19 16:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92900000000000005,"5min":1.014,"15min":1.014},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.67600000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72999999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96999999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":570556,"data_compressed":25413,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.51300000000000001},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40500000000000003},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.66500000000000004,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.309,"15min":0.309},"max":{"1min":14.067,"5min":14.067,"15min":14.067},"last":0.45000000000000001}]},{"osd":2,"last update":"Mon Feb 19 16:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54000000000000004,"15min":0.496},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72999999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.51800000000000002,"15min":0.49299999999999999},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.61099999999999999}]},{"osd":4,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.41399999999999998,"15min":0.376},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59999999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44400000000000001,"15min":0.40500000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.46500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:21:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.625},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57299999999999995}]},{"osd":7,"last update":"Mon Feb 19 16:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52700000000000002},"min":{"1min":0.373,"5min":0.316,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58499999999999996}]},{"osd":8,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66300000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66100000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52500000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.82099999999999995,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":14.105,"5min":14.105,"15min":14.105},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71099999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67200000000000004}]},{"osd":10,"last update":"Mon Feb 19 16:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.84699999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64500000000000002}]},{"osd":11,"last update":"Mon Feb 19 16:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510010,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79100000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63200000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46500000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72599999999999998,"5min":0.874,"15min":0.874},"last":0.48699999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.442},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.38,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.39500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60099999999999998,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85099999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Feb 19 16:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68799999999999994}]},{"osd":8,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89500000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55800000000000005}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.45900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73199999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95499999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.628}]},{"osd":11,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.78500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":571906,"data_compressed":25413,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Mon Feb 19 16:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.48599999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.5},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.53900000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Feb 19 16:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.433},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.40400000000000003}]},{"osd":5,"last update":"Mon Feb 19 16:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.48999999999999999,"15min":0.47799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60299999999999998,"5min":1.026,"15min":1.026},"last":0.48499999999999999}]},{"osd":6,"last update":"Mon Feb 19 16:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.629,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.503},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71199999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Feb 19 16:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.47199999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.623,"15min":0.64400000000000002},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97299999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52000000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69399999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.755,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.64700000000000002}]},{"osd":10,"last update":"Mon Feb 19 16:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.629},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.56899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.59799999999999998}]},{"osd":11,"last update":"Mon Feb 19 16:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69599999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":508816,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.35999999999999999},"min":{"1min":0.29099999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.40400000000000003,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69299999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.76000000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75700000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64200000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82399999999999995,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77800000000000002,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.747},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61899999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66800000000000004}]},{"osd":6,"last update":"Mon Feb 19 16:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90800000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.87,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60399999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.018},"last":0.76300000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.05,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Mon Feb 19 16:21:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.77300000000000002,"15min":0.78000000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":1.03,"15min":1.03},"last":0.72099999999999997}]},{"osd":11,"last update":"Mon Feb 19 16:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65900000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888132,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":513804,"data_compressed":23695,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.26300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.34899999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.373,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.51500000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.441}]},{"osd":3,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":1.03,"15min":1.03},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.60499999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Feb 19 16:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72399999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.374},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.40799999999999997}]},{"osd":6,"last update":"Mon Feb 19 16:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.504},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52200000000000002}]},{"osd":7,"last update":"Mon Feb 19 16:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79700000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.624},"min":{"1min":0.47999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85199999999999998,"5min":1.073,"15min":1.073},"last":0.53700000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":0.64100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.70299999999999996,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.65300000000000002,"5min":1.044,"15min":1.044},"last":0.58199999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64400000000000002}]},{"osd":11,"last update":"Mon Feb 19 16:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.68100000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1104472,"data_compressed":59453,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.32300000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.79300000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.51000000000000001,"15min":0.51700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73299999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53400000000000003,"15min":0.52100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64600000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.55000000000000004,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.44800000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.498,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78000000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.63}]},{"osd":4,"last update":"Mon Feb 19 16:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73399999999999999,"5min":0.871,"15min":0.871},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.39100000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.65600000000000003,"15min":0.59399999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.502}]},{"osd":7,"last update":"Mon Feb 19 16:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78400000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.54300000000000004}]},{"osd":8,"last update":"Mon Feb 19 16:21:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63100000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.44400000000000001}]},{"osd":9,"last update":"Mon Feb 19 16:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66300000000000003,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.74199999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67100000000000004}]},{"osd":10,"last update":"Mon Feb 19 16:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.873,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.71399999999999997,"15min":0.73199999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83299999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":62024,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63374,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:21:58.061 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-19T16:21:58.062 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-19T16:21:58.062 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-19T16:21:58.062 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph health --format=json 2024-02-19T16:21:58.315 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:21:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:58 smithi191 ceph-mon[37203]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:21:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:21:58 smithi191 ceph-mon[37203]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:21:58.564 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:58 smithi008 ceph-mon[32573]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:21:58.564 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:21:58 smithi008 ceph-mon[32573]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:21:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:58 smithi156 ceph-mon[36502]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:21:58.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:21:58 smithi156 ceph-mon[36502]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:00.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 -- 172.21.15.8:0/3566362226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 msgr2=0x7f06a41024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3566362226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a41024b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f06980099c0 tx=0x7f0698009cd0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 -- 172.21.15.8:0/3566362226 shutdown_connections 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3566362226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a41029f0 0x7f06a410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3566362226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a4108010 0x7f06a41083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3566362226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a41024b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 -- 172.21.15.8:0/3566362226 >> 172.21.15.8:0/3566362226 conn(0x7f06a4079110 msgr2=0x7f06a4079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 -- 172.21.15.8:0/3566362226 shutdown_connections 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.071+0000 7f06ac47d700 1 -- 172.21.15.8:0/3566362226 wait complete. 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 Processor -- start 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 -- start start 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a41029f0 0x7f06a419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a4108010 0x7f06a419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419f540 con 0x7f06a4102030 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419f6b0 con 0x7f06a4108010 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06ac47d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a419f870 con 0x7f06a41029f0 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06aa219700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06aa219700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44780/0 (socket says 172.21.15.8:44780) 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06aa219700 1 -- 172.21.15.8:0/3332226010 learned_addr learned my addr 172.21.15.8:0/3332226010 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06a9a18700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a41029f0 0x7f06a419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.072+0000 7f06aaa1a700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a4108010 0x7f06a419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aa219700 1 -- 172.21.15.8:0/3332226010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a41029f0 msgr2=0x7f06a419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aa219700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a41029f0 0x7f06a419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aa219700 1 -- 172.21.15.8:0/3332226010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a4108010 msgr2=0x7f06a419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aa219700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a4108010 0x7f06a419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aa219700 1 -- 172.21.15.8:0/3332226010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980095f0 con 0x7f06a4102030 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06a9a18700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a41029f0 0x7f06a419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aaa1a700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a4108010 0x7f06a419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06aa219700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a419a660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f069400ef10 tx=0x7f069400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069400c9c0 con 0x7f06a4102030 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a419faf0 con 0x7f06a4102030 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069400f040 con 0x7f06a4102030 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694014480 con 0x7f06a4102030 2024-02-19T16:22:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.073+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a41a73b0 con 0x7f06a4102030 2024-02-19T16:22:00.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.074+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a4068ca0 con 0x7f06a4102030 2024-02-19T16:22:00.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.075+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0694007720 con 0x7f06a4102030 2024-02-19T16:22:00.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.076+0000 7f06a37fe700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f068c0a2150 0x7f068c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:00.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.076+0000 7f06a9a18700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f068c0a2150 0x7f068c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:00.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.076+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f06940ce910 con 0x7f06a4102030 2024-02-19T16:22:00.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.076+0000 7f06a9a18700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f068c0a2150 0x7f068c0a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0698000c00 tx=0x7f0698005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:00.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.079+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f06940975a0 con 0x7f06a4102030 2024-02-19T16:22:00.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.331+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f06a419b9c0 con 0x7f06a4102030 2024-02-19T16:22:00.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.331+0000 7f06a37fe700 1 -- 172.21.15.8:0/3332226010 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f069409ac60 con 0x7f06a4102030 2024-02-19T16:22:00.333 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:00.333 INFO:teuthology.orchestra.run.smithi008.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-19T16:22:00.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f068c0a2150 msgr2=0x7f068c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:00.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f068c0a2150 0x7f068c0a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0698000c00 tx=0x7f0698005fb0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 msgr2=0x7f06a419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:00.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a419a660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f069400ef10 tx=0x7f069400c3a0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 shutdown_connections 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f068c0a2150 0x7f068c0a4610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06a4108010 0x7f06a419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06a4102030 0x7f06a419a660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 --2- 172.21.15.8:0/3332226010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06a41029f0 0x7f06a419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.333+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 >> 172.21.15.8:0/3332226010 conn(0x7f06a4079110 msgr2=0x7f06a40744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.334+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 shutdown_connections 2024-02-19T16:22:00.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:00.334+0000 7f06ac47d700 1 -- 172.21.15.8:0/3332226010 wait complete. 2024-02-19T16:22:00.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:00 smithi008 ceph-mon[32573]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-19T16:22:00.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:00 smithi156 ceph-mon[36502]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-19T16:22:00.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:00 smithi191 ceph-mon[37203]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-19T16:22:01.130 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-19T16:22:01.131 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 -- 172.21.15.8:0/1727964369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 msgr2=0x7fe7dc073900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 --2- 172.21.15.8:0/1727964369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc073900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009900 tx=0x7fe7d402bcd0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 -- 172.21.15.8:0/1727964369 shutdown_connections 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 --2- 172.21.15.8:0/1727964369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1093c0 0x7fe7dc10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 --2- 172.21.15.8:0/1727964369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc073ed0 0x7fe7dc074330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 --2- 172.21.15.8:0/1727964369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc073900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 -- 172.21.15.8:0/1727964369 >> 172.21.15.8:0/1727964369 conn(0x7fe7dc0fcaa0 msgr2=0x7fe7dc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 -- 172.21.15.8:0/1727964369 shutdown_connections 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 -- 172.21.15.8:0/1727964369 wait complete. 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 Processor -- start 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.242+0000 7fe7e370d640 1 -- start start 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc073ed0 0x7fe7dc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc1093c0 0x7fe7dc1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1a12d0 con 0x7fe7dc0754b0 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1a1440 con 0x7fe7dc073ed0 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1a1600 con 0x7fe7dc1093c0 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44794/0 (socket says 172.21.15.8:44794) 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 -- 172.21.15.8:0/3104622385 learned_addr learned my addr 172.21.15.8:0/3104622385 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:01.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e1caa640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc1093c0 0x7fe7dc1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e14a9640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc073ed0 0x7fe7dc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 -- 172.21.15.8:0/3104622385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc1093c0 msgr2=0x7fe7dc1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc1093c0 0x7fe7dc1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 -- 172.21.15.8:0/3104622385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc073ed0 msgr2=0x7fe7dc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc073ed0 0x7fe7dc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 -- 172.21.15.8:0/3104622385 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4009560 con 0x7fe7dc0754b0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e0ca8640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc199d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e9f0 tx=0x7fe7cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc009720 con 0x7fe7dc0754b0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc00cd80 con 0x7fe7dc0754b0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc1a18e0 con 0x7fe7dc0754b0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.243+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc010690 con 0x7fe7dc0754b0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.244+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc1a1df0 con 0x7fe7dc0754b0 2024-02-19T16:22:01.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.244+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc108180 con 0x7fe7dc0754b0 2024-02-19T16:22:01.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.245+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe7cc0046c0 con 0x7fe7dc0754b0 2024-02-19T16:22:01.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.246+0000 7fe7d27fc640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe7c80a2140 0x7fe7c80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.246+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fe7cc014070 con 0x7fe7dc0754b0 2024-02-19T16:22:01.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.246+0000 7fe7e14a9640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe7c80a2140 0x7fe7c80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.247+0000 7fe7e14a9640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe7c80a2140 0x7fe7c80a4600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009230 tx=0x7fe7d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:01.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.250+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe7cc096f70 con 0x7fe7dc0754b0 2024-02-19T16:22:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.435+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7dc10f110 con 0x7fe7dc0754b0 2024-02-19T16:22:01.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.436+0000 7fe7d27fc640 1 -- 172.21.15.8:0/3104622385 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24545 (secure 0 0 0) 0x7fe7cc09a630 con 0x7fe7dc0754b0 2024-02-19T16:22:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:01.441 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":93,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:21:21.872365+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:01.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe7c80a2140 msgr2=0x7fe7c80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe7c80a2140 0x7fe7c80a4600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009230 tx=0x7fe7d4005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 msgr2=0x7fe7dc199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc199d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e9f0 tx=0x7fe7cc00edf0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 shutdown_connections 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe7c80a2140 0x7fe7c80a4600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc1093c0 0x7fe7dc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc0754b0 0x7fe7dc199d20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 --2- 172.21.15.8:0/3104622385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc073ed0 0x7fe7dc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.438+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 >> 172.21.15.8:0/3104622385 conn(0x7fe7dc0fcaa0 msgr2=0x7fe7dc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.439+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 shutdown_connections 2024-02-19T16:22:01.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.439+0000 7fe7e370d640 1 -- 172.21.15.8:0/3104622385 wait complete. 2024-02-19T16:22:01.449 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:01 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3332226010' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-19T16:22:01.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:01.451 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.556+0000 7f06cf2da640 1 -- 172.21.15.8:0/3612770962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81003f0 msgr2=0x7f06c8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.556+0000 7f06cf2da640 1 --2- 172.21.15.8:0/3612770962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81003f0 0x7f06c8100810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009900 tx=0x7f06c002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.556+0000 7f06cf2da640 1 -- 172.21.15.8:0/3612770962 shutdown_connections 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.556+0000 7f06cf2da640 1 --2- 172.21.15.8:0/3612770962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06c8101fa0 0x7f06c8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.556+0000 7f06cf2da640 1 --2- 172.21.15.8:0/3612770962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c81015e0 0x7f06c8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.556+0000 7f06cf2da640 1 --2- 172.21.15.8:0/3612770962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81003f0 0x7f06c8100810 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- 172.21.15.8:0/3612770962 >> 172.21.15.8:0/3612770962 conn(0x7f06c80fb980 msgr2=0x7f06c80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- 172.21.15.8:0/3612770962 shutdown_connections 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- 172.21.15.8:0/3612770962 wait complete. 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 Processor -- start 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- start start 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06c81003f0 0x7f06c8104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 0x7f06c8102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c8101fa0 0x7f06c81030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8103850 con 0x7f06c81015e0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c81039c0 con 0x7f06c8101fa0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cf2da640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c81a1680 con 0x7f06c81003f0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cc875640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 0x7f06c8102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cc875640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 0x7f06c8102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44804/0 (socket says 172.21.15.8:44804) 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cc875640 1 -- 172.21.15.8:0/960379598 learned_addr learned my addr 172.21.15.8:0/960379598 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.557+0000 7f06cd076640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06c81003f0 0x7f06c8104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cd877640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c8101fa0 0x7f06c81030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cc875640 1 -- 172.21.15.8:0/960379598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06c81003f0 msgr2=0x7f06c8104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cc875640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06c81003f0 0x7f06c8104570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cc875640 1 -- 172.21.15.8:0/960379598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c8101fa0 msgr2=0x7f06c81030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cc875640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c8101fa0 0x7f06c81030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cc875640 1 -- 172.21.15.8:0/960379598 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8009490 con 0x7f06c81015e0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cd877640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c8101fa0 0x7f06c81030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cc875640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 0x7f06c8102bb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ea90 tx=0x7f06b800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800ce40 con 0x7f06c81015e0 2024-02-19T16:22:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c4009840 con 0x7f06c81015e0 2024-02-19T16:22:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b80035f0 con 0x7f06c81015e0 2024-02-19T16:22:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8010690 con 0x7f06c81015e0 2024-02-19T16:22:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.558+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c0009560 con 0x7f06c81015e0 2024-02-19T16:22:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.559+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c810bdd0 con 0x7f06c81015e0 2024-02-19T16:22:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.560+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f06b8003ad0 con 0x7f06c81015e0 2024-02-19T16:22:01.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.561+0000 7f06be7fc640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06b40a20c0 0x7f06b40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.561+0000 7f06cd076640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06b40a20c0 0x7f06b40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.561+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f06b8014070 con 0x7f06c81015e0 2024-02-19T16:22:01.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.561+0000 7f06cd076640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06b40a20c0 0x7f06b40a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005ab0 tx=0x7f06c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:01.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.565+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f06b80971a0 con 0x7f06c81015e0 2024-02-19T16:22:01.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:01 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3332226010' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-19T16:22:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.746+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06c81a1ff0 con 0x7f06c81015e0 2024-02-19T16:22:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.746+0000 7f06be7fc640 1 -- 172.21.15.8:0/960379598 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06b8003760 con 0x7f06c81015e0 2024-02-19T16:22:01.747 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:01.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06b40a20c0 msgr2=0x7f06b40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06b40a20c0 0x7f06b40a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005ab0 tx=0x7f06c0005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 msgr2=0x7f06c8102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 0x7f06c8102bb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ea90 tx=0x7f06b800ef50 comp rx=0 tx=0).stop 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 shutdown_connections 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f06b40a20c0 0x7f06b40a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c8101fa0 0x7f06c81030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f06c81015e0 0x7f06c8102bb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 --2- 172.21.15.8:0/960379598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06c81003f0 0x7f06c8104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 >> 172.21.15.8:0/960379598 conn(0x7f06c80fb980 msgr2=0x7f06c8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 shutdown_connections 2024-02-19T16:22:01.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.748+0000 7f06cf2da640 1 -- 172.21.15.8:0/960379598 wait complete. 2024-02-19T16:22:01.761 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:01.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:01 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3332226010' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-19T16:22:01.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.865+0000 7f4928925640 1 -- 172.21.15.8:0/1183438062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241003f0 msgr2=0x7f4924100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.865+0000 7f4928925640 1 --2- 172.21.15.8:0/1183438062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241003f0 0x7f4924100810 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f491c009900 tx=0x7f491c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 -- 172.21.15.8:0/1183438062 shutdown_connections 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 --2- 172.21.15.8:0/1183438062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924101fa0 0x7f4924108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 --2- 172.21.15.8:0/1183438062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49241015e0 0x7f4924101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 --2- 172.21.15.8:0/1183438062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241003f0 0x7f4924100810 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 -- 172.21.15.8:0/1183438062 >> 172.21.15.8:0/1183438062 conn(0x7f49240fb980 msgr2=0x7f49240fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 -- 172.21.15.8:0/1183438062 shutdown_connections 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 -- 172.21.15.8:0/1183438062 wait complete. 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 Processor -- start 2024-02-19T16:22:01.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.866+0000 7f4928925640 1 -- start start 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49241003f0 0x7f492410acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 0x7f49241092f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924101fa0 0x7f4924109830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924109f90 con 0x7f49241015e0 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492410a100 con 0x7f4924101fa0 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49241a1660 con 0x7f49241003f0 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 0x7f49241092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 0x7f49241092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44824/0 (socket says 172.21.15.8:44824) 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 -- 172.21.15.8:0/3487867919 learned_addr learned my addr 172.21.15.8:0/3487867919 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f492259c640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49241003f0 0x7f492410acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4922d9d640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924101fa0 0x7f4924109830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 -- 172.21.15.8:0/3487867919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49241003f0 msgr2=0x7f492410acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49241003f0 0x7f492410acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 -- 172.21.15.8:0/3487867919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924101fa0 msgr2=0x7f4924109830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924101fa0 0x7f4924109830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 -- 172.21.15.8:0/3487867919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c009560 con 0x7f49241015e0 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4921d9b640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 0x7f49241092f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f490c00ea20 tx=0x7f490c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c009720 con 0x7f49241015e0 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f490c00cd80 con 0x7f49241015e0 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49241a1910 con 0x7f49241015e0 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c010490 con 0x7f49241015e0 2024-02-19T16:22:01.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.867+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49241a1dd0 con 0x7f49241015e0 2024-02-19T16:22:01.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.868+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4924107830 con 0x7f49241015e0 2024-02-19T16:22:01.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.870+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f490c005020 con 0x7f49241015e0 2024-02-19T16:22:01.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.870+0000 7f491b7fe640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f49100a20c0 0x7f49100a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:01.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.870+0000 7f492259c640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f49100a20c0 0x7f49100a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:01.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.870+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f490c014070 con 0x7f49241015e0 2024-02-19T16:22:01.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.871+0000 7f492259c640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f49100a20c0 0x7f49100a4580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f491c005ab0 tx=0x7f491c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:01.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:01.875+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f490c097040 con 0x7f49241015e0 2024-02-19T16:22:02.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.056+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f492410d150 con 0x7f49241015e0 2024-02-19T16:22:02.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.057+0000 7f491b7fe640 1 -- 172.21.15.8:0/3487867919 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f490c09a700 con 0x7f49241015e0 2024-02-19T16:22:02.058 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f49100a20c0 msgr2=0x7f49100a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f49100a20c0 0x7f49100a4580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f491c005ab0 tx=0x7f491c005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 msgr2=0x7f49241092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 0x7f49241092f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f490c00ea20 tx=0x7f490c00ee20 comp rx=0 tx=0).stop 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 shutdown_connections 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f49100a20c0 0x7f49100a4580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924101fa0 0x7f4924109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49241015e0 0x7f49241092f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 --2- 172.21.15.8:0/3487867919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49241003f0 0x7f492410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 >> 172.21.15.8:0/3487867919 conn(0x7f49240fb980 msgr2=0x7f4924104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 shutdown_connections 2024-02-19T16:22:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.059+0000 7f4928925640 1 -- 172.21.15.8:0/3487867919 wait complete. 2024-02-19T16:22:02.071 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.175+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/1870378309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741015e0 msgr2=0x7fdb74101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.175+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/1870378309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741015e0 0x7fdb74101a60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009900 tx=0x7fdb6802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/1870378309 shutdown_connections 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/1870378309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb74101fa0 0x7fdb74108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/1870378309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741015e0 0x7fdb74101a60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/1870378309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb741003f0 0x7fdb74100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/1870378309 >> 172.21.15.8:0/1870378309 conn(0x7fdb740fb980 msgr2=0x7fdb740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:02.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/1870378309 shutdown_connections 2024-02-19T16:22:02.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/1870378309 wait complete. 2024-02-19T16:22:02.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 Processor -- start 2024-02-19T16:22:02.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 -- start start 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 0x7fdb74196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb741015e0 0x7fdb741969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb74101fa0 0x7fdb7419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.176+0000 7fdb7a6ed640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419df60 con 0x7fdb741003f0 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb7a6ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419e0d0 con 0x7fdb741015e0 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb7a6ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419e290 con 0x7fdb74101fa0 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 0x7fdb74196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 0x7fdb74196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44848/0 (socket says 172.21.15.8:44848) 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 -- 172.21.15.8:0/2764532528 learned_addr learned my addr 172.21.15.8:0/2764532528 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb737fe640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb741015e0 0x7fdb741969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb78c8a640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb74101fa0 0x7fdb7419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 -- 172.21.15.8:0/2764532528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb74101fa0 msgr2=0x7fdb7419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb74101fa0 0x7fdb7419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 -- 172.21.15.8:0/2764532528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb741015e0 msgr2=0x7fdb741969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb741015e0 0x7fdb741969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 -- 172.21.15.8:0/2764532528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68009560 con 0x7fdb741003f0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb737fe640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb741015e0 0x7fdb741969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb73fff640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 0x7fdb74196470 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400e9c0 tx=0x7fdb6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64009700 con 0x7fdb741003f0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7419e570 con 0x7fdb741003f0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb6400cd80 con 0x7fdb741003f0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64010430 con 0x7fdb741003f0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.177+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7419ea50 con 0x7fdb741003f0 2024-02-19T16:22:02.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.178+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb74107730 con 0x7fdb741003f0 2024-02-19T16:22:02.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.179+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fdb64003680 con 0x7fdb741003f0 2024-02-19T16:22:02.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.180+0000 7fdb717fa640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb5c0a20c0 0x7fdb5c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.180+0000 7fdb737fe640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb5c0a20c0 0x7fdb5c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.180+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fdb64014070 con 0x7fdb741003f0 2024-02-19T16:22:02.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.180+0000 7fdb737fe640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb5c0a20c0 0x7fdb5c0a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009230 tx=0x7fdb68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:02.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.184+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fdb64097020 con 0x7fdb741003f0 2024-02-19T16:22:02.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.368+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb7402d000 con 0x7fdb741003f0 2024-02-19T16:22:02.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.369+0000 7fdb717fa640 1 -- 172.21.15.8:0/2764532528 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb6409a6e0 con 0x7fdb741003f0 2024-02-19T16:22:02.370 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb5c0a20c0 msgr2=0x7fdb5c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb5c0a20c0 0x7fdb5c0a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009230 tx=0x7fdb68005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 msgr2=0x7fdb74196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 0x7fdb74196470 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400e9c0 tx=0x7fdb6400ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 shutdown_connections 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb5c0a20c0 0x7fdb5c0a4580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb74101fa0 0x7fdb7419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb741015e0 0x7fdb741969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 --2- 172.21.15.8:0/2764532528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb741003f0 0x7fdb74196470 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 >> 172.21.15.8:0/2764532528 conn(0x7fdb740fb980 msgr2=0x7fdb74104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 shutdown_connections 2024-02-19T16:22:02.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.371+0000 7fdb7a6ed640 1 -- 172.21.15.8:0/2764532528 wait complete. 2024-02-19T16:22:02.381 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:02 smithi008 ceph-mon[32573]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-19T16:22:02.382 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:02 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3104622385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:02.382 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:02 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/960379598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:02.382 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:02 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3487867919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:02.383 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 -- 172.21.15.8:0/2638957617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb781030c0 msgr2=0x7fdb78109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/2638957617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb781030c0 0x7fdb78109950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009900 tx=0x7fdb6002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 -- 172.21.15.8:0/2638957617 shutdown_connections 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/2638957617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb781030c0 0x7fdb78109950 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/2638957617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb78102700 0x7fdb78102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/2638957617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb78101510 0x7fdb78101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 -- 172.21.15.8:0/2638957617 >> 172.21.15.8:0/2638957617 conn(0x7fdb780fca60 msgr2=0x7fdb780feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 -- 172.21.15.8:0/2638957617 shutdown_connections 2024-02-19T16:22:02.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.488+0000 7fdb7f04e640 1 -- 172.21.15.8:0/2638957617 wait complete. 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 Processor -- start 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 -- start start 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb78101510 0x7fdb78197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb78102700 0x7fdb78197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 0x7fdb7819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7819f030 con 0x7fdb781030c0 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7819f1a0 con 0x7fdb78102700 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7f04e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7819f360 con 0x7fdb78101510 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 0x7fdb7819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb78102700 0x7fdb78197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 0x7fdb7819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44856/0 (socket says 172.21.15.8:44856) 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 -- 172.21.15.8:0/4120355756 learned_addr learned my addr 172.21.15.8:0/4120355756 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7cdea640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb78101510 0x7fdb78197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 -- 172.21.15.8:0/4120355756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb78101510 msgr2=0x7fdb78197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb78101510 0x7fdb78197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 -- 172.21.15.8:0/4120355756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb78102700 msgr2=0x7fdb78197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.489+0000 7fdb7d5eb640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb78102700 0x7fdb78197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb7d5eb640 1 -- 172.21.15.8:0/4120355756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70009490 con 0x7fdb781030c0 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb7d5eb640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 0x7fdb7819eaf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009230 tx=0x7fdb60004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004440 con 0x7fdb781030c0 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb60009560 con 0x7fdb781030c0 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb600052d0 con 0x7fdb781030c0 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60003cc0 con 0x7fdb781030c0 2024-02-19T16:22:02.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.490+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7819f9b0 con 0x7fdb781030c0 2024-02-19T16:22:02.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.491+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb78108830 con 0x7fdb781030c0 2024-02-19T16:22:02.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.492+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fdb60005440 con 0x7fdb781030c0 2024-02-19T16:22:02.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.493+0000 7fdb75ffb640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb640a20c0 0x7fdb640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.493+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fdb600f09c0 con 0x7fdb781030c0 2024-02-19T16:22:02.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.493+0000 7fdb7cdea640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb640a20c0 0x7fdb640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.493+0000 7fdb7cdea640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb640a20c0 0x7fdb640a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdb78102560 tx=0x7fdb70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:02.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.497+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fdb600b95f0 con 0x7fdb781030c0 2024-02-19T16:22:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:02 smithi156 ceph-mon[36502]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-19T16:22:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:02 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3104622385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:02 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/960379598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:02.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:02 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3487867919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:02.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.677+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdb7802cf90 con 0x7fdb781030c0 2024-02-19T16:22:02.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.678+0000 7fdb75ffb640 1 -- 172.21.15.8:0/4120355756 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdb600bccb0 con 0x7fdb781030c0 2024-02-19T16:22:02.679 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb640a20c0 msgr2=0x7fdb640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb640a20c0 0x7fdb640a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdb78102560 tx=0x7fdb70005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 msgr2=0x7fdb7819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 0x7fdb7819eaf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009230 tx=0x7fdb60004040 comp rx=0 tx=0).stop 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 shutdown_connections 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdb640a20c0 0x7fdb640a4580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb781030c0 0x7fdb7819eaf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb78102700 0x7fdb78197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 --2- 172.21.15.8:0/4120355756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb78101510 0x7fdb78197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 >> 172.21.15.8:0/4120355756 conn(0x7fdb780fca60 msgr2=0x7fdb78105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 shutdown_connections 2024-02-19T16:22:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.680+0000 7fdb7f04e640 1 -- 172.21.15.8:0/4120355756 wait complete. 2024-02-19T16:22:02.692 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:02.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:02 smithi191 ceph-mon[37203]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-02-19T16:22:02.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:02 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3104622385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:02.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:02 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/960379598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:02.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:02 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3487867919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:02.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 -- 172.21.15.8:0/3994519720 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 msgr2=0x7fbc74106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/3994519720 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc74106d30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009900 tx=0x7fbc6802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:02.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 -- 172.21.15.8:0/3994519720 shutdown_connections 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/3994519720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc74107270 0x7fbc74109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/3994519720 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc74106d30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/3994519720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 0x7fbc74104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 -- 172.21.15.8:0/3994519720 >> 172.21.15.8:0/3994519720 conn(0x7fbc740fb9a0 msgr2=0x7fbc740fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 -- 172.21.15.8:0/3994519720 shutdown_connections 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.794+0000 7fbc7b58c640 1 -- 172.21.15.8:0/3994519720 wait complete. 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 Processor -- start 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 -- start start 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 0x7fbc74101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc740fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc74107270 0x7fbc74100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74100c60 con 0x7fbc74104940 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74100dd0 con 0x7fbc74107270 2024-02-19T16:22:02.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc7b58c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc741a1670 con 0x7fbc74102010 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc740fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc740fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44874/0 (socket says 172.21.15.8:44874) 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 -- 172.21.15.8:0/1574868669 learned_addr learned my addr 172.21.15.8:0/1574868669 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc79b29640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc74107270 0x7fbc74100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc79328640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 0x7fbc74101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 -- 172.21.15.8:0/1574868669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 msgr2=0x7fbc74101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 0x7fbc74101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 -- 172.21.15.8:0/1574868669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc74107270 msgr2=0x7fbc74100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc74107270 0x7fbc74100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.795+0000 7fbc78b27640 1 -- 172.21.15.8:0/1574868669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68009560 con 0x7fbc74104940 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc79328640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 0x7fbc74101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc78b27640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc740fffc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000c00 tx=0x7fbc68005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680043d0 con 0x7fbc74104940 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc680052d0 con 0x7fbc74104940 2024-02-19T16:22:02.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc741a18c0 con 0x7fbc74104940 2024-02-19T16:22:02.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68003d70 con 0x7fbc74104940 2024-02-19T16:22:02.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.796+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc741a1e20 con 0x7fbc74104940 2024-02-19T16:22:02.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.797+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc740fd490 con 0x7fbc74104940 2024-02-19T16:22:02.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.798+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fbc68005440 con 0x7fbc74104940 2024-02-19T16:22:02.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.798+0000 7fbc727fc640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbc5c0a2140 0x7fbc5c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:02.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.799+0000 7fbc79328640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbc5c0a2140 0x7fbc5c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:02.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.799+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fbc680f0110 con 0x7fbc74104940 2024-02-19T16:22:02.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.799+0000 7fbc79328640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbc5c0a2140 0x7fbc5c0a4600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009c20 tx=0x7fbc6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:02.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:02.803+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbc680b8da0 con 0x7fbc74104940 2024-02-19T16:22:03.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.023+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc7402d000 con 0x7fbc74104940 2024-02-19T16:22:03.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.024+0000 7fbc727fc640 1 -- 172.21.15.8:0/1574868669 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fbc680bc460 con 0x7fbc74104940 2024-02-19T16:22:03.025 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:03.025 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:41.315237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:03.025 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:22:03.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbc5c0a2140 msgr2=0x7fbc5c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbc5c0a2140 0x7fbc5c0a4600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009c20 tx=0x7fbc6c005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 msgr2=0x7fbc740fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc740fffc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000c00 tx=0x7fbc68005e50 comp rx=0 tx=0).stop 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 shutdown_connections 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbc5c0a2140 0x7fbc5c0a4600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc74107270 0x7fbc74100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc74104940 0x7fbc740fffc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 --2- 172.21.15.8:0/1574868669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc74102010 0x7fbc74101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 >> 172.21.15.8:0/1574868669 conn(0x7fbc740fb9a0 msgr2=0x7fbc74102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 shutdown_connections 2024-02-19T16:22:03.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.026+0000 7fbc7b58c640 1 -- 172.21.15.8:0/1574868669 wait complete. 2024-02-19T16:22:03.038 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:03.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3512893525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68101fc0 msgr2=0x7f1c68108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3512893525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68101fc0 0x7f1c68108850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009ca0 tx=0x7f1c6002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3512893525 shutdown_connections 2024-02-19T16:22:03.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3512893525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68101fc0 0x7f1c68108850 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3512893525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c68101600 0x7f1c68101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3512893525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c68100410 0x7f1c68100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3512893525 >> 172.21.15.8:0/3512893525 conn(0x7f1c680fb9a0 msgr2=0x7f1c680fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3512893525 shutdown_connections 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.141+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3512893525 wait complete. 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 Processor -- start 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 -- start start 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 0x7f1c68197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c68101600 0x7f1c68197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c68101fc0 0x7f1c6819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6819f050 con 0x7f1c68100410 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6819f1c0 con 0x7f1c68101fc0 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6e4c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6819f380 con 0x7f1c68101600 2024-02-19T16:22:03.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 0x7f1c68197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 0x7f1c68197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44884/0 (socket says 172.21.15.8:44884) 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c67fff640 1 -- 172.21.15.8:0/3757298457 learned_addr learned my addr 172.21.15.8:0/3757298457 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c6ca60640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c68101fc0 0x7f1c6819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.142+0000 7f1c677fe640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c68101600 0x7f1c68197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c67fff640 1 -- 172.21.15.8:0/3757298457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c68101600 msgr2=0x7f1c68197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c67fff640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c68101600 0x7f1c68197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c67fff640 1 -- 172.21.15.8:0/3757298457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c68101fc0 msgr2=0x7f1c6819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c67fff640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c68101fc0 0x7f1c6819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c67fff640 1 -- 172.21.15.8:0/3757298457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60009900 con 0x7f1c68100410 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c67fff640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 0x7f1c68197560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9c0 tx=0x7f1c5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:03.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c009700 con 0x7f1c68100410 2024-02-19T16:22:03.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6819f660 con 0x7f1c68100410 2024-02-19T16:22:03.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c00cd80 con 0x7f1c68100410 2024-02-19T16:22:03.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c010430 con 0x7f1c68100410 2024-02-19T16:22:03.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.143+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6819fbc0 con 0x7f1c68100410 2024-02-19T16:22:03.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.144+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c68107730 con 0x7f1c68100410 2024-02-19T16:22:03.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.145+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f1c5c003680 con 0x7f1c68100410 2024-02-19T16:22:03.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.145+0000 7f1c657fa640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1c500a20c0 0x7f1c500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.146+0000 7f1c677fe640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1c500a20c0 0x7f1c500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.146+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f1c5c014070 con 0x7f1c68100410 2024-02-19T16:22:03.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.146+0000 7f1c677fe640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1c500a20c0 0x7f1c500a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c68198a70 tx=0x7f1c58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:03.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.150+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f1c5c097020 con 0x7f1c68100410 2024-02-19T16:22:03.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.337+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c6810cf20 con 0x7f1c68100410 2024-02-19T16:22:03.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.339+0000 7f1c657fa640 1 -- 172.21.15.8:0/3757298457 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24545 (secure 0 0 0) 0x7f1c5c09a6e0 con 0x7f1c68100410 2024-02-19T16:22:03.340 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:03.340 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":93,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:21:21.872365+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:03.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1c500a20c0 msgr2=0x7f1c500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1c500a20c0 0x7f1c500a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c68198a70 tx=0x7f1c58005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 msgr2=0x7f1c68197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 0x7f1c68197560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9c0 tx=0x7f1c5c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 shutdown_connections 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f1c500a20c0 0x7f1c500a4580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c68101fc0 0x7f1c6819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c68101600 0x7f1c68197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 --2- 172.21.15.8:0/3757298457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c68100410 0x7f1c68197560 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 >> 172.21.15.8:0/3757298457 conn(0x7f1c680fb9a0 msgr2=0x7f1c68104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 shutdown_connections 2024-02-19T16:22:03.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.341+0000 7f1c6e4c3640 1 -- 172.21.15.8:0/3757298457 wait complete. 2024-02-19T16:22:03.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:03.354 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/1056379959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a58101fa0 msgr2=0x7f6a58108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/1056379959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a58101fa0 0x7f6a58108830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009ca0 tx=0x7f6a5002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/1056379959 shutdown_connections 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/1056379959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a58101fa0 0x7f6a58108830 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/1056379959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a581015e0 0x7f6a58101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/1056379959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 0x7f6a58100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.459+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/1056379959 >> 172.21.15.8:0/1056379959 conn(0x7f6a580fb980 msgr2=0x7f6a580fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/1056379959 shutdown_connections 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/1056379959 wait complete. 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 Processor -- start 2024-02-19T16:22:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 -- start start 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 0x7f6a58197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 0x7f6a58197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a58101fa0 0x7f6a5819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5819f050 con 0x7f6a581015e0 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5819f1c0 con 0x7f6a581003f0 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5fa8f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5819f380 con 0x7f6a58101fa0 2024-02-19T16:22:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5d02a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 0x7f6a58197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5d82b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 0x7f6a58197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5d02a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 0x7f6a58197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44898/0 (socket says 172.21.15.8:44898) 2024-02-19T16:22:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5d02a640 1 -- 172.21.15.8:0/3178540148 learned_addr learned my addr 172.21.15.8:0/3178540148 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.460+0000 7f6a5e02c640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a58101fa0 0x7f6a5819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d02a640 1 -- 172.21.15.8:0/3178540148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a58101fa0 msgr2=0x7f6a5819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d02a640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a58101fa0 0x7f6a5819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d02a640 1 -- 172.21.15.8:0/3178540148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 msgr2=0x7f6a58197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d02a640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 0x7f6a58197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d02a640 1 -- 172.21.15.8:0/3178540148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a50009900 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d82b640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 0x7f6a58197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5d02a640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 0x7f6a58197aa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800e9f0 tx=0x7f6a4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48009720 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a4800cd80 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48010690 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5819f660 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.461+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5819fb70 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.462+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58107710 con 0x7f6a581015e0 2024-02-19T16:22:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.463+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f6a480046c0 con 0x7f6a581015e0 2024-02-19T16:22:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.463+0000 7f6a4effd640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6a440a2140 0x7f6a440a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.464+0000 7f6a5d82b640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6a440a2140 0x7f6a440a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.464+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f6a48014070 con 0x7f6a581015e0 2024-02-19T16:22:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.464+0000 7f6a5d82b640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6a440a2140 0x7f6a440a4600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a58101440 tx=0x7f6a54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.468+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f6a48096f70 con 0x7f6a581015e0 2024-02-19T16:22:03.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:03 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2764532528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:03.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:03 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4120355756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:03.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:03 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1574868669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.647+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a5802d000 con 0x7f6a581015e0 2024-02-19T16:22:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.648+0000 7f6a4effd640 1 -- 172.21.15.8:0/3178540148 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a4809a630 con 0x7f6a581015e0 2024-02-19T16:22:03.649 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6a440a2140 msgr2=0x7f6a440a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6a440a2140 0x7f6a440a4600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a58101440 tx=0x7f6a54005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 msgr2=0x7f6a58197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 0x7f6a58197aa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800e9f0 tx=0x7f6a4800edf0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 shutdown_connections 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f6a440a2140 0x7f6a440a4600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a58101fa0 0x7f6a5819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a581015e0 0x7f6a58197aa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 --2- 172.21.15.8:0/3178540148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a581003f0 0x7f6a58197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 >> 172.21.15.8:0/3178540148 conn(0x7f6a580fb980 msgr2=0x7f6a581048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 shutdown_connections 2024-02-19T16:22:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.650+0000 7f6a5fa8f640 1 -- 172.21.15.8:0/3178540148 wait complete. 2024-02-19T16:22:03.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:03 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2764532528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:03.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:03 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4120355756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:03.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:03 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1574868669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:03.662 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:03.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 -- 172.21.15.8:0/833019176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101600 msgr2=0x7fb218101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/833019176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101600 0x7fb218101a80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009900 tx=0x7fb20c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 -- 172.21.15.8:0/833019176 shutdown_connections 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/833019176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb218101fc0 0x7fb218108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/833019176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101600 0x7fb218101a80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/833019176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb218100410 0x7fb218100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 -- 172.21.15.8:0/833019176 >> 172.21.15.8:0/833019176 conn(0x7fb2180fb9a0 msgr2=0x7fb2180fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 -- 172.21.15.8:0/833019176 shutdown_connections 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.766+0000 7fb21c8c3640 1 -- 172.21.15.8:0/833019176 wait complete. 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 Processor -- start 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 -- start start 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb218100410 0x7fb218197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb218101600 0x7fb218197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 0x7fb21819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21819f150 con 0x7fb218101fc0 2024-02-19T16:22:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21819f2c0 con 0x7fb218100410 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21c8c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21819f480 con 0x7fb218101600 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb216d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 0x7fb21819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb21659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb218100410 0x7fb218197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb216d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 0x7fb21819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44912/0 (socket says 172.21.15.8:44912) 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb216d9d640 1 -- 172.21.15.8:0/2309990592 learned_addr learned my addr 172.21.15.8:0/2309990592 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.767+0000 7fb215d9b640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb218101600 0x7fb218197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb216d9d640 1 -- 172.21.15.8:0/2309990592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb218101600 msgr2=0x7fb218197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb216d9d640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb218101600 0x7fb218197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb216d9d640 1 -- 172.21.15.8:0/2309990592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb218100410 msgr2=0x7fb218197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb216d9d640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb218100410 0x7fb218197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb216d9d640 1 -- 172.21.15.8:0/2309990592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c009560 con 0x7fb218101fc0 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb216d9d640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 0x7fb21819ec10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb20800eb10 tx=0x7fb20800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208004b20 con 0x7fb218101fc0 2024-02-19T16:22:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb208004c80 con 0x7fb218101fc0 2024-02-19T16:22:03.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218102880 con 0x7fb218101fc0 2024-02-19T16:22:03.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208010650 con 0x7fb218101fc0 2024-02-19T16:22:03.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.768+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218102d60 con 0x7fb218101fc0 2024-02-19T16:22:03.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.769+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21810be20 con 0x7fb218101fc0 2024-02-19T16:22:03.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.770+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fb208005020 con 0x7fb218101fc0 2024-02-19T16:22:03.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.771+0000 7fb2077fe640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2000a2140 0x7fb2000a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:03.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.771+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fb208014070 con 0x7fb218101fc0 2024-02-19T16:22:03.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.771+0000 7fb21659c640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2000a2140 0x7fb2000a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:03.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.771+0000 7fb21659c640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2000a2140 0x7fb2000a4600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb218101460 tx=0x7fb210012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:03.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.775+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb2080971b0 con 0x7fb218101fc0 2024-02-19T16:22:03.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:03 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2764532528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:03.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:03 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4120355756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:03.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:03 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1574868669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:03.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.956+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb218103390 con 0x7fb218101fc0 2024-02-19T16:22:03.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.956+0000 7fb2077fe640 1 -- 172.21.15.8:0/2309990592 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb20809a870 con 0x7fb218101fc0 2024-02-19T16:22:03.957 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2000a2140 msgr2=0x7fb2000a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2000a2140 0x7fb2000a4600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb218101460 tx=0x7fb210012040 comp rx=0 tx=0).stop 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 msgr2=0x7fb21819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 0x7fb21819ec10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb20800eb10 tx=0x7fb20800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 shutdown_connections 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fb2000a2140 0x7fb2000a4600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb218101fc0 0x7fb21819ec10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb218101600 0x7fb218197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 --2- 172.21.15.8:0/2309990592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb218100410 0x7fb218197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:03.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 >> 172.21.15.8:0/2309990592 conn(0x7fb2180fb9a0 msgr2=0x7fb2180fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:03.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 shutdown_connections 2024-02-19T16:22:03.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:03.958+0000 7fb21c8c3640 1 -- 172.21.15.8:0/2309990592 wait complete. 2024-02-19T16:22:03.974 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:04.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1302680217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe0101500 msgr2=0x7f0fe0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1302680217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe0101500 0x7f0fe0101920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009900 tx=0x7f0fd002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1302680217 shutdown_connections 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1302680217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fe01030b0 0x7f0fe0109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1302680217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe01026f0 0x7f0fe0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1302680217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe0101500 0x7f0fe0101920 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1302680217 >> 172.21.15.8:0/1302680217 conn(0x7f0fe00fca90 msgr2=0x7f0fe00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1302680217 shutdown_connections 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.079+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1302680217 wait complete. 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 Processor -- start 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 -- start start 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 0x7f0fe0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fe01026f0 0x7f0fe0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 0x7f0fe019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019f060 con 0x7f0fe01030b0 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019f1d0 con 0x7f0fe0101500 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fe5e12640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019f390 con 0x7f0fe01026f0 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 0x7f0fe019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fdf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 0x7f0fe0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fdf7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 0x7f0fe0197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:49164/0 (socket says 172.21.15.8:49164) 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 0x7f0fe019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44936/0 (socket says 172.21.15.8:44936) 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fdf7fe640 1 -- 172.21.15.8:0/1228726068 learned_addr learned my addr 172.21.15.8:0/1228726068 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.080+0000 7f0fdeffd640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fe01026f0 0x7f0fe0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdffff640 1 -- 172.21.15.8:0/1228726068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fe01026f0 msgr2=0x7f0fe0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdffff640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fe01026f0 0x7f0fe0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdffff640 1 -- 172.21.15.8:0/1228726068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 msgr2=0x7f0fe0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdffff640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 0x7f0fe0197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdffff640 1 -- 172.21.15.8:0/1228726068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009490 con 0x7f0fe01030b0 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdf7fe640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 0x7f0fe0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdffff640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 0x7f0fe019eb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009fd0 tx=0x7f0fd800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800ce20 con 0x7f0fe01030b0 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fd8005020 con 0x7f0fe01030b0 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd0009560 con 0x7f0fe01030b0 2024-02-19T16:22:04.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8010690 con 0x7f0fe01030b0 2024-02-19T16:22:04.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.081+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe019f9f0 con 0x7f0fe01030b0 2024-02-19T16:22:04.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.082+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe01088a0 con 0x7f0fe01030b0 2024-02-19T16:22:04.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.083+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f0fd8005500 con 0x7f0fe01030b0 2024-02-19T16:22:04.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.084+0000 7f0fdcff9640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0fc80a20c0 0x7f0fc80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.084+0000 7f0fdf7fe640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0fc80a20c0 0x7f0fc80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.084+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f0fd8014070 con 0x7f0fe01030b0 2024-02-19T16:22:04.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.084+0000 7f0fdf7fe640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0fc80a20c0 0x7f0fc80a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0fd002c840 tx=0x7f0fd002cd10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:04.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.088+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f0fd8097360 con 0x7f0fe01030b0 2024-02-19T16:22:04.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.269+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fe002d000 con 0x7f0fe01030b0 2024-02-19T16:22:04.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.270+0000 7f0fdcff9640 1 -- 172.21.15.8:0/1228726068 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fd8005190 con 0x7f0fe01030b0 2024-02-19T16:22:04.271 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:04.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0fc80a20c0 msgr2=0x7f0fc80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0fc80a20c0 0x7f0fc80a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0fd002c840 tx=0x7f0fd002cd10 comp rx=0 tx=0).stop 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 msgr2=0x7f0fe019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 0x7f0fe019eb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009fd0 tx=0x7f0fd800ef10 comp rx=0 tx=0).stop 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 shutdown_connections 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f0fc80a20c0 0x7f0fc80a4580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fe01030b0 0x7f0fe019eb20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fe01026f0 0x7f0fe0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 --2- 172.21.15.8:0/1228726068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe0101500 0x7f0fe0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 >> 172.21.15.8:0/1228726068 conn(0x7f0fe00fca90 msgr2=0x7f0fe0105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 shutdown_connections 2024-02-19T16:22:04.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.272+0000 7f0fe5e12640 1 -- 172.21.15.8:0/1228726068 wait complete. 2024-02-19T16:22:04.284 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:04.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 -- 172.21.15.8:0/2992159338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3640ffb80 msgr2=0x7fe3640fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 --2- 172.21.15.8:0/2992159338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3640ffb80 0x7fe3640fffa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe360009900 tx=0x7fe36002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 -- 172.21.15.8:0/2992159338 shutdown_connections 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 --2- 172.21.15.8:0/2992159338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe364101730 0x7fe364107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 --2- 172.21.15.8:0/2992159338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe364100d70 0x7fe3641011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 --2- 172.21.15.8:0/2992159338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3640ffb80 0x7fe3640fffa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 -- 172.21.15.8:0/2992159338 >> 172.21.15.8:0/2992159338 conn(0x7fe3640fb110 msgr2=0x7fe3640fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 -- 172.21.15.8:0/2992159338 shutdown_connections 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 -- 172.21.15.8:0/2992159338 wait complete. 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 Processor -- start 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.389+0000 7fe36b29d640 1 -- start start 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3640ffb80 0x7fe364196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 0x7fe3641969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe364101730 0x7fe36419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36419df80 con 0x7fe364100d70 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36419e0f0 con 0x7fe3640ffb80 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36419e2b0 con 0x7fe364101730 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 0x7fe3641969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 0x7fe3641969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44940/0 (socket says 172.21.15.8:44940) 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 -- 172.21.15.8:0/819375312 learned_addr learned my addr 172.21.15.8:0/819375312 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe369039640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3640ffb80 0x7fe364196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36983a640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe364101730 0x7fe36419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 -- 172.21.15.8:0/819375312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe364101730 msgr2=0x7fe36419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe364101730 0x7fe36419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 -- 172.21.15.8:0/819375312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3640ffb80 msgr2=0x7fe364196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3640ffb80 0x7fe364196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 -- 172.21.15.8:0/819375312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360009560 con 0x7fe364100d70 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe368838640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 0x7fe3641969d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe35400e9f0 tx=0x7fe35400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:04.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354009700 con 0x7fe364100d70 2024-02-19T16:22:04.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe35400cd80 con 0x7fe364100d70 2024-02-19T16:22:04.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36419e590 con 0x7fe364100d70 2024-02-19T16:22:04.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354018920 con 0x7fe364100d70 2024-02-19T16:22:04.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.390+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36419eaf0 con 0x7fe364100d70 2024-02-19T16:22:04.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.391+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364106f90 con 0x7fe364100d70 2024-02-19T16:22:04.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.393+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fe354018a80 con 0x7fe364100d70 2024-02-19T16:22:04.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.393+0000 7fe35a7fc640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe3500a2140 0x7fe3500a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.393+0000 7fe369039640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe3500a2140 0x7fe3500a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.393+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fe354014070 con 0x7fe364100d70 2024-02-19T16:22:04.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.394+0000 7fe369039640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe3500a2140 0x7fe3500a4600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe360005ab0 tx=0x7fe360005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:04.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.398+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe354053230 con 0x7fe364100d70 2024-02-19T16:22:04.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:04 smithi156 ceph-mon[36502]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:22:04.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:04 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3757298457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:04.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:04 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3178540148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:04 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2309990592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:04 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1228726068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.579+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe36402d000 con 0x7fe364100d70 2024-02-19T16:22:04.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.579+0000 7fe35a7fc640 1 -- 172.21.15.8:0/819375312 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe354097010 con 0x7fe364100d70 2024-02-19T16:22:04.580 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:04.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe3500a2140 msgr2=0x7fe3500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe3500a2140 0x7fe3500a4600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe360005ab0 tx=0x7fe360005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 msgr2=0x7fe3641969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 0x7fe3641969d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe35400e9f0 tx=0x7fe35400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 shutdown_connections 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe3500a2140 0x7fe3500a4600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe364101730 0x7fe36419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe364100d70 0x7fe3641969d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 --2- 172.21.15.8:0/819375312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3640ffb80 0x7fe364196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 >> 172.21.15.8:0/819375312 conn(0x7fe3640fb110 msgr2=0x7fe364104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 shutdown_connections 2024-02-19T16:22:04.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.582+0000 7fe36b29d640 1 -- 172.21.15.8:0/819375312 wait complete. 2024-02-19T16:22:04.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:04 smithi008 ceph-mon[32573]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:22:04.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:04 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3757298457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:04.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:04 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3178540148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:04 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2309990592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:04 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1228726068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.594 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:04.677 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:04 smithi191 ceph-mon[37203]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-19T16:22:04.677 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:04 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3757298457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:04.677 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:04 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3178540148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.677 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:04 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2309990592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.677 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:04 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1228726068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3113089089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 msgr2=0x7f4ff8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3113089089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8108870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009ca0 tx=0x7f4ff002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3113089089 shutdown_connections 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3113089089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8108870 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3113089089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 0x7f4ff8101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3113089089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 0x7f4ff8100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3113089089 >> 172.21.15.8:0/3113089089 conn(0x7f4ff80fb9a0 msgr2=0x7f4ff80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:04.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3113089089 shutdown_connections 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.697+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3113089089 wait complete. 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 Processor -- start 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 -- start start 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 0x7f4ff81043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 0x7f4ff8102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81036a0 con 0x7f4ff8101fe0 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8103810 con 0x7f4ff8100430 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ffcbe2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81a16a0 con 0x7f4ff8101620 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44954/0 (socket says 172.21.15.8:44954) 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 -- 172.21.15.8:0/3301139706 learned_addr learned my addr 172.21.15.8:0/3301139706 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4feffff640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 0x7f4ff8102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff659c640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 0x7f4ff81043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 -- 172.21.15.8:0/3301139706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 msgr2=0x7f4ff8102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 0x7f4ff8102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 -- 172.21.15.8:0/3301139706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 msgr2=0x7f4ff81043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 0x7f4ff81043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff6d9d640 1 -- 172.21.15.8:0/3301139706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009900 con 0x7f4ff8101fe0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4feffff640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 0x7f4ff8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.698+0000 7f4ff659c640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 0x7f4ff81043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.699+0000 7f4ff6d9d640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8102f40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0032650 tx=0x7f4ff0032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.699+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0034e20 con 0x7f4ff8101fe0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.699+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff81a18f0 con 0x7f4ff8101fe0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.699+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0003e50 con 0x7f4ff8101fe0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.699+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0004880 con 0x7f4ff8101fe0 2024-02-19T16:22:04.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.699+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff81a1df0 con 0x7f4ff8101fe0 2024-02-19T16:22:04.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.700+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff810be40 con 0x7f4ff8101fe0 2024-02-19T16:22:04.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.701+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4ff000a040 con 0x7f4ff8101fe0 2024-02-19T16:22:04.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.702+0000 7f4fef7fe640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fe40a20c0 0x7f4fe40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:04.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.702+0000 7f4ff659c640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fe40a20c0 0x7f4fe40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:04.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.702+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f4ff00f0ca0 con 0x7f4ff8101fe0 2024-02-19T16:22:04.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.702+0000 7f4ff659c640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fe40a20c0 0x7f4fe40a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8101480 tx=0x7f4fe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:04.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.706+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f4ff00b98d0 con 0x7f4ff8101fe0 2024-02-19T16:22:04.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.931+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ff81a1f80 con 0x7f4ff8101fe0 2024-02-19T16:22:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.932+0000 7f4fef7fe640 1 -- 172.21.15.8:0/3301139706 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f4ff00bcf90 con 0x7f4ff8101fe0 2024-02-19T16:22:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:04.933 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:41.315237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fe40a20c0 msgr2=0x7f4fe40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fe40a20c0 0x7f4fe40a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8101480 tx=0x7f4fe8005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 msgr2=0x7f4ff8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8102f40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0032650 tx=0x7f4ff0032fa0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 shutdown_connections 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fe40a20c0 0x7f4fe40a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff8101fe0 0x7f4ff8102f40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8101620 0x7f4ff8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 --2- 172.21.15.8:0/3301139706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ff8100430 0x7f4ff81043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:04.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 >> 172.21.15.8:0/3301139706 conn(0x7f4ff80fb9a0 msgr2=0x7f4ff80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:04.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 shutdown_connections 2024-02-19T16:22:04.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:04.934+0000 7f4ffcbe2640 1 -- 172.21.15.8:0/3301139706 wait complete. 2024-02-19T16:22:04.947 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:05.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.048+0000 7f89ef24e640 1 -- 172.21.15.8:0/243137043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101fc0 msgr2=0x7f89e8108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.048+0000 7f89ef24e640 1 --2- 172.21.15.8:0/243137043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101fc0 0x7f89e8108850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009ca0 tx=0x7f89e002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- 172.21.15.8:0/243137043 shutdown_connections 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 --2- 172.21.15.8:0/243137043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101fc0 0x7f89e8108850 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 --2- 172.21.15.8:0/243137043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89e8101600 0x7f89e8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 --2- 172.21.15.8:0/243137043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8100410 0x7f89e8100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- 172.21.15.8:0/243137043 >> 172.21.15.8:0/243137043 conn(0x7f89e80fb980 msgr2=0x7f89e80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- 172.21.15.8:0/243137043 shutdown_connections 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- 172.21.15.8:0/243137043 wait complete. 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 Processor -- start 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- start start 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89e8100410 0x7f89e8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 0x7f89e8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8101fc0 0x7f89e819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e819f000 con 0x7f89e8101600 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e819f170 con 0x7f89e8100410 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89ef24e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e819f330 con 0x7f89e8101fc0 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.049+0000 7f89e7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 0x7f89e8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 0x7f89e8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44970/0 (socket says 172.21.15.8:44970) 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 -- 172.21.15.8:0/2533881922 learned_addr learned my addr 172.21.15.8:0/2533881922 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89ed7eb640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8101fc0 0x7f89e819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89ecfea640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89e8100410 0x7f89e8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 -- 172.21.15.8:0/2533881922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8101fc0 msgr2=0x7f89e819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8101fc0 0x7f89e819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 -- 172.21.15.8:0/2533881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89e8100410 msgr2=0x7f89e8197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89e8100410 0x7f89e8197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 -- 172.21.15.8:0/2533881922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e0009900 con 0x7f89e8101600 2024-02-19T16:22:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e7fff640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 0x7f89e8197a50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00e9f0 tx=0x7f89dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc009700 con 0x7f89e8101600 2024-02-19T16:22:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89dc00cd80 con 0x7f89e8101600 2024-02-19T16:22:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e819f610 con 0x7f89e8101600 2024-02-19T16:22:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc018920 con 0x7f89e8101600 2024-02-19T16:22:05.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.050+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e819fb70 con 0x7f89e8101600 2024-02-19T16:22:05.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.051+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e810be20 con 0x7f89e8101600 2024-02-19T16:22:05.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.052+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f89dc018a80 con 0x7f89e8101600 2024-02-19T16:22:05.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.053+0000 7f89e5ffb640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f89d00a2140 0x7f89d00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.053+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f89dc014070 con 0x7f89e8101600 2024-02-19T16:22:05.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.053+0000 7f89ecfea640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f89d00a2140 0x7f89d00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.054+0000 7f89ecfea640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f89d00a2140 0x7f89d00a4600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f89e8101460 tx=0x7f89d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.057+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f89dc053230 con 0x7f89e8101600 2024-02-19T16:22:05.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.240+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e810bfb0 con 0x7f89e8101600 2024-02-19T16:22:05.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.242+0000 7f89e5ffb640 1 -- 172.21.15.8:0/2533881922 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24545 (secure 0 0 0) 0x7f89dc097010 con 0x7f89e8101600 2024-02-19T16:22:05.243 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:05.243 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":93,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:21:21.872365+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f89d00a2140 msgr2=0x7f89d00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f89d00a2140 0x7f89d00a4600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f89e8101460 tx=0x7f89d8005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 msgr2=0x7f89e8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 0x7f89e8197a50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00e9f0 tx=0x7f89dc00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 shutdown_connections 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f89d00a2140 0x7f89d00a4600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8101fc0 0x7f89e819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89e8101600 0x7f89e8197a50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 --2- 172.21.15.8:0/2533881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89e8100410 0x7f89e8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 >> 172.21.15.8:0/2533881922 conn(0x7f89e80fb980 msgr2=0x7f89e80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 shutdown_connections 2024-02-19T16:22:05.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.244+0000 7f89ef24e640 1 -- 172.21.15.8:0/2533881922 wait complete. 2024-02-19T16:22:05.256 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 -- 172.21.15.8:0/1247130924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101600 msgr2=0x7f338c101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 --2- 172.21.15.8:0/1247130924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101600 0x7f338c101a80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3380009900 tx=0x7f338002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 -- 172.21.15.8:0/1247130924 shutdown_connections 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 --2- 172.21.15.8:0/1247130924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f338c101fc0 0x7f338c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 --2- 172.21.15.8:0/1247130924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101600 0x7f338c101a80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 --2- 172.21.15.8:0/1247130924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c100410 0x7f338c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 -- 172.21.15.8:0/1247130924 >> 172.21.15.8:0/1247130924 conn(0x7f338c0fb980 msgr2=0x7f338c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 -- 172.21.15.8:0/1247130924 shutdown_connections 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.358+0000 7f33942f4640 1 -- 172.21.15.8:0/1247130924 wait complete. 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 Processor -- start 2024-02-19T16:22:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 -- start start 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f338c100410 0x7f338c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c101600 0x7f338c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 0x7f338c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c19f070 con 0x7f338c101fc0 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c19f1e0 con 0x7f338c100410 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f33942f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c19f3a0 con 0x7f338c101600 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392891640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 0x7f338c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392891640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 0x7f338c19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44986/0 (socket says 172.21.15.8:44986) 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392891640 1 -- 172.21.15.8:0/3743265001 learned_addr learned my addr 172.21.15.8:0/3743265001 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392090640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f338c100410 0x7f338c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f339188f640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c101600 0x7f338c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392891640 1 -- 172.21.15.8:0/3743265001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c101600 msgr2=0x7f338c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392891640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c101600 0x7f338c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.359+0000 7f3392891640 1 -- 172.21.15.8:0/3743265001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f338c100410 msgr2=0x7f338c197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f3392891640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f338c100410 0x7f338c197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f3392891640 1 -- 172.21.15.8:0/3743265001 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384009490 con 0x7f338c101fc0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f3392891640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 0x7f338c19eb30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f337c00eb10 tx=0x7f337c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c004b20 con 0x7f338c101fc0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f337c004c80 con 0x7f338c101fc0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3380009560 con 0x7f338c101fc0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c010650 con 0x7f338c101fc0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.360+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c19f9d0 con 0x7f338c101fc0 2024-02-19T16:22:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.361+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c107800 con 0x7f338c101fc0 2024-02-19T16:22:05.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.362+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f337c0107f0 con 0x7f338c101fc0 2024-02-19T16:22:05.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.363+0000 7f338b7fe640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f33740a2140 0x7f33740a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.363+0000 7f3392090640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f33740a2140 0x7f33740a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.363+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f337c014070 con 0x7f338c101fc0 2024-02-19T16:22:05.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.363+0000 7f3392090640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f33740a2140 0x7f33740a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f338c101460 tx=0x7f3384005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.366+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f337c097380 con 0x7f338c101fc0 2024-02-19T16:22:05.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:05 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/819375312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:05.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:05 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3301139706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:05.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:05 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2533881922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:05.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:05 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/819375312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:05.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:05 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3301139706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:05.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:05 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2533881922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:05.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.586+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f338c10cf20 con 0x7f338c101fc0 2024-02-19T16:22:05.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.586+0000 7f338b7fe640 1 -- 172.21.15.8:0/3743265001 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f337c09aa40 con 0x7f338c101fc0 2024-02-19T16:22:05.587 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:05.587 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:41.315237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:05.587 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.588+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f33740a2140 msgr2=0x7f33740a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.588+0000 7f33942f4640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f33740a2140 0x7f33740a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f338c101460 tx=0x7f3384005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 msgr2=0x7f338c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 0x7f338c19eb30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f337c00eb10 tx=0x7f337c00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 shutdown_connections 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f33740a2140 0x7f33740a4600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f338c101fc0 0x7f338c19eb30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c101600 0x7f338c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 --2- 172.21.15.8:0/3743265001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f338c100410 0x7f338c197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 >> 172.21.15.8:0/3743265001 conn(0x7f338c0fb980 msgr2=0x7f338c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 shutdown_connections 2024-02-19T16:22:05.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.589+0000 7f33942f4640 1 -- 172.21.15.8:0/3743265001 wait complete. 2024-02-19T16:22:05.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-19T16:20:21.002558+0000', 'modified': '2024-02-19T16:20:41.315237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24502, 'mds_1': 24488, 'mds_2': 24475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6833/2702528663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 2702528663}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 2702528663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/3993132637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3993132637}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3993132637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/562441805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 562441805}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 562441805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.191:6837/1523568014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 1523568014}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1523568014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6839/3057541491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 3057541491}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 3057541491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.191:6839/2229091821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2229091821}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2229091821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-19T16:22:05.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-19T16:22:05.601 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-19T16:22:05.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1784562336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f881030b0 msgr2=0x7f2f88109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1784562336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f881030b0 0x7f2f88109940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009ca0 tx=0x7f2f8002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1784562336 shutdown_connections 2024-02-19T16:22:05.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1784562336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f881030b0 0x7f2f88109940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1784562336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f881026f0 0x7f2f88102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1784562336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f88101500 0x7f2f88101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1784562336 >> 172.21.15.8:0/1784562336 conn(0x7f2f880fca90 msgr2=0x7f2f880feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1784562336 shutdown_connections 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.706+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1784562336 wait complete. 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 Processor -- start 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 -- start start 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 0x7f2f88197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f881026f0 0x7f2f88197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f881030b0 0x7f2f8819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8819f050 con 0x7f2f88101500 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8819f1c0 con 0x7f2f881026f0 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f8d8d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8819f380 con 0x7f2f881030b0 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 0x7f2f88197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 0x7f2f88197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49598/0 (socket says 172.21.15.8:49598) 2024-02-19T16:22:05.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 -- 172.21.15.8:0/1199890597 learned_addr learned my addr 172.21.15.8:0/1199890597 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f881026f0 0x7f2f88197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f877fe640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f881030b0 0x7f2f8819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 -- 172.21.15.8:0/1199890597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f881030b0 msgr2=0x7f2f8819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f881030b0 0x7f2f8819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 -- 172.21.15.8:0/1199890597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f881026f0 msgr2=0x7f2f88197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f881026f0 0x7f2f88197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.707+0000 7f2f86ffd640 1 -- 172.21.15.8:0/1199890597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f80009900 con 0x7f2f88101500 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.708+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 0x7f2f88197560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800e9c0 tx=0x7f2f7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.708+0000 7f2f8c8d2640 1 -- 172.21.15.8:0/1199890597 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78009700 con 0x7f2f88101500 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.708+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8819f660 con 0x7f2f88101500 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.708+0000 7f2f8c8d2640 1 -- 172.21.15.8:0/1199890597 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f7800cd80 con 0x7f2f88101500 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.708+0000 7f2f8c8d2640 1 -- 172.21.15.8:0/1199890597 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78010430 con 0x7f2f88101500 2024-02-19T16:22:05.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.708+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8819fb70 con 0x7f2f88101500 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f8c8d2640 1 -- 172.21.15.8:0/1199890597 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2f78003680 con 0x7f2f88101500 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f90582740 1 Processor -- start 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f90582740 1 -- start start 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f90582740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ee93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f90582740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5eee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f90582740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.710+0000 7f2f90582740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5ea4cd0 con 0x564fe5ee8f80 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f90582740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5e57f20 con 0x564fe5eeeae0 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f90582740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5e55450 con 0x564fe5ee9970 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f8c8d2640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f740a20c0 0x7f2f740a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f877fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ee93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5eee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f877fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ee93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49604/0 (socket says 172.21.15.8:49604) 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f877fe640 1 -- 172.21.15.8:0/1949732754 learned_addr learned my addr 172.21.15.8:0/1949732754 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f877fe640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f740a20c0 0x7f2f740a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1949732754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 msgr2=0x564fe5eee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5eee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1949732754 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 msgr2=0x564fe5ee93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ee93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f8c8d2640 1 -- 172.21.15.8:0/1199890597 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f2f78014070 con 0x7f2f88101500 2024-02-19T16:22:05.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1949732754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fe5e05f20 con 0x564fe5eeeae0 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f877fe640 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ee93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f877fe640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f740a20c0 0x7f2f740a4580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x564fe5eea1d0 tx=0x7f2f800326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef0f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009bf0 tx=0x7f2f70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b6ef478f5586b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.711+0000 7f2f7dffb640 1 -- 172.21.15.8:0/1949732754 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70004050 con 0x564fe5eeeae0 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f7dffb640 1 -- 172.21.15.8:0/1949732754 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f70005500 con 0x564fe5eeeae0 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f7dffb640 1 -- 172.21.15.8:0/1949732754 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70004d90 con 0x564fe5eeeae0 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 -- 172.21.15.8:0/1949732754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 msgr2=0x564fe5ef0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef0f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009bf0 tx=0x7f2f70011810 comp rx=0 tx=0).stop 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 -- 172.21.15.8:0/1949732754 shutdown_connections 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef0f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5eee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 --2- 172.21.15.8:0/1949732754 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ee93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 -- 172.21.15.8:0/1949732754 >> 172.21.15.8:0/1949732754 conn(0x564fe5ee44f0 msgr2=0x564fe5ee48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 -- 172.21.15.8:0/1949732754 shutdown_connections 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 -- 172.21.15.8:0/1949732754 wait complete. 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.712+0000 7f2f90582740 1 -- start start 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 Processor -- start 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee8f80 0x564fe5ef3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 0x564fe5ef3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5efa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5e50c30 con 0x564fe5ee9970 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5df5d30 con 0x564fe5ee8f80 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5b24310 con 0x564fe5eeeae0 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f90582740 10 client.0 _ll_register_callbacks cb 0x564fe5dfdd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 0x564fe5ef3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 0x564fe5ef3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49614/0 (socket says 172.21.15.8:49614) 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1249533127 learned_addr learned my addr 172.21.15.8:0/1249533127 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee8f80 0x564fe5ef3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f877fe640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5efa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 msgr2=0x564fe5efa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5efa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1249533127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee8f80 msgr2=0x564fe5ef3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee8f80 0x564fe5ef3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f7ffff640 1 -- 172.21.15.8:0/1249533127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f70009850 con 0x564fe5ee9970 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.713+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee8f80 0x564fe5ef3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 0x564fe5ef3640 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2f7001b040 tx=0x7f2f70009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375107d27486965 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f66ffd640 1 -- 172.21.15.8:0/1249533127 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70004f20 con 0x564fe5ee9970 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f66ffd640 1 -- 172.21.15.8:0/1249533127 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f2f70005080 con 0x564fe5ee9970 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fe5b506c0 con 0x564fe5ee9970 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f66ffd640 1 -- 172.21.15.8:0/1249533127 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f70005240 con 0x564fe5ee9970 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 10 client.15213 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:05.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564fe5e4f2b0 con 0x564fe5ee9970 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f66ffd640 1 -- 172.21.15.8:0/1249533127 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564fe5b506c0 con 0x564fe5ee9970 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f66ffd640 1 -- 172.21.15.8:0/1249533127 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f70021040 con 0x564fe5ee9970 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 10 client.15213 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 10 client.15213 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 20 client.15213 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5eff9b0 0x564fe5efec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 4 client.15213 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.714+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fe5ba1270 con 0x564fe5eff9b0 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.715+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5eff9b0 0x564fe5efec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.715+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5eff9b0 0x564fe5efec30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1fea6ca42f899ce server_cookie=8e64828897dbf28f in_seq=0 out_seq=0 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.715+0000 7f2f66ffd640 10 client.15213 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f66ffd640 1 -- 172.21.15.8:0/1249533127 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564fe5ba1270 con 0x564fe5eff9b0 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f66ffd640 10 client.15213 handle_command_reply: tid=0 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 client.15213 shutdown 2024-02-19T16:22:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 msgr2=0x564fe5ef3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 0x564fe5ef3640 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2f7001b040 tx=0x7f2f70009fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 shutdown_connections 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5eff9b0 0x564fe5efec30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5efa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee9970 0x564fe5ef3640 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 --2- 172.21.15.8:0/1249533127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee8f80 0x564fe5ef3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f66ffd640 0 client.15213 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 >> 172.21.15.8:0/1249533127 conn(0x564fe5ee44f0 msgr2=0x564fe5ee37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 shutdown_connections 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 1 -- 172.21.15.8:0/1249533127 wait complete. 2024-02-19T16:22:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.716+0000 7f2f90582740 20 client.15213 trim_cache size 0 max 16384 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.732+0000 7f2f90582740 1 Processor -- start 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.732+0000 7f2f90582740 1 -- start start 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f90582740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f90582740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee9970 0x564fe5efa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f90582740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5ef1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f90582740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5dd6490 con 0x564fe5ee8f80 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f90582740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5ef2950 con 0x564fe5ee9970 2024-02-19T16:22:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f90582740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5efee60 con 0x564fe5eeeae0 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49628/0 (socket says 172.21.15.8:49628) 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 -- 172.21.15.8:0/3240535067 learned_addr learned my addr 172.21.15.8:0/3240535067 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f877fe640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5ef1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee9970 0x564fe5efa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 -- 172.21.15.8:0/3240535067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 msgr2=0x564fe5ef1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5ef1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 -- 172.21.15.8:0/3240535067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee9970 msgr2=0x564fe5efa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee9970 0x564fe5efa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 -- 172.21.15.8:0/3240535067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fe5deba20 con 0x564fe5ee8f80 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f877fe640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5ef1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee9970 0x564fe5efa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef0ed0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2f780971e0 tx=0x7f2f7806de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ece66fa665d4b58 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f7d7fa640 1 -- 172.21.15.8:0/3240535067 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7809eed0 con 0x564fe5ee8f80 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f7d7fa640 1 -- 172.21.15.8:0/3240535067 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f780a3520 con 0x564fe5ee8f80 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.733+0000 7f2f7d7fa640 1 -- 172.21.15.8:0/3240535067 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7809a430 con 0x564fe5ee8f80 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 -- 172.21.15.8:0/3240535067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 msgr2=0x564fe5ef0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef0ed0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2f780971e0 tx=0x7f2f7806de40 comp rx=0 tx=0).stop 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 -- 172.21.15.8:0/3240535067 shutdown_connections 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5eeeae0 0x564fe5ef1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5ee9970 0x564fe5efa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 --2- 172.21.15.8:0/3240535067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef0ed0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 -- 172.21.15.8:0/3240535067 >> 172.21.15.8:0/3240535067 conn(0x564fe5ee44f0 msgr2=0x564fe5edf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 -- 172.21.15.8:0/3240535067 shutdown_connections 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 -- 172.21.15.8:0/3240535067 wait complete. 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 -- start start 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.734+0000 7f2f90582740 1 Processor -- start 2024-02-19T16:22:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5efa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5ef6f30 con 0x564fe5ee8f80 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5ef70a0 con 0x564fe5eeeae0 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe5ef7260 con 0x564fe5ee9970 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 10 client.0 _ll_register_callbacks cb 0x7f2f70005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49642/0 (socket says 172.21.15.8:49642) 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 -- 172.21.15.8:0/2877619525 learned_addr learned my addr 172.21.15.8:0/2877619525 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5efa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f877fe640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 -- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 msgr2=0x564fe5efa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5efa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 -- 172.21.15.8:0/2877619525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 msgr2=0x564fe5ef69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 -- 172.21.15.8:0/2877619525 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8819fb70 con 0x564fe5ee8f80 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f86ffd640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef9e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2f78003a30 tx=0x7f2f780cbaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1fd929a155b51b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f66ffd640 1 -- 172.21.15.8:0/2877619525 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7809eed0 con 0x564fe5ee8f80 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f66ffd640 1 -- 172.21.15.8:0/2877619525 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f2f7809a1d0 con 0x564fe5ee8f80 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fe5f01040 con 0x564fe5ee8f80 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.735+0000 7f2f66ffd640 1 -- 172.21.15.8:0/2877619525 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f7809a390 con 0x564fe5ee8f80 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 10 client.15219 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:05.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564fe5e11780 con 0x564fe5ee8f80 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f66ffd640 1 -- 172.21.15.8:0/2877619525 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564fe5f01040 con 0x564fe5ee8f80 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f66ffd640 1 -- 172.21.15.8:0/2877619525 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f7801f060 con 0x564fe5ee8f80 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 10 client.15219 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 10 client.15219 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 20 client.15219 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5f5f590 0x564fe5f5bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 4 client.15219 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f2f70005500 con 0x564fe5f5f590 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.736+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5f5f590 0x564fe5f5bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f66ffd640 10 client.15219 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f7ffff640 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5f5f590 0x564fe5f5bb10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e64cf842f54dcabc server_cookie=38946e8f4f7c2151 in_seq=0 out_seq=0 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f66ffd640 1 -- 172.21.15.8:0/2877619525 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f2f70005500 con 0x564fe5f5f590 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f66ffd640 10 client.15219 handle_command_reply: tid=0 2024-02-19T16:22:05.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 client.15219 shutdown 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 msgr2=0x564fe5ef9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef9e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2f78003a30 tx=0x7f2f780cbaf0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 shutdown_connections 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x564fe5f5f590 0x564fe5f5bb10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564fe5eeeae0 0x564fe5ef69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564fe5ee9970 0x564fe5efa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 --2- 172.21.15.8:0/2877619525 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564fe5ee8f80 0x564fe5ef9e80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f66ffd640 0 client.15219 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 >> 172.21.15.8:0/2877619525 conn(0x564fe5ee44f0 msgr2=0x564fe5edf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 shutdown_connections 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 1 -- 172.21.15.8:0/2877619525 wait complete. 2024-02-19T16:22:05.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.737+0000 7f2f90582740 20 client.15219 trim_cache size 0 max 16384 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 2, 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24475, 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:22:05.752 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 84.938051963000007, 2024-02-19T16:22:05.753 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 23, 2024-02-19T16:22:05.753 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 93, 2024-02-19T16:22:05.753 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 91, 2024-02-19T16:22:05.753 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 122.35865876299999 2024-02-19T16:22:05.753 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:05.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f740a20c0 msgr2=0x7f2f740a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f740a20c0 0x7f2f740a4580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x564fe5eea1d0 tx=0x7f2f800326b0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 msgr2=0x7f2f88197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 0x7f2f88197560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800e9c0 tx=0x7f2f7800ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:05.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 shutdown_connections 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2f740a20c0 0x7f2f740a4580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f881030b0 0x7f2f8819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f881026f0 0x7f2f88197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 --2- 172.21.15.8:0/1199890597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f88101500 0x7f2f88197560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 >> 172.21.15.8:0/1199890597 conn(0x7f2f880fca90 msgr2=0x7f2f88105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 shutdown_connections 2024-02-19T16:22:05.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.738+0000 7f2f8d8d4640 1 -- 172.21.15.8:0/1199890597 wait complete. 2024-02-19T16:22:05.755 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-19T16:22:05.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/819375312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:05.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3301139706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:05.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:05 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2533881922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 -- 172.21.15.8:0/3609027423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40fff30 msgr2=0x7f95c4109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3609027423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40fff30 0x7f95c4109ae0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009ca0 tx=0x7f95b402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 -- 172.21.15.8:0/3609027423 shutdown_connections 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3609027423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40fff30 0x7f95c4109ae0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3609027423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95c40ff570 0x7f95c40ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3609027423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102b00 0x7f95c4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 -- 172.21.15.8:0/3609027423 >> 172.21.15.8:0/3609027423 conn(0x7f95c40fb0f0 msgr2=0x7f95c40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 -- 172.21.15.8:0/3609027423 shutdown_connections 2024-02-19T16:22:05.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 -- 172.21.15.8:0/3609027423 wait complete. 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 Processor -- start 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.858+0000 7f95c95b5640 1 -- start start 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 0x7f95c4102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95c40fff30 0x7f95c4100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102b00 0x7f95c4100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c41014e0 con 0x7f95c40ff570 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4101650 con 0x7f95c4102b00 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c41a15e0 con 0x7f95c40fff30 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 0x7f95c4102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 0x7f95c4102200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49664/0 (socket says 172.21.15.8:49664) 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3745649962 learned_addr learned my addr 172.21.15.8:0/3745649962 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c37fe640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102b00 0x7f95c4100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c27fc640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95c40fff30 0x7f95c4100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3745649962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95c40fff30 msgr2=0x7f95c4100840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95c40fff30 0x7f95c4100840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3745649962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102b00 msgr2=0x7f95c4100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102b00 0x7f95c4100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3745649962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b4009900 con 0x7f95c40ff570 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 0x7f95c4102200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95bbfff640 1 -- 172.21.15.8:0/3745649962 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc009700 con 0x7f95c40ff570 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c41a1890 con 0x7f95c40ff570 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95bbfff640 1 -- 172.21.15.8:0/3745649962 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95bc00cd80 con 0x7f95c40ff570 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95bbfff640 1 -- 172.21.15.8:0/3745649962 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc010690 con 0x7f95c40ff570 2024-02-19T16:22:05.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.859+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41a1df0 con 0x7f95c40ff570 2024-02-19T16:22:05.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.861+0000 7f95bbfff640 1 -- 172.21.15.8:0/3745649962 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f95bc003710 con 0x7f95c40ff570 2024-02-19T16:22:05.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 Processor -- start 2024-02-19T16:22:05.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 -- start start 2024-02-19T16:22:05.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 0x5646311c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56463117ecd0 con 0x5646311c3970 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564631131f20 con 0x5646311c2f80 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95cc190740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56463112f450 con 0x5646311c8ae0 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95bbfff640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95b00a20c0 0x7f95b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 0x5646311c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 0x5646311c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49674/0 (socket says 172.21.15.8:49674) 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95c2ffd640 1 -- 172.21.15.8:0/1084892128 learned_addr learned my addr 172.21.15.8:0/1084892128 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95c27fc640 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95c37fe640 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.862+0000 7f95c37fe640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95b00a20c0 0x7f95b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95bbfff640 1 -- 172.21.15.8:0/3745649962 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f95bc014070 con 0x7f95c40ff570 2024-02-19T16:22:05.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c27fc640 1 -- 172.21.15.8:0/1084892128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 msgr2=0x5646311c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c27fc640 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c27fc640 1 -- 172.21.15.8:0/1084892128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 msgr2=0x5646311c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c27fc640 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 0x5646311c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c27fc640 1 -- 172.21.15.8:0/1084892128 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646310dff20 con 0x5646311c8ae0 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 0x5646311c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c37fe640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95b00a20c0 0x7f95b00a4580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f95b4005e30 tx=0x7f95b40326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95c27fc640 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311caf70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009bf0 tx=0x7f95ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e593fad719557e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95b97fa640 1 -- 172.21.15.8:0/1084892128 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004050 con 0x5646311c8ae0 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95b97fa640 1 -- 172.21.15.8:0/1084892128 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95ac005500 con 0x5646311c8ae0 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.863+0000 7f95b97fa640 1 -- 172.21.15.8:0/1084892128 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004c30 con 0x5646311c8ae0 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- 172.21.15.8:0/1084892128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 msgr2=0x5646311caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311caf70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009bf0 tx=0x7f95ac011810 comp rx=0 tx=0).stop 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- 172.21.15.8:0/1084892128 shutdown_connections 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311caf70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c3970 0x5646311c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- 172.21.15.8:0/1084892128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- 172.21.15.8:0/1084892128 >> 172.21.15.8:0/1084892128 conn(0x5646311be4f0 msgr2=0x5646311be8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- 172.21.15.8:0/1084892128 shutdown_connections 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- 172.21.15.8:0/1084892128 wait complete. 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- start start 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 Processor -- start 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x5646311d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56463112ac30 con 0x5646311c8ae0 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646310cfd30 con 0x5646311c2f80 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.864+0000 7f95cc190740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564630dfe310 con 0x5646311c3970 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95cc190740 10 client.0 _ll_register_callbacks cb 0x5646310d7d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:05.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x5646311d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x5646311d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49682/0 (socket says 172.21.15.8:49682) 2024-02-19T16:22:05.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 -- 172.21.15.8:0/217810769 learned_addr learned my addr 172.21.15.8:0/217810769 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c27fc640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 -- 172.21.15.8:0/217810769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 msgr2=0x5646311cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 -- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 msgr2=0x5646311cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 -- 172.21.15.8:0/217810769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac009850 con 0x5646311c8ae0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95c37fe640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x5646311d4670 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009ca0 tx=0x7f95b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32417c7c09e035a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95aa7fc640 1 -- 172.21.15.8:0/217810769 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4003b80 con 0x5646311c8ae0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95aa7fc640 1 -- 172.21.15.8:0/217810769 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f95b4003ce0 con 0x5646311c8ae0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564630e2a6c0 con 0x5646311c8ae0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95aa7fc640 1 -- 172.21.15.8:0/217810769 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95b4003ea0 con 0x5646311c8ae0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95cc190740 10 client.15231 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.865+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5646311292b0 con 0x5646311c8ae0 2024-02-19T16:22:05.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95aa7fc640 1 -- 172.21.15.8:0/217810769 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95b405b020 con 0x5646311c8ae0 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95aa7fc640 1 -- 172.21.15.8:0/217810769 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95b4058070 con 0x5646311c8ae0 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95cc190740 10 client.15231 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95cc190740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95cc190740 20 client.15231 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95cc190740 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5646311d9b00 0x5646311d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95cc190740 4 client.15231 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564630e7b270 con 0x5646311d9b00 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5646311d9b00 0x5646311d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95aa7fc640 10 client.15231 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.866+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5646311d9b00 0x5646311d8d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7197a28c6e049ee0 server_cookie=16a5031cfc4b1643 in_seq=0 out_seq=0 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95aa7fc640 1 -- 172.21.15.8:0/217810769 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564630e7b270 con 0x5646311d9b00 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95aa7fc640 10 client.15231 handle_command_reply: tid=0 2024-02-19T16:22:05.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 client.15231 shutdown 2024-02-19T16:22:05.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 msgr2=0x5646311d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x5646311d4670 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009ca0 tx=0x7f95b40095c0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 shutdown_connections 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5646311d9b00 0x5646311d8d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x5646311d4670 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95aa7fc640 0 client.15231 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 --2- 172.21.15.8:0/217810769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 >> 172.21.15.8:0/217810769 conn(0x5646311be4f0 msgr2=0x5646311bd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 shutdown_connections 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 1 -- 172.21.15.8:0/217810769 wait complete. 2024-02-19T16:22:05.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.867+0000 7f95cc190740 20 client.15231 trim_cache size 0 max 16384 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 Processor -- start 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 -- start start 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 0x5646311caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c3970 0x5646311d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646310b0490 con 0x5646311c2f80 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646311cb630 con 0x5646311c3970 2024-02-19T16:22:05.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.884+0000 7f95cc190740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564630e7b270 con 0x5646311c8ae0 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 0x5646311caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 0x5646311caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49688/0 (socket says 172.21.15.8:49688) 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3767032116 learned_addr learned my addr 172.21.15.8:0/3767032116 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c27fc640 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c3970 0x5646311d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3767032116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 msgr2=0x5646311cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3767032116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c3970 msgr2=0x5646311d4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c3970 0x5646311d4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 -- 172.21.15.8:0/3767032116 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646310c5a20 con 0x5646311c2f80 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 0x5646311caed0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f95bc097100 tx=0x7f95bc09abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1e06835406efd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95b8ff9640 1 -- 172.21.15.8:0/3767032116 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0a3490 con 0x5646311c2f80 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95b8ff9640 1 -- 172.21.15.8:0/3767032116 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95bc09ee40 con 0x5646311c2f80 2024-02-19T16:22:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.885+0000 7f95b8ff9640 1 -- 172.21.15.8:0/3767032116 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc099080 con 0x5646311c2f80 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- 172.21.15.8:0/3767032116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 msgr2=0x5646311caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 0x5646311caed0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f95bc097100 tx=0x7f95bc09abf0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- 172.21.15.8:0/3767032116 shutdown_connections 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c8ae0 0x5646311cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c3970 0x5646311d4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- 172.21.15.8:0/3767032116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c2f80 0x5646311caed0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- 172.21.15.8:0/3767032116 >> 172.21.15.8:0/3767032116 conn(0x5646311be4f0 msgr2=0x5646311c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- 172.21.15.8:0/3767032116 shutdown_connections 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- 172.21.15.8:0/3767032116 wait complete. 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- start start 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 Processor -- start 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x564631234b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646311d1df0 con 0x5646311c8ae0 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646311d1f60 con 0x5646311c2f80 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646312350a0 con 0x5646311c3970 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95cc190740 10 client.0 _ll_register_callbacks cb 0x7f95ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.886+0000 7f95c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x564631234b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c27fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:39862/0 (socket says 172.21.15.8:39862) 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x564631234b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49702/0 (socket says 172.21.15.8:49702) 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c2ffd640 1 -- 172.21.15.8:0/944851002 learned_addr learned my addr 172.21.15.8:0/944851002 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 -- 172.21.15.8:0/944851002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 msgr2=0x5646311d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 -- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 msgr2=0x5646311d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:05.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 -- 172.21.15.8:0/944851002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41a1890 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95c37fe640 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x564631234b60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95b4004770 tx=0x7f95b4003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7e32cbae9c96d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95aa7fc640 1 -- 172.21.15.8:0/944851002 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4032ec0 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95aa7fc640 1 -- 172.21.15.8:0/944851002 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f95b405c080 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646311da4d0 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95aa7fc640 1 -- 172.21.15.8:0/944851002 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95b4004890 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95cc190740 10 client.15237 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5646310eb780 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.887+0000 7f95aa7fc640 1 -- 172.21.15.8:0/944851002 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5646311da4d0 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95aa7fc640 1 -- 172.21.15.8:0/944851002 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95b4058040 con 0x5646311c8ae0 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 10 client.15237 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-19T16:22:05.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 20 client.15237 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x56463123a0d0 0x564631236550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 4 client.15237 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f95ac005500 con 0x56463123a0d0 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x56463123a0d0 0x564631236550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95aa7fc640 10 client.15237 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95c2ffd640 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x56463123a0d0 0x564631236550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d42f2f5efae4b899 server_cookie=eb2a36cf14438831 in_seq=0 out_seq=0 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95aa7fc640 1 -- 172.21.15.8:0/944851002 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f95ac005500 con 0x56463123a0d0 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95aa7fc640 10 client.15237 handle_command_reply: tid=0 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 client.15237 shutdown 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 msgr2=0x564631234b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x564631234b60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95b4004770 tx=0x7f95b4003e30 comp rx=0 tx=0).stop 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 shutdown_connections 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x56463123a0d0 0x564631236550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5646311c8ae0 0x564631234b60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646311c3970 0x5646311d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 --2- 172.21.15.8:0/944851002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646311c2f80 0x5646311d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95aa7fc640 0 client.15237 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.888+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 >> 172.21.15.8:0/944851002 conn(0x5646311be4f0 msgr2=0x5646311cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 shutdown_connections 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95cc190740 1 -- 172.21.15.8:0/944851002 wait complete. 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95cc190740 20 client.15237 trim_cache size 0 max 16384 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:05.899 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 1, 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24488, 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 86.097485073000001, 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 23, 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 93, 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 91, 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 108.38809663799999 2024-02-19T16:22:05.900 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95b00a20c0 msgr2=0x7f95b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95b00a20c0 0x7f95b00a4580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f95b4005e30 tx=0x7f95b40326b0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 msgr2=0x7f95c4102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 0x7f95c4102200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 shutdown_connections 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95b00a20c0 0x7f95b00a4580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95c4102b00 0x7f95c4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95c40fff30 0x7f95c4100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 --2- 172.21.15.8:0/3745649962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95c40ff570 0x7f95c4102200 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 >> 172.21.15.8:0/3745649962 conn(0x7f95c40fb0f0 msgr2=0x7f95c4105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 shutdown_connections 2024-02-19T16:22:05.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:05.889+0000 7f95c95b5640 1 -- 172.21.15.8:0/3745649962 wait complete. 2024-02-19T16:22:05.903 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1904547205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604101510 msgr2=0x7fa604101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1904547205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604101510 0x7fa604101930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1904547205 shutdown_connections 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1904547205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6041030c0 0x7fa604109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1904547205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa604102700 0x7fa604102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1904547205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604101510 0x7fa604101930 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1904547205 >> 172.21.15.8:0/1904547205 conn(0x7fa6040fca80 msgr2=0x7fa6040feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1904547205 shutdown_connections 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1904547205 wait complete. 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 Processor -- start 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.007+0000 7fa60ab8c640 1 -- start start 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa60ab8c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 0x7fa6041975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa60ab8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604102700 0x7fa604197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa60ab8c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6041030c0 0x7fa60419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa60ab8c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60419f0e0 con 0x7fa604101510 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa60ab8c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60419f250 con 0x7fa6041030c0 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa60ab8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60419f410 con 0x7fa604102700 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 0x7fa6041975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 0x7fa6041975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49728/0 (socket says 172.21.15.8:49728) 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 -- 172.21.15.8:0/1913838478 learned_addr learned my addr 172.21.15.8:0/1913838478 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:06.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa603fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604102700 0x7fa604197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa609129640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6041030c0 0x7fa60419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 -- 172.21.15.8:0/1913838478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604102700 msgr2=0x7fa604197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604102700 0x7fa604197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 -- 172.21.15.8:0/1913838478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6041030c0 msgr2=0x7fa60419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6041030c0 0x7fa60419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 -- 172.21.15.8:0/1913838478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc009840 con 0x7fa604101510 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa609129640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6041030c0 0x7fa60419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa603fff640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604102700 0x7fa604197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa608928640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 0x7fa6041975f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.008+0000 7fa601ffb640 1 -- 172.21.15.8:0/1913838478 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80043d0 con 0x7fa604101510 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.009+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f8009560 con 0x7fa604101510 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.009+0000 7fa601ffb640 1 -- 172.21.15.8:0/1913838478 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80052d0 con 0x7fa604101510 2024-02-19T16:22:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.009+0000 7fa601ffb640 1 -- 172.21.15.8:0/1913838478 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8003cd0 con 0x7fa604101510 2024-02-19T16:22:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.009+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60419fa10 con 0x7fa604101510 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 Processor -- start 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa601ffb640 1 -- 172.21.15.8:0/1913838478 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fa5f8005440 con 0x7fa604101510 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 -- start start 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b0f80 0x55b4819b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b1970 0x55b4819b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48196ccd0 con 0x55b4819b6ae0 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48191ff20 con 0x55b4819b0f80 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa60d76a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48191d450 con 0x55b4819b1970 2024-02-19T16:22:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa603fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa609129640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b0f80 0x55b4819b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa603fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819b8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49742/0 (socket says 172.21.15.8:49742) 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa603fff640 1 -- 172.21.15.8:0/871939898 learned_addr learned my addr 172.21.15.8:0/871939898 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.011+0000 7fa608928640 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b1970 0x55b4819b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa601ffb640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa5f40a20c0 0x7fa5f40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 -- 172.21.15.8:0/871939898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b1970 msgr2=0x55b4819b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b1970 0x55b4819b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 -- 172.21.15.8:0/871939898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 msgr2=0x55b4819b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 -- 172.21.15.8:0/871939898 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4818cdf20 con 0x55b4819b0f80 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa601ffb640 1 -- 172.21.15.8:0/1913838478 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fa5f80f0110 con 0x7fa604101510 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa5f40a20c0 0x7fa5f40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa609129640 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b0f80 0x55b4819b13a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e860 tx=0x7fa5fc00ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f7d049bd386da1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa5eeffd640 1 -- 172.21.15.8:0/871939898 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc01e070 con 0x55b4819b0f80 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa5eeffd640 1 -- 172.21.15.8:0/871939898 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5fc0198e0 con 0x55b4819b0f80 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.012+0000 7fa5eeffd640 1 -- 172.21.15.8:0/871939898 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc018a00 con 0x55b4819b0f80 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 -- 172.21.15.8:0/871939898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b0f80 msgr2=0x55b4819b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa609129640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa5f40a20c0 0x7fa5f40a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc01d000 tx=0x7fa5fc018c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b0f80 0x55b4819b13a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e860 tx=0x7fa5fc00ed20 comp rx=0 tx=0).stop 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 -- 172.21.15.8:0/871939898 shutdown_connections 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b1970 0x55b4819b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 --2- 172.21.15.8:0/871939898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b0f80 0x55b4819b13a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 -- 172.21.15.8:0/871939898 >> 172.21.15.8:0/871939898 conn(0x55b4819ac4f0 msgr2=0x55b4819ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 -- 172.21.15.8:0/871939898 shutdown_connections 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 -- 172.21.15.8:0/871939898 wait complete. 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 -- start start 2024-02-19T16:22:06.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.013+0000 7fa60d76a740 1 Processor -- start 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b481918c30 con 0x55b4819b1970 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4818bdd30 con 0x55b4819b6ae0 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4815ec310 con 0x55b4819b0f80 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 10 client.0 _ll_register_callbacks cb 0x55b4818c5d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49754/0 (socket says 172.21.15.8:49754) 2024-02-19T16:22:06.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 -- 172.21.15.8:0/1685705514 learned_addr learned my addr 172.21.15.8:0/1685705514 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa608928640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa609129640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 -- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 msgr2=0x55b4819bb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 -- 172.21.15.8:0/1685705514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 msgr2=0x55b4819c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 -- 172.21.15.8:0/1685705514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc00e540 con 0x55b4819b1970 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa603fff640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bb5f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0012d20 tx=0x7fa5f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82ae1aaede32aa7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa5e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa5e3fff640 1 -- 172.21.15.8:0/1685705514 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0017070 con 0x55b4819b1970 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa5e3fff640 1 -- 172.21.15.8:0/1685705514 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fa5f0012110 con 0x55b4819b1970 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.014+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4816186c0 con 0x55b4819b1970 2024-02-19T16:22:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa5e3fff640 1 -- 172.21.15.8:0/1685705514 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5f00122d0 con 0x55b4819b1970 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 10 client.15249 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b4819172b0 con 0x55b4819b1970 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa5e3fff640 1 -- 172.21.15.8:0/1685705514 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b4816186c0 con 0x55b4819b1970 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa5e3fff640 1 -- 172.21.15.8:0/1685705514 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5f0010430 con 0x55b4819b1970 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 10 client.15249 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 20 client.15249 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b4819c7ae0 0x55b4819c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 4 client.15249 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b481669270 con 0x55b4819c7ae0 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.015+0000 7fa608928640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b4819c7ae0 0x55b4819c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.016+0000 7fa5e3fff640 10 client.15249 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.016+0000 7fa608928640 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b4819c7ae0 0x55b4819c6d60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc50f113fac64dd9 server_cookie=d68a3a653fda3e65 in_seq=0 out_seq=0 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa5e3fff640 1 -- 172.21.15.8:0/1685705514 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b481669270 con 0x55b4819c7ae0 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa5e3fff640 10 client.15249 handle_command_reply: tid=0 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 client.15249 shutdown 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 msgr2=0x55b4819bb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bb5f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0012d20 tx=0x7fa5f000f790 comp rx=0 tx=0).stop 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 shutdown_connections 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bb5f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b4819c7ae0 0x55b4819c6d60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 --2- 172.21.15.8:0/1685705514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa5e3fff640 0 client.15249 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 >> 172.21.15.8:0/1685705514 conn(0x55b4819ac4f0 msgr2=0x55b4819ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 shutdown_connections 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 1 -- 172.21.15.8:0/1685705514 wait complete. 2024-02-19T16:22:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.017+0000 7fa60d76a740 20 client.15249 trim_cache size 0 max 16384 2024-02-19T16:22:06.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 Processor -- start 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 -- start start 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b1970 0x55b4819c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819bb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48189e490 con 0x55b4819b6ae0 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4816186f0 con 0x55b4819b1970 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa60d76a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b481911ad0 con 0x55b4819b0f80 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa609129640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819bb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa609129640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819bb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49760/0 (socket says 172.21.15.8:49760) 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa609129640 1 -- 172.21.15.8:0/3595972914 learned_addr learned my addr 172.21.15.8:0/3595972914 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa608928640 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.034+0000 7fa603fff640 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b1970 0x55b4819c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa609129640 1 -- 172.21.15.8:0/3595972914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 msgr2=0x55b4819bacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa609129640 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa609129640 1 -- 172.21.15.8:0/3595972914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b1970 msgr2=0x55b4819c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa609129640 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b1970 0x55b4819c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa609129640 1 -- 172.21.15.8:0/3595972914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4818b3a20 con 0x55b4819b6ae0 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa603fff640 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b1970 0x55b4819c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa609129640 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819bb200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e860 tx=0x7fa5fc028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0264bbb63d64d1c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa5ee7fc640 1 -- 172.21.15.8:0/3595972914 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc01e070 con 0x55b4819b6ae0 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa5ee7fc640 1 -- 172.21.15.8:0/3595972914 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5fc01f040 con 0x55b4819b6ae0 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa5ee7fc640 1 -- 172.21.15.8:0/3595972914 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc019cb0 con 0x55b4819b6ae0 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 -- 172.21.15.8:0/3595972914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 msgr2=0x55b4819bb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819bb200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e860 tx=0x7fa5fc028000 comp rx=0 tx=0).stop 2024-02-19T16:22:06.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 -- 172.21.15.8:0/3595972914 shutdown_connections 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b6ae0 0x55b4819bb200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b1970 0x55b4819c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 --2- 172.21.15.8:0/3595972914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 -- 172.21.15.8:0/3595972914 >> 172.21.15.8:0/3595972914 conn(0x55b4819ac4f0 msgr2=0x55b4819ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.035+0000 7fa60d76a740 1 -- 172.21.15.8:0/3595972914 shutdown_connections 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 -- 172.21.15.8:0/3595972914 wait complete. 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 -- start start 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 Processor -- start 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4819bfd30 con 0x55b4819b1970 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b481a22b70 con 0x55b4819b6ae0 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b481a22d30 con 0x55b4819b0f80 2024-02-19T16:22:06.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa60d76a740 10 client.0 _ll_register_callbacks cb 0x7fa5f0009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa603fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa603fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49770/0 (socket says 172.21.15.8:49770) 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa603fff640 1 -- 172.21.15.8:0/2878333242 learned_addr learned my addr 172.21.15.8:0/2878333242 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa608928640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa609129640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.036+0000 7fa603fff640 1 -- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 msgr2=0x55b4819c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa603fff640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa603fff640 1 -- 172.21.15.8:0/2878333242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 msgr2=0x55b4819bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa603fff640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa603fff640 1 -- 172.21.15.8:0/2878333242 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f0021040 con 0x55b4819b1970 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa603fff640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bf0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0010860 tx=0x7fa5f0010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f400fdfd0a49875 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa5e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:06.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa5e3fff640 1 -- 172.21.15.8:0/2878333242 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0017040 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa5e3fff640 1 -- 172.21.15.8:0/2878333242 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fa5f0010be0 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5fc0198e0 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa5e3fff640 1 -- 172.21.15.8:0/2878333242 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5f0010dc0 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 10 client.15255 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b4818cdf20 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa5e3fff640 1 -- 172.21.15.8:0/2878333242 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5fc0198e0 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa5e3fff640 1 -- 172.21.15.8:0/2878333242 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5f0004960 con 0x55b4819b1970 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 10 client.15255 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 10 client.15255 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 20 client.15255 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b481a28090 0x55b481a244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 4 client.15255 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.037+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b481655610 con 0x55b481a28090 2024-02-19T16:22:06.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa608928640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b481a28090 0x55b481a244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa5e3fff640 10 client.15255 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa608928640 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b481a28090 0x55b481a244f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f83f9c1052effe server_cookie=8ea819684199edef in_seq=0 out_seq=0 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa5e3fff640 1 -- 172.21.15.8:0/2878333242 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b481655610 con 0x55b481a28090 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa5e3fff640 10 client.15255 handle_command_reply: tid=0 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 client.15255 shutdown 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 msgr2=0x55b4819bf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bf0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0010860 tx=0x7fa5f0010660 comp rx=0 tx=0).stop 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 shutdown_connections 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4819b6ae0 0x55b4819bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b4819b1970 0x55b4819bf0b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b481a28090 0x55b481a244f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 --2- 172.21.15.8:0/2878333242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4819b0f80 0x55b4819c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa5e3fff640 0 client.15255 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:06.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 >> 172.21.15.8:0/2878333242 conn(0x55b4819ac4f0 msgr2=0x55b4819bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 shutdown_connections 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 1 -- 172.21.15.8:0/2878333242 wait complete. 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.038+0000 7fa60d76a740 20 client.15255 trim_cache size 0 max 16384 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24502, 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:22:06.047 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:22:06.048 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 104.31119384599999, 2024-02-19T16:22:06.048 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 23, 2024-02-19T16:22:06.048 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 93, 2024-02-19T16:22:06.048 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 91, 2024-02-19T16:22:06.048 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 117.059103329 2024-02-19T16:22:06.048 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:06.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa5f40a20c0 msgr2=0x7fa5f40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa5f40a20c0 0x7fa5f40a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc01d000 tx=0x7fa5fc018c40 comp rx=0 tx=0).stop 2024-02-19T16:22:06.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 msgr2=0x7fa6041975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 0x7fa6041975f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f8005e50 comp rx=0 tx=0).stop 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 shutdown_connections 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa5f40a20c0 0x7fa5f40a4580 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6041030c0 0x7fa60419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa604102700 0x7fa604197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 --2- 172.21.15.8:0/1913838478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa604101510 0x7fa6041975f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 >> 172.21.15.8:0/1913838478 conn(0x7fa6040fca80 msgr2=0x7fa604105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 shutdown_connections 2024-02-19T16:22:06.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:06.039+0000 7fa60ab8c640 1 -- 172.21.15.8:0/1913838478 wait complete. 2024-02-19T16:22:06.051 INFO:tasks.cephadm:Setup complete, yielding 2024-02-19T16:22:06.051 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T16:22:06.062 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:22:06.062 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph orch status' 2024-02-19T16:22:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:06.569 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:06 smithi008 ceph-mon[32573]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:22:06.569 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:06 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3743265001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:06.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:06 smithi156 ceph-mon[36502]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:22:06.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:06 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3743265001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:06.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:06 smithi191 ceph-mon[37203]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-19T16:22:06.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:06 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3743265001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 -- 172.21.15.8:0/2025215726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0102010 msgr2=0x7f17e0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 --2- 172.21.15.8:0/2025215726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0102010 0x7f17e0102430 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f17d00099c0 tx=0x7f17d0009cd0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 -- 172.21.15.8:0/2025215726 shutdown_connections 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 --2- 172.21.15.8:0/2025215726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17e0103bb0 0x7f17e010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 --2- 172.21.15.8:0/2025215726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0102010 0x7f17e0102430 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 --2- 172.21.15.8:0/2025215726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0103210 0x7f17e0103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 -- 172.21.15.8:0/2025215726 >> 172.21.15.8:0/2025215726 conn(0x7f17e00fd5f0 msgr2=0x7f17e00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:08.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.113+0000 7f17e532e700 1 -- 172.21.15.8:0/2025215726 shutdown_connections 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 -- 172.21.15.8:0/2025215726 wait complete. 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 Processor -- start 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 -- start start 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 0x7f17e0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17e0103210 0x7f17e00743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 0x7f17e0074910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0075090 con 0x7f17e0103bb0 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0075200 con 0x7f17e0103210 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.114+0000 7f17e532e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e01a2040 con 0x7f17e0102010 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 0x7f17e0074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17deffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 0x7f17e0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 0x7f17e0074910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49776/0 (socket says 172.21.15.8:49776) 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 -- 172.21.15.8:0/2246180550 learned_addr learned my addr 172.21.15.8:0/2246180550 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:08.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17deffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 0x7f17e0075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:55888/0 (socket says 172.21.15.8:55888) 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17de7fc700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17e0103210 0x7f17e00743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 -- 172.21.15.8:0/2246180550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 msgr2=0x7f17e0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 0x7f17e0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 -- 172.21.15.8:0/2246180550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17e0103210 msgr2=0x7f17e00743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17e0103210 0x7f17e00743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 -- 172.21.15.8:0/2246180550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d8009c40 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17deffd700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 0x7f17e0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17df7fe700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 0x7f17e0074910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c370 tx=0x7f17d800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8017070 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17d800f040 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d00095f0 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d800e050 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.115+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e01a25e0 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.116+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e01094f0 con 0x7f17e0103bb0 2024-02-19T16:22:08.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.117+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f17d800cb60 con 0x7f17e0103bb0 2024-02-19T16:22:08.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.118+0000 7f17c7fff700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f17c80a2150 0x7f17c80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:08.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.118+0000 7f17deffd700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f17c80a2150 0x7f17c80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:08.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.118+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f17d80cd900 con 0x7f17e0103bb0 2024-02-19T16:22:08.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.119+0000 7f17deffd700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f17c80a2150 0x7f17c80a4610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006010 tx=0x7f17d000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:08.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.123+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f17d8096590 con 0x7f17e0103bb0 2024-02-19T16:22:08.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.346+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17e0062420 con 0x7f17c80a2150 2024-02-19T16:22:08.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.347+0000 7f17c7fff700 1 -- 172.21.15.8:0/2246180550 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f17e0062420 con 0x7f17c80a2150 2024-02-19T16:22:08.347 INFO:teuthology.orchestra.run.smithi008.stdout:Backend: cephadm 2024-02-19T16:22:08.348 INFO:teuthology.orchestra.run.smithi008.stdout:Available: Yes 2024-02-19T16:22:08.348 INFO:teuthology.orchestra.run.smithi008.stdout:Paused: No 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f17c80a2150 msgr2=0x7f17c80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f17c80a2150 0x7f17c80a4610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006010 tx=0x7f17d000b540 comp rx=0 tx=0).stop 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 msgr2=0x7f17e0074910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 0x7f17e0074910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c370 tx=0x7f17d800c680 comp rx=0 tx=0).stop 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 shutdown_connections 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f17c80a2150 0x7f17c80a4610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17e0103bb0 0x7f17e0074910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17e0102010 0x7f17e0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 --2- 172.21.15.8:0/2246180550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17e0103210 0x7f17e00743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 >> 172.21.15.8:0/2246180550 conn(0x7f17e00fd5f0 msgr2=0x7f17e0106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 shutdown_connections 2024-02-19T16:22:08.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:08.349+0000 7f17e532e700 1 -- 172.21.15.8:0/2246180550 wait complete. 2024-02-19T16:22:08.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:08 smithi156 ceph-mon[36502]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-02-19T16:22:08.738 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:08 smithi008 ceph-mon[32573]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-02-19T16:22:08.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:08 smithi191 ceph-mon[37203]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-02-19T16:22:09.068 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-19T16:22:09.312 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:09.559 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:09 smithi008 ceph-mon[32573]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:09.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:09 smithi156 ceph-mon[36502]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:09.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:09 smithi191 ceph-mon[37203]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:10 smithi008 ceph-mon[32573]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:10.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:10 smithi156 ceph-mon[36502]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:10.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:10 smithi191 ceph-mon[37203]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:11.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 -- 172.21.15.8:0/2936942341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298105420 msgr2=0x7f9298107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:11.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 --2- 172.21.15.8:0/2936942341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298105420 0x7f9298107810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9288009990 tx=0x7f9288009ca0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 -- 172.21.15.8:0/2936942341 shutdown_connections 2024-02-19T16:22:11.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 --2- 172.21.15.8:0/2936942341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298107d50 0x7f929810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 --2- 172.21.15.8:0/2936942341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298102af0 0x7f9298104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 --2- 172.21.15.8:0/2936942341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298105420 0x7f9298107810 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 -- 172.21.15.8:0/2936942341 >> 172.21.15.8:0/2936942341 conn(0x7f92980fc4a0 msgr2=0x7f92980fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 -- 172.21.15.8:0/2936942341 shutdown_connections 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 -- 172.21.15.8:0/2936942341 wait complete. 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 Processor -- start 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.134+0000 7f929f6a1700 1 -- start start 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929f6a1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298102af0 0x7f9298197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929f6a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298105420 0x7f9298198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929f6a1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 0x7f929819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929f6a1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929819fb70 con 0x7f9298107d50 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929f6a1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929819fcb0 con 0x7f9298102af0 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929f6a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929819fe40 con 0x7f9298105420 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 0x7f929819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:11.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 0x7f929819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49790/0 (socket says 172.21.15.8:49790) 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 -- 172.21.15.8:0/1133617107 learned_addr learned my addr 172.21.15.8:0/1133617107 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929cc3c700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298105420 0x7f9298198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929d43d700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298102af0 0x7f9298197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 -- 172.21.15.8:0/1133617107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298105420 msgr2=0x7f9298198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298105420 0x7f9298198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 -- 172.21.15.8:0/1133617107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298102af0 msgr2=0x7f9298197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298102af0 0x7f9298197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 -- 172.21.15.8:0/1133617107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9290009c40 con 0x7f9298107d50 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929cc3c700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298105420 0x7f9298198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929d43d700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298102af0 0x7f9298197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.135+0000 7f929dc3e700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 0x7f929819f5a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f929000c140 tx=0x7f929000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.136+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290017070 con 0x7f9298107d50 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.136+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f929000f040 con 0x7f9298107d50 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.136+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92880095f0 con 0x7f9298107d50 2024-02-19T16:22:11.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.136+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000e050 con 0x7f9298107d50 2024-02-19T16:22:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.136+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92981a03e0 con 0x7f9298107d50 2024-02-19T16:22:11.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.136+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92980fe020 con 0x7f9298107d50 2024-02-19T16:22:11.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.138+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f9290003a00 con 0x7f9298107d50 2024-02-19T16:22:11.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.138+0000 7f928e7fc700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f92840a2150 0x7f92840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:11.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.138+0000 7f929d43d700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f92840a2150 0x7f92840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:11.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.139+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f92900cdf60 con 0x7f9298107d50 2024-02-19T16:22:11.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.139+0000 7f929d43d700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f92840a2150 0x7f92840a4610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9294011490 tx=0x7f9294009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:11.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.142+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9290096bf0 con 0x7f9298107d50 2024-02-19T16:22:11.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.356+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9298062420 con 0x7f92840a2150 2024-02-19T16:22:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.370+0000 7f928e7fc700 1 -- 172.21.15.8:0/1133617107 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f9298062420 con 0x7f92840a2150 2024-02-19T16:22:11.371 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T16:22:11.371 INFO:teuthology.orchestra.run.smithi008.stdout:mds.a smithi008 running (117s) 20s ago 116s 17.3M - 19.0.0-1451-g9f8908ee cec87a2a42a4 50d1bfbf92ce 2024-02-19T16:22:11.371 INFO:teuthology.orchestra.run.smithi008.stdout:mds.b smithi156 running (114s) 47s ago 113s 17.0M - 19.0.0-1451-g9f8908ee cec87a2a42a4 9c1f4ac98c1f 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.c smithi191 running (2m) 47s ago 2m 19.5M - 19.0.0-1451-g9f8908ee cec87a2a42a4 7fedf19021a3 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.d smithi008 running (2m) 20s ago 2m 13.8M - 19.0.0-1451-g9f8908ee cec87a2a42a4 2e861e0d5659 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.e smithi156 running (2m) 47s ago 2m 16.4M - 19.0.0-1451-g9f8908ee cec87a2a42a4 7f17d82d3cf7 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.f smithi191 running (2m) 47s ago 2m 17.5M - 19.0.0-1451-g9f8908ee cec87a2a42a4 8e976bff3738 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.g smithi008 running (119s) 20s ago 119s 15.9M - 19.0.0-1451-g9f8908ee cec87a2a42a4 7169860ae5ab 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.h smithi156 running (2m) 47s ago 2m 15.4M - 19.0.0-1451-g9f8908ee cec87a2a42a4 297efbd4b3c8 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.i smithi191 running (2m) 47s ago 2m 28.1M - 19.0.0-1451-g9f8908ee cec87a2a42a4 b6a981c3c635 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.j smithi008 running (2m) 20s ago 2m 14.0M - 19.0.0-1451-g9f8908ee cec87a2a42a4 664f851e55ed 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.k smithi156 running (2m) 47s ago 2m 14.3M - 19.0.0-1451-g9f8908ee cec87a2a42a4 9981358b9c47 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mds.l smithi191 running (2m) 47s ago 2m 17.8M - 19.0.0-1451-g9f8908ee cec87a2a42a4 ba5a4dcad170 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.x smithi008 *:9283,8765 running (9m) 20s ago 9m 513M - 19.0.0-1451-g9f8908ee cec87a2a42a4 8c5a5d54627c 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.y smithi156 *:8443,8765 running (7m) 47s ago 7m 452M - 19.0.0-1451-g9f8908ee cec87a2a42a4 71c335939279 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.z smithi191 *:8443,8765 running (7m) 47s ago 7m 452M - 19.0.0-1451-g9f8908ee cec87a2a42a4 36a6644c1d97 2024-02-19T16:22:11.372 INFO:teuthology.orchestra.run.smithi008.stdout:mon.a smithi008 running (9m) 20s ago 9m 50.0M 2048M 19.0.0-1451-g9f8908ee cec87a2a42a4 92f87135603b 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:mon.b smithi156 running (7m) 47s ago 7m 39.5M 2048M 19.0.0-1451-g9f8908ee cec87a2a42a4 134876071ffd 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:mon.c smithi191 running (8m) 47s ago 8m 38.6M 2048M 19.0.0-1451-g9f8908ee cec87a2a42a4 29df719b1c89 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (7m) 20s ago 7m 59.0M 4273M 19.0.0-1451-g9f8908ee cec87a2a42a4 e3c224778791 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi156 running (6m) 47s ago 6m 56.2M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 aa06416c100a 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi191 running (6m) 47s ago 6m 58.0M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 f96940a104cd 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi008 running (6m) 20s ago 6m 47.2M 4273M 19.0.0-1451-g9f8908ee cec87a2a42a4 f14561727a20 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi156 running (5m) 47s ago 5m 56.8M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 09cf8c743316 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi191 running (5m) 47s ago 5m 45.1M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 85c0f47f4ab8 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.6 smithi008 running (4m) 20s ago 4m 52.5M 4273M 19.0.0-1451-g9f8908ee cec87a2a42a4 a991015ef6b4 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.7 smithi156 running (4m) 47s ago 4m 45.1M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 ce88c1be1894 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.8 smithi191 running (4m) 47s ago 4m 45.9M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 0e13f5ec443c 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.9 smithi008 running (3m) 20s ago 3m 44.2M 4273M 19.0.0-1451-g9f8908ee cec87a2a42a4 e24c1bd9e1d4 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.10 smithi156 running (3m) 47s ago 3m 43.7M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 6d4563213b81 2024-02-19T16:22:11.373 INFO:teuthology.orchestra.run.smithi008.stdout:osd.11 smithi191 running (2m) 47s ago 2m 42.4M 4270M 19.0.0-1451-g9f8908ee cec87a2a42a4 0750298011cc 2024-02-19T16:22:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f92840a2150 msgr2=0x7f92840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f92840a2150 0x7f92840a4610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9294011490 tx=0x7f9294009260 comp rx=0 tx=0).stop 2024-02-19T16:22:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 msgr2=0x7f929819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 0x7f929819f5a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f929000c140 tx=0x7f929000c450 comp rx=0 tx=0).stop 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 shutdown_connections 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f92840a2150 0x7f92840a4610 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9298107d50 0x7f929819f5a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298102af0 0x7f9298197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 --2- 172.21.15.8:0/1133617107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9298105420 0x7f9298198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 >> 172.21.15.8:0/1133617107 conn(0x7f92980fc4a0 msgr2=0x7f9298103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.372+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 shutdown_connections 2024-02-19T16:22:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:11.373+0000 7f929f6a1700 1 -- 172.21.15.8:0/1133617107 wait complete. 2024-02-19T16:22:12.181 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-19T16:22:12.430 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:12.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:12 smithi008 ceph-mon[32573]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:12.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:12 smithi008 ceph-mon[32573]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:12 smithi156 ceph-mon[36502]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:12.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:12 smithi156 ceph-mon[36502]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:12.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:12 smithi191 ceph-mon[37203]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:12.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:12 smithi191 ceph-mon[37203]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:14.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2230245296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c103240 msgr2=0x7f5d8c1036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:14.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2230245296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c103240 0x7f5d8c1036a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d800099c0 tx=0x7f5d80009cd0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2230245296 shutdown_connections 2024-02-19T16:22:14.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2230245296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 0x7f5d8c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2230245296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d8c102040 0x7f5d8c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2230245296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c103240 0x7f5d8c1036a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2230245296 >> 172.21.15.8:0/2230245296 conn(0x7f5d8c0fd5e0 msgr2=0x7f5d8c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:14.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2230245296 shutdown_connections 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.238+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2230245296 wait complete. 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 Processor -- start 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 -- start start 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c102040 0x7f5d8c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d8c103240 0x7f5d8c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 0x7f5d8c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c19fad0 con 0x7f5d8c102040 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c19fc10 con 0x7f5d8c103be0 2024-02-19T16:22:14.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d93e2c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c19fda0 con 0x7f5d8c103240 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d91bc8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c102040 0x7f5d8c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d91bc8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c102040 0x7f5d8c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49816/0 (socket says 172.21.15.8:49816) 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d91bc8700 1 -- 172.21.15.8:0/2033294661 learned_addr learned my addr 172.21.15.8:0/2033294661 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d913c7700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d8c103240 0x7f5d8c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.239+0000 7f5d923c9700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 0x7f5d8c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d923c9700 1 -- 172.21.15.8:0/2033294661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d8c103240 msgr2=0x7f5d8c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:14.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d923c9700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d8c103240 0x7f5d8c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d923c9700 1 -- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c102040 msgr2=0x7f5d8c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d923c9700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c102040 0x7f5d8c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d923c9700 1 -- 172.21.15.8:0/2033294661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d800095f0 con 0x7f5d8c103be0 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d923c9700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 0x7f5d8c19f500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400efc0 tx=0x7f5d8400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84018070 con 0x7f5d8c103be0 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d8400f040 con 0x7f5d8c103be0 2024-02-19T16:22:14.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400ce00 con 0x7f5d8c103be0 2024-02-19T16:22:14.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1a0020 con 0x7f5d8c103be0 2024-02-19T16:22:14.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.240+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1a04d0 con 0x7f5d8c103be0 2024-02-19T16:22:14.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.241+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c1093a0 con 0x7f5d8c103be0 2024-02-19T16:22:14.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.242+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5d84004ad0 con 0x7f5d8c103be0 2024-02-19T16:22:14.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.243+0000 7f5d7effd700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d780a2150 0x7f5d780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:14.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.243+0000 7f5d91bc8700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d780a2150 0x7f5d780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:14.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.243+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f5d840cdb10 con 0x7f5d8c103be0 2024-02-19T16:22:14.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.243+0000 7f5d91bc8700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d780a2150 0x7f5d780a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005950 tx=0x7f5d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:14.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.247+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5d840967d0 con 0x7f5d8c103be0 2024-02-19T16:22:14.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.487+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d8c02da30 con 0x7f5d780a2150 2024-02-19T16:22:14.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.493+0000 7f5d7effd700 1 -- 172.21.15.8:0/2033294661 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f5d8c02da30 con 0x7f5d780a2150 2024-02-19T16:22:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-19T16:22:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:mds.all 12/12 50s ago 2m smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-19T16:22:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:mgr 3/3 50s ago 7m smithi008=x;smithi156=y;smithi191=z;count:3 2024-02-19T16:22:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:mon 3/3 50s ago 8m smithi008:172.21.15.8=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c;count:3 2024-02-19T16:22:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:osd 12 50s ago - 2024-02-19T16:22:14.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d780a2150 msgr2=0x7f5d780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:14.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d780a2150 0x7f5d780a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005950 tx=0x7f5d88009260 comp rx=0 tx=0).stop 2024-02-19T16:22:14.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 msgr2=0x7f5d8c19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:14.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 0x7f5d8c19f500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400efc0 tx=0x7f5d8400c3a0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 shutdown_connections 2024-02-19T16:22:14.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f5d780a2150 0x7f5d780a4610 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c103be0 0x7f5d8c19f500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5d8c102040 0x7f5d8c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 --2- 172.21.15.8:0/2033294661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d8c103240 0x7f5d8c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:14.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 >> 172.21.15.8:0/2033294661 conn(0x7f5d8c0fd5e0 msgr2=0x7f5d8c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:14.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 shutdown_connections 2024-02-19T16:22:14.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:14.495+0000 7f5d93e2c700 1 -- 172.21.15.8:0/2033294661 wait complete. 2024-02-19T16:22:14.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:14 smithi156 ceph-mon[36502]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:22:14.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:14 smithi008 ceph-mon[32573]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:22:14.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:14 smithi191 ceph-mon[37203]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:22:15.258 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-19T16:22:15.506 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:15.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:15 smithi008 ceph-mon[32573]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:15.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:15 smithi156 ceph-mon[36502]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:15.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:15 smithi191 ceph-mon[37203]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:16.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:16 smithi156 ceph-mon[36502]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:16.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:16 smithi008 ceph-mon[32573]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:16.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:16 smithi191 ceph-mon[37203]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- 172.21.15.8:0/3736444859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac103bb0 msgr2=0x7fd8ac10a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- 172.21.15.8:0/3736444859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac103bb0 0x7fd8ac10a460 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b0d0 tx=0x7fd8a400b3e0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- 172.21.15.8:0/3736444859 shutdown_connections 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- 172.21.15.8:0/3736444859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac103bb0 0x7fd8ac10a460 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- 172.21.15.8:0/3736444859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac102010 0x7fd8ac102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- 172.21.15.8:0/3736444859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac103210 0x7fd8ac103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- 172.21.15.8:0/3736444859 >> 172.21.15.8:0/3736444859 conn(0x7fd8ac0fd5f0 msgr2=0x7fd8ac0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- 172.21.15.8:0/3736444859 shutdown_connections 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- 172.21.15.8:0/3736444859 wait complete. 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 Processor -- start 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- start start 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 0x7fd8ac19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac103210 0x7fd8ac19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac103bb0 0x7fd8ac1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1a1e40 con 0x7fd8ac102010 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1a1f80 con 0x7fd8ac103bb0 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.362+0000 7fd8b4598700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1a2110 con 0x7fd8ac103210 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 0x7fd8ac19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b1b33700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac103210 0x7fd8ac19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 0x7fd8ac19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38864/0 (socket says 172.21.15.8:38864) 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 -- 172.21.15.8:0/1033812870 learned_addr learned my addr 172.21.15.8:0/1033812870 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2b35700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac103bb0 0x7fd8ac1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 -- 172.21.15.8:0/1033812870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac103210 msgr2=0x7fd8ac19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac103210 0x7fd8ac19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 -- 172.21.15.8:0/1033812870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac103bb0 msgr2=0x7fd8ac1a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac103bb0 0x7fd8ac1a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 -- 172.21.15.8:0/1033812870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a4009d00 con 0x7fd8ac102010 2024-02-19T16:22:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b2334700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 0x7fd8ac19a2b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800d9b0 tx=0x7fd8a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800b950 con 0x7fd8ac102010 2024-02-19T16:22:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8007620 con 0x7fd8ac102010 2024-02-19T16:22:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac1a23f0 con 0x7fd8ac102010 2024-02-19T16:22:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800f3c0 con 0x7fd8ac102010 2024-02-19T16:22:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.363+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac1a28f0 con 0x7fd8ac102010 2024-02-19T16:22:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.364+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac068ca0 con 0x7fd8ac102010 2024-02-19T16:22:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.366+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fd8a80105f0 con 0x7fd8ac102010 2024-02-19T16:22:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.367+0000 7fd8a37fe700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a2150 0x7fd8980a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.367+0000 7fd8b1b33700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a2150 0x7fd8980a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.367+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fd8a80ce660 con 0x7fd8ac102010 2024-02-19T16:22:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.367+0000 7fd8b1b33700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a2150 0x7fd8980a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd89c007b60 tx=0x7fd89c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.370+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd8a80972f0 con 0x7fd8ac102010 2024-02-19T16:22:17.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.578+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8ac075930 con 0x7fd8980a2150 2024-02-19T16:22:17.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.579+0000 7fd8a37fe700 1 -- 172.21.15.8:0/1033812870 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fd8ac075930 con 0x7fd8980a2150 2024-02-19T16:22:17.581 INFO:teuthology.orchestra.run.smithi008.stdout:HOST ADDR LABELS STATUS 2024-02-19T16:22:17.581 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008 172.21.15.8 2024-02-19T16:22:17.581 INFO:teuthology.orchestra.run.smithi008.stdout:smithi156 172.21.15.156 2024-02-19T16:22:17.581 INFO:teuthology.orchestra.run.smithi008.stdout:smithi191 172.21.15.191 2024-02-19T16:22:17.581 INFO:teuthology.orchestra.run.smithi008.stdout:3 hosts in cluster 2024-02-19T16:22:17.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a2150 msgr2=0x7fd8980a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:17.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a2150 0x7fd8980a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd89c007b60 tx=0x7fd89c0073d0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 msgr2=0x7fd8ac19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:17.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 0x7fd8ac19a2b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800d9b0 tx=0x7fd8a800dcc0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 shutdown_connections 2024-02-19T16:22:17.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a2150 0x7fd8980a4610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac103bb0 0x7fd8ac1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac102010 0x7fd8ac19a2b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 --2- 172.21.15.8:0/1033812870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac103210 0x7fd8ac19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:17.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 >> 172.21.15.8:0/1033812870 conn(0x7fd8ac0fd5f0 msgr2=0x7fd8ac074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:17.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 shutdown_connections 2024-02-19T16:22:17.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:17.581+0000 7fd8b4598700 1 -- 172.21.15.8:0/1033812870 wait complete. 2024-02-19T16:22:18.367 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-19T16:22:18.614 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:18.735 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:18 smithi008 ceph-mon[32573]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:22:18.735 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:18 smithi008 ceph-mon[32573]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:18.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:18 smithi191 ceph-mon[37203]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:22:18.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:18 smithi191 ceph-mon[37203]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:18.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:18 smithi156 ceph-mon[36502]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:22:18.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:18 smithi156 ceph-mon[36502]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 -- 172.21.15.8:0/3973846789 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 msgr2=0x7f77bc1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 --2- 172.21.15.8:0/3973846789 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc1006f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009990 tx=0x7f77ac009ca0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 -- 172.21.15.8:0/3973846789 shutdown_connections 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 --2- 172.21.15.8:0/3973846789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 0x7f77bc10a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 --2- 172.21.15.8:0/3973846789 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc1006f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 --2- 172.21.15.8:0/3973846789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77bc1032b0 0x7f77bc103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 -- 172.21.15.8:0/3973846789 >> 172.21.15.8:0/3973846789 conn(0x7f77bc0fbbf0 msgr2=0x7f77bc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 -- 172.21.15.8:0/3973846789 shutdown_connections 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 -- 172.21.15.8:0/3973846789 wait complete. 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 Processor -- start 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.349+0000 7f77c2826700 1 -- start start 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c2826700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c2826700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77bc1032b0 0x7f77bc101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c2826700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 0x7f77bc1017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c2826700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc101f20 con 0x7f77bc1002d0 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c2826700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc102090 con 0x7f77bc103c50 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c2826700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1a2080 con 0x7f77bc1032b0 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bb7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77bc1032b0 0x7f77bc101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:20.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc102c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38876/0 (socket says 172.21.15.8:38876) 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 -- 172.21.15.8:0/3913598456 learned_addr learned my addr 172.21.15.8:0/3913598456 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c0dc3700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 0x7f77bc1017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 -- 172.21.15.8:0/3913598456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77bc1032b0 msgr2=0x7f77bc101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77bc1032b0 0x7f77bc101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 -- 172.21.15.8:0/3913598456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 msgr2=0x7f77bc1017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 0x7f77bc1017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 -- 172.21.15.8:0/3913598456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0095f0 con 0x7f77bc1002d0 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77c0dc3700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 0x7f77bc1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77bbfff700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc102c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b6f0 tx=0x7f77ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.350+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac01d070 con 0x7f77bc1002d0 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.351+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77ac005030 con 0x7f77bc1002d0 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.351+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac004c40 con 0x7f77bc1002d0 2024-02-19T16:22:20.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.351+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc1a2300 con 0x7f77bc1002d0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.351+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc1a26b0 con 0x7f77bc1002d0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.352+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc1094f0 con 0x7f77bc1002d0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.353+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f77ac005510 con 0x7f77bc1002d0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.354+0000 7f77b97fa700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f77a40a2150 0x7f77a40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.354+0000 7f77bb7fe700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f77a40a2150 0x7f77a40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.354+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f77ac0cf310 con 0x7f77bc1002d0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.354+0000 7f77bb7fe700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f77a40a2150 0x7f77a40a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009c60 tx=0x7f77b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:20.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.357+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f77ac097fd0 con 0x7f77bc1002d0 2024-02-19T16:22:20.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:20 smithi008 ceph-mon[32573]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:20.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.571+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f77bc02da90 con 0x7f77a40a2150 2024-02-19T16:22:20.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.573+0000 7f77b97fa700 1 -- 172.21.15.8:0/3913598456 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f77bc02da90 con 0x7f77a40a2150 2024-02-19T16:22:20.575 INFO:teuthology.orchestra.run.smithi008.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-19T16:22:20.575 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476004A480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-19T16:22:20.575 INFO:teuthology.orchestra.run.smithi008.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-19T16:22:20.575 INFO:teuthology.orchestra.run.smithi008.stdout:smithi191 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303008M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-19T16:22:20.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f77a40a2150 msgr2=0x7f77a40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f77a40a2150 0x7f77a40a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009c60 tx=0x7f77b0009260 comp rx=0 tx=0).stop 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 msgr2=0x7f77bc102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc102c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b6f0 tx=0x7f77ac004730 comp rx=0 tx=0).stop 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 shutdown_connections 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f77a40a2150 0x7f77a40a4610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77bc103c50 0x7f77bc1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77bc1002d0 0x7f77bc102c10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 --2- 172.21.15.8:0/3913598456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77bc1032b0 0x7f77bc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 >> 172.21.15.8:0/3913598456 conn(0x7f77bc0fbbf0 msgr2=0x7f77bc106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:20.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 shutdown_connections 2024-02-19T16:22:20.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:20.575+0000 7f77c2826700 1 -- 172.21.15.8:0/3913598456 wait complete. 2024-02-19T16:22:20.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:20 smithi191 ceph-mon[37203]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:20 smithi156 ceph-mon[36502]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:21.343 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T16:22:21.353 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:22:21.353 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-19T16:22:21.598 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:21.720 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:21 smithi008 ceph-mon[32573]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:21.720 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-02-19T16:22:21.721 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-19T16:22:21.721 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:21 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-19T16:22:21.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:21 smithi191 ceph-mon[37203]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:21.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-02-19T16:22:21.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-19T16:22:21.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:21 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-19T16:22:21.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:21 smithi156 ceph-mon[36502]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:21.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-02-19T16:22:21.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-19T16:22:21.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:21 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-19T16:22:22.750 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:22 smithi008 ceph-mon[32573]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:22.750 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-02-19T16:22:22.750 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-19T16:22:22.751 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:22 smithi008 ceph-mon[32573]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-19T16:22:22.751 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:22 smithi008 ceph-mon[32573]: osdmap e94: 12 total, 12 up, 12 in 2024-02-19T16:22:22.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:22 smithi191 ceph-mon[37203]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:22.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-02-19T16:22:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-19T16:22:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:22 smithi191 ceph-mon[37203]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-19T16:22:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:22 smithi191 ceph-mon[37203]: osdmap e94: 12 total, 12 up, 12 in 2024-02-19T16:22:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:22 smithi156 ceph-mon[36502]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-02-19T16:22:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-19T16:22:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:22 smithi156 ceph-mon[36502]: from='mgr.14152 172.21.15.8:0/1975732967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-19T16:22:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:22 smithi156 ceph-mon[36502]: osdmap e94: 12 total, 12 up, 12 in 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.333+0000 7f87eb393700 1 -- 172.21.15.8:0/1414125419 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4103bb0 msgr2=0x7f87e410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.333+0000 7f87eb393700 1 --2- 172.21.15.8:0/1414125419 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4103bb0 0x7f87e410a460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00b0d0 tx=0x7f87dc00b3e0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 -- 172.21.15.8:0/1414125419 shutdown_connections 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 --2- 172.21.15.8:0/1414125419 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4103bb0 0x7f87e410a460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 --2- 172.21.15.8:0/1414125419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4102010 0x7f87e4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 --2- 172.21.15.8:0/1414125419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103210 0x7f87e4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 -- 172.21.15.8:0/1414125419 >> 172.21.15.8:0/1414125419 conn(0x7f87e40fd5f0 msgr2=0x7f87e40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 -- 172.21.15.8:0/1414125419 shutdown_connections 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.334+0000 7f87eb393700 1 -- 172.21.15.8:0/1414125419 wait complete. 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 Processor -- start 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 -- start start 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 0x7f87e41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4103210 0x7f87e4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103bb0 0x7f87e419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e419fc50 con 0x7f87e4102010 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e419fd90 con 0x7f87e4103bb0 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87eb393700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e419ff20 con 0x7f87e4103210 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 0x7f87e41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 0x7f87e41980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38892/0 (socket says 172.21.15.8:38892) 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 -- 172.21.15.8:0/149582440 learned_addr learned my addr 172.21.15.8:0/149582440 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:23.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e892e700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4103210 0x7f87e4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e9930700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103bb0 0x7f87e419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 -- 172.21.15.8:0/149582440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4103210 msgr2=0x7f87e4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4103210 0x7f87e4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 -- 172.21.15.8:0/149582440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103bb0 msgr2=0x7f87e419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103bb0 0x7f87e419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 -- 172.21.15.8:0/149582440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009d00 con 0x7f87e4102010 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e9930700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103bb0 0x7f87e419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.335+0000 7f87e912f700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 0x7f87e41980c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e980 tx=0x7f87e000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.336+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0009770 con 0x7f87e4102010 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.336+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87e000eed0 con 0x7f87e4102010 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.336+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e41a0200 con 0x7f87e4102010 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.336+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0003d90 con 0x7f87e4102010 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.336+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41a0700 con 0x7f87e4102010 2024-02-19T16:22:23.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.337+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e41093d0 con 0x7f87e4102010 2024-02-19T16:22:23.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.338+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f87e0004000 con 0x7f87e4102010 2024-02-19T16:22:23.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.338+0000 7f87da7fc700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87d00a21d0 0x7f87d00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:23.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.339+0000 7f87e892e700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87d00a21d0 0x7f87d00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:23.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.339+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f87e0014070 con 0x7f87e4102010 2024-02-19T16:22:23.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.339+0000 7f87e892e700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87d00a21d0 0x7f87d00a4690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009c60 tx=0x7f87d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:23.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.342+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f87e0096ff0 con 0x7f87e4102010 2024-02-19T16:22:23.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.590+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f87e410da00 con 0x7f87e4102010 2024-02-19T16:22:23.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:23 smithi008 ceph-mon[32573]: osdmap e95: 12 total, 12 up, 12 in 2024-02-19T16:22:23.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.591+0000 7f87da7fc700 1 -- 172.21.15.8:0/149582440 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2942 (secure 0 0 0) 0x7f87e009a6b0 con 0x7f87e4102010 2024-02-19T16:22:23.593 INFO:teuthology.orchestra.run.smithi008.stdout:e23 2024-02-19T16:22:23.593 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 23 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-02-19T16:20:21.002558+0000 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-02-19T16:20:41.315237+0000 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 300 2024-02-19T16:22:23.594 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 0 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 3 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1,2 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=24502,1=24488,2=24475} 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-02-19T16:22:23.595 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-02-19T16:22:23.596 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3,4] 2024-02-19T16:22:23.596 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-02-19T16:22:23.596 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-02-19T16:22:23.596 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-02-19T16:22:23.596 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 0 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.i{0:24502} state up:active seq 5 addr [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.f{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.191:6834/562441805,v1:172.21.15.191:6835/562441805] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.b{1:24488} state up:active seq 7 addr [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.h{1:24452} state up:standby-replay seq 1 addr [v2:172.21.15.156:6836/3993132637,v1:172.21.15.156:6837/3993132637] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.c{2:24475} state up:active seq 11 addr [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.l{2:24424} state up:standby-replay seq 1 addr [v2:172.21.15.191:6832/2702528663,v1:172.21.15.191:6833/2702528663] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.8:6834/3977271875,v1:172.21.15.8:6835/3977271875] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.8:6836/1583103315,v1:172.21.15.8:6837/1583103315] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.g{-1:14613} state up:standby seq 1 addr [v2:172.21.15.8:6838/1717311742,v1:172.21.15.8:6839/1717311742] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.597 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.a{-1:14625} state up:standby seq 1 addr [v2:172.21.15.8:6840/1752496516,v1:172.21.15.8:6841/1752496516] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.598 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.k{-1:24401} state up:standby seq 1 addr [v2:172.21.15.156:6832/2706765134,v1:172.21.15.156:6833/2706765134] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.598 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.e{-1:24422} state up:standby seq 1 addr [v2:172.21.15.156:6834/4229713412,v1:172.21.15.156:6835/4229713412] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T16:22:23.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87d00a21d0 msgr2=0x7f87d00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87d00a21d0 0x7f87d00a4690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009c60 tx=0x7f87d4009260 comp rx=0 tx=0).stop 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 msgr2=0x7f87e41980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 0x7f87e41980c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e980 tx=0x7f87e000ec90 comp rx=0 tx=0).stop 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 shutdown_connections 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87d00a21d0 0x7f87d00a4690 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87e4103bb0 0x7f87e419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87e4102010 0x7f87e41980c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 --2- 172.21.15.8:0/149582440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4103210 0x7f87e4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 >> 172.21.15.8:0/149582440 conn(0x7f87e40fd5f0 msgr2=0x7f87e4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 shutdown_connections 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:23.593+0000 7f87eb393700 1 -- 172.21.15.8:0/149582440 wait complete. 2024-02-19T16:22:23.599 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:22:23.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:23 smithi191 ceph-mon[37203]: osdmap e95: 12 total, 12 up, 12 in 2024-02-19T16:22:23.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:23 smithi156 ceph-mon[36502]: osdmap e95: 12 total, 12 up, 12 in 2024-02-19T16:22:24.333 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-19T16:22:24.578 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:24.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:24 smithi008 ceph-mon[32573]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-19T16:22:24.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:24 smithi008 ceph-mon[32573]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-02-19T16:22:24.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:24 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/149582440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T16:22:24.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:24 smithi191 ceph-mon[37203]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-19T16:22:24.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:24 smithi191 ceph-mon[37203]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-02-19T16:22:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:24 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/149582440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T16:22:24.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:24 smithi156 ceph-mon[36502]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-19T16:22:24.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:24 smithi156 ceph-mon[36502]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-02-19T16:22:24.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:24 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/149582440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 -- 172.21.15.8:0/1689983746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 msgr2=0x7fdf141025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 --2- 172.21.15.8:0/1689983746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 0x7fdf141025e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf080099c0 tx=0x7fdf08009cd0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 -- 172.21.15.8:0/1689983746 shutdown_connections 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 --2- 172.21.15.8:0/1689983746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14102b20 0x7fdf141093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 --2- 172.21.15.8:0/1689983746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14100f80 0x7fdf141013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 --2- 172.21.15.8:0/1689983746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 0x7fdf141025e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 -- 172.21.15.8:0/1689983746 >> 172.21.15.8:0/1689983746 conn(0x7fdf140fc540 msgr2=0x7fdf140fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 -- 172.21.15.8:0/1689983746 shutdown_connections 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.261+0000 7fdf19b52700 1 -- 172.21.15.8:0/1689983746 wait complete. 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 Processor -- start 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 -- start start 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 0x7fdf14104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 0x7fdf14103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14102b20 0x7fdf14103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14104240 con 0x7fdf14102180 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141043b0 con 0x7fdf14100f80 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf19b52700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141a2050 con 0x7fdf14102b20 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 0x7fdf14104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 0x7fdf14104ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:44530/0 (socket says 172.21.15.8:44530) 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf137fe700 1 -- 172.21.15.8:0/2510725701 learned_addr learned my addr 172.21.15.8:0/2510725701 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.262+0000 7fdf12ffd700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 0x7fdf14103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf13fff700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14102b20 0x7fdf14103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf137fe700 1 -- 172.21.15.8:0/2510725701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14102b20 msgr2=0x7fdf14103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf137fe700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14102b20 0x7fdf14103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf137fe700 1 -- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 msgr2=0x7fdf14103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf137fe700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 0x7fdf14103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf137fe700 1 -- 172.21.15.8:0/2510725701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080095f0 con 0x7fdf14100f80 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf13fff700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14102b20 0x7fdf14103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf137fe700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 0x7fdf14104ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e970 tx=0x7fdf0400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04009790 con 0x7fdf14100f80 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf0400eec0 con 0x7fdf14100f80 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04003e00 con 0x7fdf14100f80 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf141a22e0 con 0x7fdf14100f80 2024-02-19T16:22:26.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.263+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf141a2790 con 0x7fdf14100f80 2024-02-19T16:22:26.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.264+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf14068ca0 con 0x7fdf14100f80 2024-02-19T16:22:26.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.266+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fdf04003f60 con 0x7fdf14100f80 2024-02-19T16:22:26.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.266+0000 7fdf10ff9700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdefc0a21d0 0x7fdefc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:26.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.266+0000 7fdf12ffd700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdefc0a21d0 0x7fdefc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.266+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fdf04014070 con 0x7fdf14100f80 2024-02-19T16:22:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.267+0000 7fdf12ffd700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdefc0a21d0 0x7fdefc0a4690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdf08003890 tx=0x7fdf08005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:26.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.270+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fdf04096ee0 con 0x7fdf14100f80 2024-02-19T16:22:26.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.474+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fdf1410ca80 con 0x7fdf14100f80 2024-02-19T16:22:26.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.477+0000 7fdf10ff9700 1 -- 172.21.15.8:0/2510725701 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+5624 (secure 0 0 0) 0x7fdf0409a5a0 con 0x7fdf14100f80 2024-02-19T16:22:26.477 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 95 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:fsid 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-02-19T16:12:25.223147+0000 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-02-19T16:22:22.345953+0000 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:crush_version 27 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:full_ratio 0.95 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:backfillfull_ratio 0.9 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:nearfull_ratio 0.85 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:require_min_compat_client luminous 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:min_compat_client luminous 2024-02-19T16:22:26.478 INFO:teuthology.orchestra.run.smithi008.stdout:require_osd_release squid 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:stretch_mode_enabled false 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:max_osd 12 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6802/94559077,v1:172.21.15.8:6803/94559077] [v2:172.21.15.8:6804/94559077,v1:172.21.15.8:6805/94559077] exists,up bb7652b6-3998-473c-bd04-422f89e95197 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/295266432,v1:172.21.15.156:6801/295266432] [v2:172.21.15.156:6802/295266432,v1:172.21.15.156:6803/295266432] exists,up eaca12a9-0ccf-4c84-b071-32648d5dcca8 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6800/3756928102,v1:172.21.15.191:6801/3756928102] [v2:172.21.15.191:6802/3756928102,v1:172.21.15.191:6803/3756928102] exists,up c5bc2023-4102-4aec-a85d-4c8f175a3926 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6810/1491647664,v1:172.21.15.8:6811/1491647664] [v2:172.21.15.8:6812/1491647664,v1:172.21.15.8:6813/1491647664] exists,up c8fecdd8-fbd2-4e72-8607-dedc9f0ff018 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/3469489229,v1:172.21.15.156:6809/3469489229] [v2:172.21.15.156:6810/3469489229,v1:172.21.15.156:6811/3469489229] exists,up 356b4cc1-0311-4770-9e1f-02809310c1ac 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6808/499228077,v1:172.21.15.191:6809/499228077] [v2:172.21.15.191:6810/499228077,v1:172.21.15.191:6811/499228077] exists,up ca5e23ef-499f-4cca-ac7a-d85f36ffbedb 2024-02-19T16:22:26.479 INFO:teuthology.orchestra.run.smithi008.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6818/2596853517,v1:172.21.15.8:6819/2596853517] [v2:172.21.15.8:6820/2596853517,v1:172.21.15.8:6821/2596853517] exists,up 84d121f0-81a1-4702-bd47-4c34f40b5b9f 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:osd.7 up in weight 1 up_from 52 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/3871421578,v1:172.21.15.156:6817/3871421578] [v2:172.21.15.156:6818/3871421578,v1:172.21.15.156:6819/3871421578] exists,up 20a26fbd-a93b-472b-a894-7693384be848 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:osd.8 up in weight 1 up_from 60 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6816/2037482563,v1:172.21.15.191:6817/2037482563] [v2:172.21.15.191:6818/2037482563,v1:172.21.15.191:6819/2037482563] exists,up 0c6e14c6-7bb9-46da-9e3a-1581f86e1764 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:osd.9 up in weight 1 up_from 68 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6826/1070903169,v1:172.21.15.8:6827/1070903169] [v2:172.21.15.8:6828/1070903169,v1:172.21.15.8:6829/1070903169] exists,up 00fb3fac-7fc8-4d1f-8bae-dae54e114041 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:osd.10 up in weight 1 up_from 73 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/1977244529,v1:172.21.15.156:6825/1977244529] [v2:172.21.15.156:6826/1977244529,v1:172.21.15.156:6827/1977244529] exists,up 816f1015-93ed-443a-9ad8-e0291b9970cf 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:osd.11 up in weight 1 up_from 81 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6824/759584656,v1:172.21.15.191:6825/759584656] [v2:172.21.15.191:6826/759584656,v1:172.21.15.191:6827/759584656] exists,up a1b3c59f-d69e-4a0a-a285-6dd5b20b738a 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.1a [8,1] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.1d [2,4] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.1f [7,11] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.0 [5,9] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.13 [4,9] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.1e [3,9] 2024-02-19T16:22:26.480 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.23 [4,8] 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.f [5,8] 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.12 [9,10] 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.17 [9,8] 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.1c [9,8] 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/3227434625 expires 2024-02-20T16:13:20.630755+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6800/886492473 expires 2024-02-20T16:13:20.630755+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6801/886492473 expires 2024-02-20T16:13:20.630755+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6800/3125167978 expires 2024-02-20T16:12:53.197791+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/1092270301 expires 2024-02-20T16:13:20.630755+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/2946360420 expires 2024-02-20T16:12:53.197791+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6801/3125167978 expires 2024-02-20T16:12:53.197791+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/414318988 expires 2024-02-20T16:12:53.197791+0000 2024-02-19T16:22:26.481 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/2746045673 expires 2024-02-20T16:12:53.197791+0000 2024-02-19T16:22:26.482 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/4121837085 expires 2024-02-20T16:13:20.630755+0000 2024-02-19T16:22:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdefc0a21d0 msgr2=0x7fdefc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdefc0a21d0 0x7fdefc0a4690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdf08003890 tx=0x7fdf08005ad0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 msgr2=0x7fdf14104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 0x7fdf14104ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e970 tx=0x7fdf0400ec80 comp rx=0 tx=0).stop 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 shutdown_connections 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fdefc0a21d0 0x7fdefc0a4690 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf14102b20 0x7fdf14103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf14100f80 0x7fdf14104ec0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 --2- 172.21.15.8:0/2510725701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14102180 0x7fdf14103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 >> 172.21.15.8:0/2510725701 conn(0x7fdf140fc540 msgr2=0x7fdf140fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 shutdown_connections 2024-02-19T16:22:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:26.478+0000 7fdf19b52700 1 -- 172.21.15.8:0/2510725701 wait complete. 2024-02-19T16:22:26.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:26 smithi008 ceph-mon[32573]: pgmap v357: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-19T16:22:26.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:26 smithi191 ceph-mon[37203]: pgmap v357: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-19T16:22:26.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:26 smithi156 ceph-mon[36502]: pgmap v357: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-19T16:22:27.402 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-19T16:22:27.414 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 -- 172.21.15.8:0/2372109526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f8101510 msgr2=0x7f95f8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/2372109526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f8101510 0x7f95f8101930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009900 tx=0x7f95f002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 -- 172.21.15.8:0/2372109526 shutdown_connections 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/2372109526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f81030c0 0x7f95f8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/2372109526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8102700 0x7f95f8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/2372109526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f8101510 0x7f95f8101930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 -- 172.21.15.8:0/2372109526 >> 172.21.15.8:0/2372109526 conn(0x7f95f80fcaa0 msgr2=0x7f95f80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 -- 172.21.15.8:0/2372109526 shutdown_connections 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 -- 172.21.15.8:0/2372109526 wait complete. 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.527+0000 7f95fd5b5640 1 Processor -- start 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 -- start start 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 0x7f95f80752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 0x7f95f80738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f81030c0 0x7f95f8073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f8074580 con 0x7f95f81030c0 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80746f0 con 0x7f95f8101510 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95fd5b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f81a15f0 con 0x7f95f8102700 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 0x7f95f80738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 0x7f95f80752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f81030c0 0x7f95f8073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 0x7f95f80738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:50300/0 (socket says 172.21.15.8:50300) 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 0x7f95f80752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:44548/0 (socket says 172.21.15.8:44548) 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 -- 172.21.15.8:0/876863933 learned_addr learned my addr 172.21.15.8:0/876863933 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 -- 172.21.15.8:0/876863933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 msgr2=0x7f95f80752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 0x7f95f80752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 -- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f81030c0 msgr2=0x7f95f8073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f81030c0 0x7f95f8073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f67fc640 1 -- 172.21.15.8:0/876863933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009560 con 0x7f95f8102700 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f6ffd640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 0x7f95f80752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.528+0000 7f95f77fe640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f81030c0 0x7f95f8073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.529+0000 7f95f67fc640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 0x7f95f80738e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95e000e9f0 tx=0x7f95e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.529+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0009700 con 0x7f95f8102700 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.529+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95e000cd80 con 0x7f95f8102700 2024-02-19T16:22:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.529+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f81a18a0 con 0x7f95f8102700 2024-02-19T16:22:27.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.529+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0018920 con 0x7f95f8102700 2024-02-19T16:22:27.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.529+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f81a1db0 con 0x7f95f8102700 2024-02-19T16:22:27.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.530+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f81088b0 con 0x7f95f8102700 2024-02-19T16:22:27.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.531+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f95e0018a80 con 0x7f95f8102700 2024-02-19T16:22:27.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.531+0000 7f95effff640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95e40a20c0 0x7f95e40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.532+0000 7f95f6ffd640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95e40a20c0 0x7f95e40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.532+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f95e0014070 con 0x7f95f8102700 2024-02-19T16:22:27.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.532+0000 7f95f6ffd640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95e40a20c0 0x7f95e40a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95f0005ab0 tx=0x7f95f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:27.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.535+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f95e00970a0 con 0x7f95f8102700 2024-02-19T16:22:27.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.717+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f810cf20 con 0x7f95f8102700 2024-02-19T16:22:27.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.718+0000 7f95effff640 1 -- 172.21.15.8:0/876863933 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7f95e009a760 con 0x7f95f8102700 2024-02-19T16:22:27.719 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:27.720 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:27.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95e40a20c0 msgr2=0x7f95e40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95e40a20c0 0x7f95e40a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95f0005ab0 tx=0x7f95f0005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 msgr2=0x7f95f80738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 0x7f95f80738e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95e000e9f0 tx=0x7f95e000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 shutdown_connections 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f95e40a20c0 0x7f95e40a4580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95f81030c0 0x7f95f8073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f8102700 0x7f95f80738e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 --2- 172.21.15.8:0/876863933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f8101510 0x7f95f80752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 >> 172.21.15.8:0/876863933 conn(0x7f95f80fcaa0 msgr2=0x7f95f8105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 shutdown_connections 2024-02-19T16:22:27.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.720+0000 7f95fd5b5640 1 -- 172.21.15.8:0/876863933 wait complete. 2024-02-19T16:22:27.731 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:27 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2510725701' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-19T16:22:27.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:27.733 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:27.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:27 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2510725701' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-19T16:22:27.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:27 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2510725701' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-19T16:22:27.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 -- 172.21.15.8:0/815688677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8294103170 msgr2=0x7f8294109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 --2- 172.21.15.8:0/815688677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8294103170 0x7f8294109a00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f828c009ca0 tx=0x7f828c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 -- 172.21.15.8:0/815688677 shutdown_connections 2024-02-19T16:22:27.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 --2- 172.21.15.8:0/815688677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8294103170 0x7f8294109a00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 --2- 172.21.15.8:0/815688677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82941027b0 0x7f8294102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 --2- 172.21.15.8:0/815688677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82940ff7e0 0x7f82940ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 -- 172.21.15.8:0/815688677 >> 172.21.15.8:0/815688677 conn(0x7f82940fb0f0 msgr2=0x7f82940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 -- 172.21.15.8:0/815688677 shutdown_connections 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.839+0000 7f8298e5b640 1 -- 172.21.15.8:0/815688677 wait complete. 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 Processor -- start 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 -- start start 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82940ff7e0 0x7f8294102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 0x7f8294100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8294103170 0x7f8294100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8294101400 con 0x7f82941027b0 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8294101570 con 0x7f82940ff7e0 2024-02-19T16:22:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8298e5b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82941a15a0 con 0x7f8294103170 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 0x7f8294100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 0x7f8294100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34668/0 (socket says 172.21.15.8:34668) 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 -- 172.21.15.8:0/132700441 learned_addr learned my addr 172.21.15.8:0/132700441 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f8292d9d640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8294103170 0x7f8294100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f829259c640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82940ff7e0 0x7f8294102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 -- 172.21.15.8:0/132700441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8294103170 msgr2=0x7f8294100ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8294103170 0x7f8294100ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 -- 172.21.15.8:0/132700441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82940ff7e0 msgr2=0x7f8294102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82940ff7e0 0x7f8294102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.840+0000 7f828bfff640 1 -- 172.21.15.8:0/132700441 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c009900 con 0x7f82941027b0 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.841+0000 7f828bfff640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 0x7f8294100760 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f827c00ea20 tx=0x7f827c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.841+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c009720 con 0x7f82941027b0 2024-02-19T16:22:27.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.841+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f827c00cd80 con 0x7f82941027b0 2024-02-19T16:22:27.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.841+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82941a1850 con 0x7f82941027b0 2024-02-19T16:22:27.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.841+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c010490 con 0x7f82941027b0 2024-02-19T16:22:27.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.841+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82941a1d10 con 0x7f82941027b0 2024-02-19T16:22:27.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.842+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8294108880 con 0x7f82941027b0 2024-02-19T16:22:27.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.843+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f827c005000 con 0x7f82941027b0 2024-02-19T16:22:27.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.843+0000 7f828b7fe640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82800a20c0 0x7f82800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:27.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.844+0000 7f829259c640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82800a20c0 0x7f82800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:27.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.844+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f827c014070 con 0x7f82941027b0 2024-02-19T16:22:27.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.844+0000 7f829259c640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82800a20c0 0x7f82800a4580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8294069900 tx=0x7f8284005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:27.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:27.847+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f827c097260 con 0x7f82941027b0 2024-02-19T16:22:28.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.028+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f829410d0d0 con 0x7f82941027b0 2024-02-19T16:22:28.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.029+0000 7f828b7fe640 1 -- 172.21.15.8:0/132700441 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f829410d0d0 con 0x7f82941027b0 2024-02-19T16:22:28.030 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82800a20c0 msgr2=0x7f82800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82800a20c0 0x7f82800a4580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8294069900 tx=0x7f8284005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 msgr2=0x7f8294100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 0x7f8294100760 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f827c00ea20 tx=0x7f827c00ee20 comp rx=0 tx=0).stop 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 shutdown_connections 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82800a20c0 0x7f82800a4580 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8294103170 0x7f8294100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82941027b0 0x7f8294100760 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 --2- 172.21.15.8:0/132700441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82940ff7e0 0x7f8294102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 >> 172.21.15.8:0/132700441 conn(0x7f82940fb0f0 msgr2=0x7f8294105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 shutdown_connections 2024-02-19T16:22:28.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.031+0000 7f8298e5b640 1 -- 172.21.15.8:0/132700441 wait complete. 2024-02-19T16:22:28.043 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:28.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 -- 172.21.15.8:0/48518244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 msgr2=0x7f3bbc106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/48518244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 0x7f3bbc106d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009860 tx=0x7f3bb002bb70 comp rx=0 tx=0).stop 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 -- 172.21.15.8:0/48518244 shutdown_connections 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/48518244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc107290 0x7f3bbc109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/48518244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 0x7f3bbc106d50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/48518244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bbc102030 0x7f3bbc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 -- 172.21.15.8:0/48518244 >> 172.21.15.8:0/48518244 conn(0x7f3bbc0fb980 msgr2=0x7f3bbc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.147+0000 7f3bc3b81640 1 -- 172.21.15.8:0/48518244 shutdown_connections 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 -- 172.21.15.8:0/48518244 wait complete. 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 Processor -- start 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 -- start start 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 0x7f3bbc197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 0x7f3bbc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bbc107290 0x7f3bbc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19f110 con 0x7f3bbc102030 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19f280 con 0x7f3bbc104960 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc3b81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19f440 con 0x7f3bbc107290 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc191d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 0x7f3bbc197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc191d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 0x7f3bbc197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34682/0 (socket says 172.21.15.8:34682) 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc191d640 1 -- 172.21.15.8:0/1338668577 learned_addr learned my addr 172.21.15.8:0/1338668577 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc211e640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bbc107290 0x7f3bbc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.148+0000 7f3bc111c640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 0x7f3bbc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc191d640 1 -- 172.21.15.8:0/1338668577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bbc107290 msgr2=0x7f3bbc19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc191d640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bbc107290 0x7f3bbc19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc191d640 1 -- 172.21.15.8:0/1338668577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 msgr2=0x7f3bbc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc191d640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 0x7f3bbc197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc191d640 1 -- 172.21.15.8:0/1338668577 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac009900 con 0x7f3bbc102030 2024-02-19T16:22:28.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc191d640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 0x7f3bbc197670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400e9f0 tx=0x7f3bb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4009700 con 0x7f3bbc102030 2024-02-19T16:22:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb0009490 con 0x7f3bbc102030 2024-02-19T16:22:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bb400cd80 con 0x7f3bbc102030 2024-02-19T16:22:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4018920 con 0x7f3bbc102030 2024-02-19T16:22:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.149+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc19fac0 con 0x7f3bbc102030 2024-02-19T16:22:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.150+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc0fd470 con 0x7f3bbc102030 2024-02-19T16:22:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.151+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f3bb4018a80 con 0x7f3bbc102030 2024-02-19T16:22:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.151+0000 7f3bbaffd640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3ba40a20c0 0x7f3ba40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.152+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f3bb4014070 con 0x7f3bbc102030 2024-02-19T16:22:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.152+0000 7f3bc111c640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3ba40a20c0 0x7f3ba40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.152+0000 7f3bc111c640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3ba40a20c0 0x7f3ba40a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009830 tx=0x7f3bb0034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:28.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.155+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f3bb4097530 con 0x7f3bbc102030 2024-02-19T16:22:28.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.336+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bbc10cf10 con 0x7f3bbc102030 2024-02-19T16:22:28.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.336+0000 7f3bbaffd640 1 -- 172.21.15.8:0/1338668577 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bb409abf0 con 0x7f3bbc102030 2024-02-19T16:22:28.337 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:28.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3ba40a20c0 msgr2=0x7f3ba40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3ba40a20c0 0x7f3ba40a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009830 tx=0x7f3bb0034c00 comp rx=0 tx=0).stop 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 msgr2=0x7f3bbc197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 0x7f3bbc197670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400e9f0 tx=0x7f3bb400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 shutdown_connections 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f3ba40a20c0 0x7f3ba40a4580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.338+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bbc107290 0x7f3bbc19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.339+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bbc104960 0x7f3bbc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.339+0000 7f3bc3b81640 1 --2- 172.21.15.8:0/1338668577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bbc102030 0x7f3bbc197670 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.339+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 >> 172.21.15.8:0/1338668577 conn(0x7f3bbc0fb980 msgr2=0x7f3bbc102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.339+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 shutdown_connections 2024-02-19T16:22:28.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.339+0000 7f3bc3b81640 1 -- 172.21.15.8:0/1338668577 wait complete. 2024-02-19T16:22:28.350 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:28.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 -- 172.21.15.8:0/3033117489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc0ff560 msgr2=0x7f99cc0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 --2- 172.21.15.8:0/3033117489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc0ff560 0x7f99cc0ff9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009900 tx=0x7f99c002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:28.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 -- 172.21.15.8:0/3033117489 shutdown_connections 2024-02-19T16:22:28.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 --2- 172.21.15.8:0/3033117489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99cc0fff20 0x7f99cc109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 --2- 172.21.15.8:0/3033117489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc0ff560 0x7f99cc0ff9e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 --2- 172.21.15.8:0/3033117489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99cc102af0 0x7f99cc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 -- 172.21.15.8:0/3033117489 >> 172.21.15.8:0/3033117489 conn(0x7f99cc0fb120 msgr2=0x7f99cc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 -- 172.21.15.8:0/3033117489 shutdown_connections 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.455+0000 7f99d3098640 1 -- 172.21.15.8:0/3033117489 wait complete. 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 Processor -- start 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 -- start start 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99cc0ff560 0x7f99cc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99cc0fff20 0x7f99cc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 0x7f99cc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19f080 con 0x7f99cc102af0 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19f1f0 con 0x7f99cc0fff20 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d3098640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19f3b0 con 0x7f99cc0ff560 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 0x7f99cc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 0x7f99cc19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34696/0 (socket says 172.21.15.8:34696) 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 -- 172.21.15.8:0/2667170176 learned_addr learned my addr 172.21.15.8:0/2667170176 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99cbfff640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99cc0fff20 0x7f99cc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d0e34640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99cc0ff560 0x7f99cc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 -- 172.21.15.8:0/2667170176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99cc0ff560 msgr2=0x7f99cc197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99cc0ff560 0x7f99cc197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 -- 172.21.15.8:0/2667170176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99cc0fff20 msgr2=0x7f99cc197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99cc0fff20 0x7f99cc197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.456+0000 7f99d1635640 1 -- 172.21.15.8:0/2667170176 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0009560 con 0x7f99cc102af0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.457+0000 7f99d1635640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 0x7f99cc19eb40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e970 tx=0x7f99bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.457+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc009720 con 0x7f99cc102af0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.457+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99bc005500 con 0x7f99cc102af0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.457+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc19f690 con 0x7f99cc102af0 2024-02-19T16:22:28.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.457+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc010430 con 0x7f99cc102af0 2024-02-19T16:22:28.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.457+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc19fb20 con 0x7f99cc102af0 2024-02-19T16:22:28.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.458+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc1089b0 con 0x7f99cc102af0 2024-02-19T16:22:28.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.459+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f99bc005020 con 0x7f99cc102af0 2024-02-19T16:22:28.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.459+0000 7f99c9ffb640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f99b40a20c0 0x7f99b40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.460+0000 7f99d0e34640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f99b40a20c0 0x7f99b40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.460+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f99bc014070 con 0x7f99cc102af0 2024-02-19T16:22:28.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.460+0000 7f99d0e34640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f99b40a20c0 0x7f99b40a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f99cc069860 tx=0x7f99c4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:28.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.464+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f99bc097000 con 0x7f99cc102af0 2024-02-19T16:22:28.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.647+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99cc02d000 con 0x7f99cc102af0 2024-02-19T16:22:28.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.648+0000 7f99c9ffb640 1 -- 172.21.15.8:0/2667170176 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99bc09a6c0 con 0x7f99cc102af0 2024-02-19T16:22:28.649 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:28.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f99b40a20c0 msgr2=0x7f99b40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f99b40a20c0 0x7f99b40a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f99cc069860 tx=0x7f99c4009500 comp rx=0 tx=0).stop 2024-02-19T16:22:28.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 msgr2=0x7f99cc19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 0x7f99cc19eb40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e970 tx=0x7f99bc00ed70 comp rx=0 tx=0).stop 2024-02-19T16:22:28.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 shutdown_connections 2024-02-19T16:22:28.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f99b40a20c0 0x7f99b40a4580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99cc102af0 0x7f99cc19eb40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99cc0fff20 0x7f99cc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 --2- 172.21.15.8:0/2667170176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99cc0ff560 0x7f99cc197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 >> 172.21.15.8:0/2667170176 conn(0x7f99cc0fb120 msgr2=0x7f99cc105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 shutdown_connections 2024-02-19T16:22:28.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.650+0000 7f99d3098640 1 -- 172.21.15.8:0/2667170176 wait complete. 2024-02-19T16:22:28.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:28 smithi008 ceph-mon[32573]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:22:28.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:28 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/876863933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:28.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:28 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/132700441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:28.661 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:28 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1338668577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:28.662 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:28.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.767+0000 7f909b5e4640 1 -- 172.21.15.8:0/1463554420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90941003f0 msgr2=0x7f9094100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.767+0000 7f909b5e4640 1 --2- 172.21.15.8:0/1463554420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90941003f0 0x7f9094100810 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9090009900 tx=0x7f909002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- 172.21.15.8:0/1463554420 shutdown_connections 2024-02-19T16:22:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 --2- 172.21.15.8:0/1463554420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9094101fa0 0x7f9094108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 --2- 172.21.15.8:0/1463554420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f9094101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 --2- 172.21.15.8:0/1463554420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90941003f0 0x7f9094100810 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- 172.21.15.8:0/1463554420 >> 172.21.15.8:0/1463554420 conn(0x7f90940fb980 msgr2=0x7f90940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- 172.21.15.8:0/1463554420 shutdown_connections 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- 172.21.15.8:0/1463554420 wait complete. 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 Processor -- start 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- start start 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90941003f0 0x7f90940691e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f909419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 0x7f909419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909419f100 con 0x7f9094101fa0 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909419f270 con 0x7f90941003f0 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f909b5e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909419f430 con 0x7f90941015e0 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.768+0000 7f9099b81640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 0x7f909419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f909419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9099380640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90941003f0 0x7f90940691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f909419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:50346/0 (socket says 172.21.15.8:50346) 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 -- 172.21.15.8:0/2783221210 learned_addr learned my addr 172.21.15.8:0/2783221210 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9099b81640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 0x7f909419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34722/0 (socket says 172.21.15.8:34722) 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 -- 172.21.15.8:0/2783221210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90941003f0 msgr2=0x7f90940691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90941003f0 0x7f90940691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 -- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 msgr2=0x7f909419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 0x7f909419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 -- 172.21.15.8:0/2783221210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9090009560 con 0x7f90941015e0 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9099b81640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 0x7f909419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9099380640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90941003f0 0x7f90940691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f9098b7f640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f909419c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f908400e9f0 tx=0x7f908400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084009720 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909419f710 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f908400cd80 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084010690 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.769+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909419fbd0 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.770+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9094107830 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.771+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f90840046c0 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.772+0000 7f908a7fc640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90800a20c0 0x7f90800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.772+0000 7f9099380640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90800a20c0 0x7f90800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.772+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f9084014070 con 0x7f90941015e0 2024-02-19T16:22:28.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.772+0000 7f9099380640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90800a20c0 0x7f90800a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9090005ab0 tx=0x7f9090005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:28.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.775+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9084097200 con 0x7f90941015e0 2024-02-19T16:22:28.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:28 smithi191 ceph-mon[37203]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:22:28.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:28 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/876863933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:28.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:28 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/132700441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:28.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:28 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1338668577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:28 smithi156 ceph-mon[36502]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:22:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:28 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/876863933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:28 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/132700441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:28 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1338668577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:28.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.960+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f909410c050 con 0x7f90941015e0 2024-02-19T16:22:28.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.960+0000 7f908a7fc640 1 -- 172.21.15.8:0/2783221210 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f908409a8c0 con 0x7f90941015e0 2024-02-19T16:22:28.961 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:28.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90800a20c0 msgr2=0x7f90800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90800a20c0 0x7f90800a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9090005ab0 tx=0x7f9090005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 msgr2=0x7f909419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:28.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f909419c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f908400e9f0 tx=0x7f908400edf0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 shutdown_connections 2024-02-19T16:22:28.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f90800a20c0 0x7f90800a4580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9094101fa0 0x7f909419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941015e0 0x7f909419c330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 --2- 172.21.15.8:0/2783221210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90941003f0 0x7f90940691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:28.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 >> 172.21.15.8:0/2783221210 conn(0x7f90940fb980 msgr2=0x7f9094104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:28.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 shutdown_connections 2024-02-19T16:22:28.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:28.963+0000 7f909b5e4640 1 -- 172.21.15.8:0/2783221210 wait complete. 2024-02-19T16:22:28.975 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 -- 172.21.15.8:0/1712119747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 msgr2=0x7f2abc109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1712119747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 0x7f2abc109970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009be0 tx=0x7f2ab402bef0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 -- 172.21.15.8:0/1712119747 shutdown_connections 2024-02-19T16:22:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1712119747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 0x7f2abc109970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1712119747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc102720 0x7f2abc102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1712119747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2abc101530 0x7f2abc101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 -- 172.21.15.8:0/1712119747 >> 172.21.15.8:0/1712119747 conn(0x7f2abc0fcaa0 msgr2=0x7f2abc0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 -- 172.21.15.8:0/1712119747 shutdown_connections 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 -- 172.21.15.8:0/1712119747 wait complete. 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 Processor -- start 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.078+0000 7f2ac365c640 1 -- start start 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 0x7f2abc197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2abc102720 0x7f2abc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 0x7f2abc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19f060 con 0x7f2abc101530 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19f1d0 con 0x7f2abc102720 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19f390 con 0x7f2abc1030e0 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 0x7f2abc197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 0x7f2abc197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34738/0 (socket says 172.21.15.8:34738) 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 -- 172.21.15.8:0/1673268841 learned_addr learned my addr 172.21.15.8:0/1673268841 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac1bf9640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 0x7f2abc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac0bf7640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2abc102720 0x7f2abc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 -- 172.21.15.8:0/1673268841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 msgr2=0x7f2abc19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 0x7f2abc19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 -- 172.21.15.8:0/1673268841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2abc102720 msgr2=0x7f2abc197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2abc102720 0x7f2abc197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 -- 172.21.15.8:0/1673268841 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac009560 con 0x7f2abc101530 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac13f8640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 0x7f2abc197570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800e9c0 tx=0x7f2ab800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8009700 con 0x7f2abc101530 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab4009840 con 0x7f2abc101530 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ab800cd80 con 0x7f2abc101530 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8010430 con 0x7f2abc101530 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.079+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc19fa40 con 0x7f2abc101530 2024-02-19T16:22:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.081+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc1087f0 con 0x7f2abc101530 2024-02-19T16:22:29.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.082+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2ab8010590 con 0x7f2abc101530 2024-02-19T16:22:29.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.082+0000 7f2ab27fc640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2aa80a20c0 0x7f2aa80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.082+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f2ab8014070 con 0x7f2abc101530 2024-02-19T16:22:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.082+0000 7f2ac0bf7640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2aa80a20c0 0x7f2aa80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.083+0000 7f2ac0bf7640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2aa80a20c0 0x7f2aa80a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2abc198a80 tx=0x7f2aac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:29.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.086+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2ab8097090 con 0x7f2abc101530 2024-02-19T16:22:29.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.307+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2abc19f550 con 0x7f2abc101530 2024-02-19T16:22:29.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.308+0000 7f2ab27fc640 1 -- 172.21.15.8:0/1673268841 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f2ab809a750 con 0x7f2abc101530 2024-02-19T16:22:29.308 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:29.309 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:41.315237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:29.309 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:22:29.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.309+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2aa80a20c0 msgr2=0x7f2aa80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.309+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2aa80a20c0 0x7f2aa80a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2abc198a80 tx=0x7f2aac009260 comp rx=0 tx=0).stop 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 msgr2=0x7f2abc197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 0x7f2abc197570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800e9c0 tx=0x7f2ab800ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 shutdown_connections 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2aa80a20c0 0x7f2aa80a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2abc1030e0 0x7f2abc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2abc102720 0x7f2abc197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 --2- 172.21.15.8:0/1673268841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2abc101530 0x7f2abc197570 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 >> 172.21.15.8:0/1673268841 conn(0x7f2abc0fcaa0 msgr2=0x7f2abc1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 shutdown_connections 2024-02-19T16:22:29.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.310+0000 7f2ac365c640 1 -- 172.21.15.8:0/1673268841 wait complete. 2024-02-19T16:22:29.322 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 -- 172.21.15.8:0/2642599865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30102700 msgr2=0x7efd30102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2642599865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30102700 0x7efd30102b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd24009900 tx=0x7efd2402bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 -- 172.21.15.8:0/2642599865 shutdown_connections 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2642599865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd301030c0 0x7efd30109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2642599865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30102700 0x7efd30102b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2642599865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd30101510 0x7efd30101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 -- 172.21.15.8:0/2642599865 >> 172.21.15.8:0/2642599865 conn(0x7efd300fcaa0 msgr2=0x7efd300feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 -- 172.21.15.8:0/2642599865 shutdown_connections 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.424+0000 7efd37f6c640 1 -- 172.21.15.8:0/2642599865 wait complete. 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 Processor -- start 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 -- start start 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 0x7efd301975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd30102700 0x7efd30197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd301030c0 0x7efd3019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3019f0a0 con 0x7efd30101510 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3019f210 con 0x7efd30102700 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd37f6c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3019f3d0 con 0x7efd301030c0 2024-02-19T16:22:29.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd35d08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 0x7efd301975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd35d08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 0x7efd301975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34756/0 (socket says 172.21.15.8:34756) 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd35d08640 1 -- 172.21.15.8:0/2489063147 learned_addr learned my addr 172.21.15.8:0/2489063147 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd35507640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd30102700 0x7efd30197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.425+0000 7efd36509640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd301030c0 0x7efd3019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd35d08640 1 -- 172.21.15.8:0/2489063147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd301030c0 msgr2=0x7efd3019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd35d08640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd301030c0 0x7efd3019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd35d08640 1 -- 172.21.15.8:0/2489063147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd30102700 msgr2=0x7efd30197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd35d08640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd30102700 0x7efd30197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd35d08640 1 -- 172.21.15.8:0/2489063147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24009560 con 0x7efd30101510 2024-02-19T16:22:29.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd35d08640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 0x7efd301975b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd2800e9c0 tx=0x7efd2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:29.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28009700 con 0x7efd30101510 2024-02-19T16:22:29.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3019f6b0 con 0x7efd30101510 2024-02-19T16:22:29.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2800cd80 con 0x7efd30101510 2024-02-19T16:22:29.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28010430 con 0x7efd30101510 2024-02-19T16:22:29.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.426+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3019fb90 con 0x7efd30101510 2024-02-19T16:22:29.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.427+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd30108830 con 0x7efd30101510 2024-02-19T16:22:29.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.428+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7efd28003680 con 0x7efd30101510 2024-02-19T16:22:29.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.429+0000 7efd2effd640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2140 0x7efd180a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.429+0000 7efd35507640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2140 0x7efd180a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.429+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7efd28014070 con 0x7efd30101510 2024-02-19T16:22:29.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.429+0000 7efd35507640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2140 0x7efd180a4600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd24009230 tx=0x7efd24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:29.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.432+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7efd280971a0 con 0x7efd30101510 2024-02-19T16:22:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.615+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd3010d110 con 0x7efd30101510 2024-02-19T16:22:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.616+0000 7efd2effd640 1 -- 172.21.15.8:0/2489063147 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7efd3010d110 con 0x7efd30101510 2024-02-19T16:22:29.618 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:29.618 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2140 msgr2=0x7efd180a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2140 0x7efd180a4600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd24009230 tx=0x7efd24005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 msgr2=0x7efd301975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 0x7efd301975b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd2800e9c0 tx=0x7efd2800ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 shutdown_connections 2024-02-19T16:22:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2140 0x7efd180a4600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd301030c0 0x7efd3019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd30102700 0x7efd30197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 --2- 172.21.15.8:0/2489063147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd30101510 0x7efd301975b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 >> 172.21.15.8:0/2489063147 conn(0x7efd300fcaa0 msgr2=0x7efd30105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 shutdown_connections 2024-02-19T16:22:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.618+0000 7efd37f6c640 1 -- 172.21.15.8:0/2489063147 wait complete. 2024-02-19T16:22:29.629 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:29 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2667170176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:29.630 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:29 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2783221210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:29.630 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:29 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1673268841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:29.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:29.633 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:29.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 -- 172.21.15.8:0/2216072491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 msgr2=0x7ff494101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2216072491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff494101a60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff484009900 tx=0x7ff48402bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 -- 172.21.15.8:0/2216072491 shutdown_connections 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2216072491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 0x7ff494108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2216072491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff494101a60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2216072491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4941003f0 0x7ff494100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.739+0000 7ff49a89d640 1 -- 172.21.15.8:0/2216072491 >> 172.21.15.8:0/2216072491 conn(0x7ff4940fb980 msgr2=0x7ff4940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 -- 172.21.15.8:0/2216072491 shutdown_connections 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 -- 172.21.15.8:0/2216072491 wait complete. 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 Processor -- start 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 -- start start 2024-02-19T16:22:29.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4941003f0 0x7ff494196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff4941969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 0x7ff49419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49419df60 con 0x7ff4941015e0 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49419e0d0 con 0x7ff494101fa0 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.740+0000 7ff49a89d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49419e290 con 0x7ff4941003f0 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff4941969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff4941969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34778/0 (socket says 172.21.15.8:34778) 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 -- 172.21.15.8:0/2128436249 learned_addr learned my addr 172.21.15.8:0/2128436249 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff493fff640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4941003f0 0x7ff494196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff498e3a640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 0x7ff49419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 -- 172.21.15.8:0/2128436249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4941003f0 msgr2=0x7ff494196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4941003f0 0x7ff494196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 -- 172.21.15.8:0/2128436249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 msgr2=0x7ff49419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 0x7ff49419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 -- 172.21.15.8:0/2128436249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484009560 con 0x7ff4941015e0 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff498e3a640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 0x7ff49419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4937fe640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff4941969b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff484006010 tx=0x7ff484005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840043d0 con 0x7ff4941015e0 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49419e510 con 0x7ff4941015e0 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4840052d0 con 0x7ff4941015e0 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484003cc0 con 0x7ff4941015e0 2024-02-19T16:22:29.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.741+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49419ea20 con 0x7ff4941015e0 2024-02-19T16:22:29.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.742+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff494107730 con 0x7ff4941015e0 2024-02-19T16:22:29.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.743+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7ff484005440 con 0x7ff4941015e0 2024-02-19T16:22:29.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.744+0000 7ff4917fa640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff4800a20c0 0x7ff4800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:29.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.744+0000 7ff493fff640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff4800a20c0 0x7ff4800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:29.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.744+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7ff4840f0370 con 0x7ff4941015e0 2024-02-19T16:22:29.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.744+0000 7ff493fff640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff4800a20c0 0x7ff4800a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff494101440 tx=0x7ff47c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:29.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.748+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff4840b8f40 con 0x7ff4941015e0 2024-02-19T16:22:29.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:29 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2667170176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:29.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:29 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2783221210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:29.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:29 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1673268841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:29.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:29 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2667170176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:29.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:29 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2783221210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:29.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:29 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1673268841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:29.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.928+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff49402d000 con 0x7ff4941015e0 2024-02-19T16:22:29.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.928+0000 7ff4917fa640 1 -- 172.21.15.8:0/2128436249 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4840bc600 con 0x7ff4941015e0 2024-02-19T16:22:29.929 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:29.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff4800a20c0 msgr2=0x7ff4800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff4800a20c0 0x7ff4800a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff494101440 tx=0x7ff47c005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:29.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 msgr2=0x7ff4941969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff4941969b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff484006010 tx=0x7ff484005e80 comp rx=0 tx=0).stop 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 shutdown_connections 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7ff4800a20c0 0x7ff4800a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff494101fa0 0x7ff49419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4941015e0 0x7ff4941969b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.930+0000 7ff49a89d640 1 --2- 172.21.15.8:0/2128436249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4941003f0 0x7ff494196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.931+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 >> 172.21.15.8:0/2128436249 conn(0x7ff4940fb980 msgr2=0x7ff494104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.931+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 shutdown_connections 2024-02-19T16:22:29.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:29.931+0000 7ff49a89d640 1 -- 172.21.15.8:0/2128436249 wait complete. 2024-02-19T16:22:29.943 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:30.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 -- 172.21.15.8:0/528416856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 msgr2=0x7fcd8410b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 --2- 172.21.15.8:0/528416856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd8410b840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009ca0 tx=0x7fcd7402c180 comp rx=0 tx=0).stop 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 -- 172.21.15.8:0/528416856 shutdown_connections 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 --2- 172.21.15.8:0/528416856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd8410b840 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 --2- 172.21.15.8:0/528416856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd84073ed0 0x7fcd84074330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 --2- 172.21.15.8:0/528416856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd840754b0 0x7fcd84073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 -- 172.21.15.8:0/528416856 >> 172.21.15.8:0/528416856 conn(0x7fcd840fca50 msgr2=0x7fcd840feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 -- 172.21.15.8:0/528416856 shutdown_connections 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.047+0000 7fcd89483640 1 -- 172.21.15.8:0/528416856 wait complete. 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 Processor -- start 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 -- start start 2024-02-19T16:22:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd84073ed0 0x7fcd841996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd840754b0 0x7fcd84199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd841a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd841a11e0 con 0x7fcd841093b0 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd841a1350 con 0x7fcd840754b0 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd89483640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd841a1510 con 0x7fcd84073ed0 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd837fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd841a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd837fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd841a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34796/0 (socket says 172.21.15.8:34796) 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd837fe640 1 -- 172.21.15.8:0/245677959 learned_addr learned my addr 172.21.15.8:0/245677959 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.048+0000 7fcd82ffd640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd84073ed0 0x7fcd841996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd827fc640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd840754b0 0x7fcd84199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd837fe640 1 -- 172.21.15.8:0/245677959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd84073ed0 msgr2=0x7fcd841996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd837fe640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd84073ed0 0x7fcd841996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd837fe640 1 -- 172.21.15.8:0/245677959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd840754b0 msgr2=0x7fcd84199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd837fe640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd840754b0 0x7fcd84199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd837fe640 1 -- 172.21.15.8:0/245677959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009490 con 0x7fcd841093b0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd837fe640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd841a0ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcd74037040 tx=0x7fcd74034ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74003d80 con 0x7fcd841093b0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd74032d70 con 0x7fcd841093b0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd74009900 con 0x7fcd841093b0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004860 con 0x7fcd841093b0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.049+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd841a1ab0 con 0x7fcd841093b0 2024-02-19T16:22:30.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.050+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd84108170 con 0x7fcd841093b0 2024-02-19T16:22:30.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.051+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fcd7400a040 con 0x7fcd841093b0 2024-02-19T16:22:30.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.052+0000 7fcd7bfff640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcd700a20c0 0x7fcd700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.052+0000 7fcd82ffd640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcd700a20c0 0x7fcd700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.052+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fcd740f0d40 con 0x7fcd841093b0 2024-02-19T16:22:30.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.052+0000 7fcd82ffd640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcd700a20c0 0x7fcd700a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009bd0 tx=0x7fcd7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.056+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fcd740b9910 con 0x7fcd841093b0 2024-02-19T16:22:30.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.237+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd8402d000 con 0x7fcd841093b0 2024-02-19T16:22:30.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.237+0000 7fcd7bfff640 1 -- 172.21.15.8:0/245677959 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd740bcfd0 con 0x7fcd841093b0 2024-02-19T16:22:30.238 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:30.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcd700a20c0 msgr2=0x7fcd700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcd700a20c0 0x7fcd700a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009bd0 tx=0x7fcd7c005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:30.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 msgr2=0x7fcd841a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd841a0ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcd74037040 tx=0x7fcd74034ea0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 shutdown_connections 2024-02-19T16:22:30.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fcd700a20c0 0x7fcd700a4580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd841093b0 0x7fcd841a0ca0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd840754b0 0x7fcd84199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 --2- 172.21.15.8:0/245677959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd84073ed0 0x7fcd841996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 >> 172.21.15.8:0/245677959 conn(0x7fcd840fca50 msgr2=0x7fcd84105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 shutdown_connections 2024-02-19T16:22:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.239+0000 7fcd89483640 1 -- 172.21.15.8:0/245677959 wait complete. 2024-02-19T16:22:30.251 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:30.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 -- 172.21.15.8:0/2476822824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc0101500 msgr2=0x7f4fc0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 --2- 172.21.15.8:0/2476822824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc0101500 0x7f4fc0101920 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009900 tx=0x7f4fbc02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 -- 172.21.15.8:0/2476822824 shutdown_connections 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 --2- 172.21.15.8:0/2476822824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc01030b0 0x7f4fc0109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 --2- 172.21.15.8:0/2476822824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc01026f0 0x7f4fc0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 --2- 172.21.15.8:0/2476822824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc0101500 0x7f4fc0101920 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 -- 172.21.15.8:0/2476822824 >> 172.21.15.8:0/2476822824 conn(0x7f4fc00fca90 msgr2=0x7f4fc00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 -- 172.21.15.8:0/2476822824 shutdown_connections 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.356+0000 7f4fc6948640 1 -- 172.21.15.8:0/2476822824 wait complete. 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 Processor -- start 2024-02-19T16:22:30.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 -- start start 2024-02-19T16:22:30.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc0101500 0x7f4fc0197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 0x7f4fc0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc01030b0 0x7f4fc019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc019f0f0 con 0x7f4fc01026f0 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc019f260 con 0x7f4fc0101500 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fc6948640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc019f420 con 0x7f4fc01030b0 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 0x7f4fc0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 0x7f4fc0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34816/0 (socket says 172.21.15.8:34816) 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.357+0000 7f4fb77fe640 1 -- 172.21.15.8:0/1377498687 learned_addr learned my addr 172.21.15.8:0/1377498687 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb7fff640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc0101500 0x7f4fc0197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fc4ee5640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc01030b0 0x7f4fc019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb77fe640 1 -- 172.21.15.8:0/1377498687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc01030b0 msgr2=0x7f4fc019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb77fe640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc01030b0 0x7f4fc019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb77fe640 1 -- 172.21.15.8:0/1377498687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc0101500 msgr2=0x7f4fc0197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb77fe640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc0101500 0x7f4fc0197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb77fe640 1 -- 172.21.15.8:0/1377498687 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc009560 con 0x7f4fc01026f0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb77fe640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 0x7f4fc0197b40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e9f0 tx=0x7f4fb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0009700 con 0x7f4fc01026f0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fb000cd80 con 0x7f4fc01026f0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc019f700 con 0x7f4fc01026f0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0018920 con 0x7f4fc01026f0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.358+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc019fc10 con 0x7f4fc01026f0 2024-02-19T16:22:30.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.359+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc0108870 con 0x7f4fc01026f0 2024-02-19T16:22:30.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.360+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4fb0018a80 con 0x7f4fc01026f0 2024-02-19T16:22:30.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.361+0000 7f4fb57fa640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fa80a20c0 0x7f4fa80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.361+0000 7f4fb7fff640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fa80a20c0 0x7f4fa80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.361+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f4fb0014070 con 0x7f4fc01026f0 2024-02-19T16:22:30.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.361+0000 7f4fb7fff640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fa80a20c0 0x7f4fa80a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc005ab0 tx=0x7f4fbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.365+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f4fb0053230 con 0x7f4fc01026f0 2024-02-19T16:22:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:30 smithi008 ceph-mon[32573]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:30 smithi008 ceph-mon[32573]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-02-19T16:22:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:30 smithi008 ceph-mon[32573]: Cluster is now healthy 2024-02-19T16:22:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:30 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2489063147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:30 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2128436249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:30.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:30 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/245677959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:30.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.547+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fc002d000 con 0x7f4fc01026f0 2024-02-19T16:22:30.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.548+0000 7f4fb57fa640 1 -- 172.21.15.8:0/1377498687 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fb0097090 con 0x7f4fc01026f0 2024-02-19T16:22:30.549 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:30.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fa80a20c0 msgr2=0x7f4fa80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fa80a20c0 0x7f4fa80a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc005ab0 tx=0x7f4fbc005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 msgr2=0x7f4fc0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 0x7f4fc0197b40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e9f0 tx=0x7f4fb000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 shutdown_connections 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4fa80a20c0 0x7f4fa80a4580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc01030b0 0x7f4fc019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fc01026f0 0x7f4fc0197b40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 --2- 172.21.15.8:0/1377498687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc0101500 0x7f4fc0197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 >> 172.21.15.8:0/1377498687 conn(0x7f4fc00fca90 msgr2=0x7f4fc0105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 shutdown_connections 2024-02-19T16:22:30.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.550+0000 7f4fc6948640 1 -- 172.21.15.8:0/1377498687 wait complete. 2024-02-19T16:22:30.562 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 -- 172.21.15.8:0/1143744076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c101f70 msgr2=0x7f084c104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 --2- 172.21.15.8:0/1143744076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c101f70 0x7f084c104360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0844009900 tx=0x7f084402bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 -- 172.21.15.8:0/1143744076 shutdown_connections 2024-02-19T16:22:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 --2- 172.21.15.8:0/1143744076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f084c1071d0 0x7f084c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 --2- 172.21.15.8:0/1143744076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f084c1048a0 0x7f084c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 --2- 172.21.15.8:0/1143744076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c101f70 0x7f084c104360 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 -- 172.21.15.8:0/1143744076 >> 172.21.15.8:0/1143744076 conn(0x7f084c0fb980 msgr2=0x7f084c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 -- 172.21.15.8:0/1143744076 shutdown_connections 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.667+0000 7f0851348640 1 -- 172.21.15.8:0/1143744076 wait complete. 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 Processor -- start 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 -- start start 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f084c101f70 0x7f084c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f084c1048a0 0x7f084c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 0x7f084c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c19f0f0 con 0x7f084c1071d0 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c19f260 con 0x7f084c101f70 2024-02-19T16:22:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f0851348640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c19f420 con 0x7f084c1048a0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f084b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 0x7f084c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f084b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 0x7f084c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34836/0 (socket says 172.21.15.8:34836) 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f084b7fe640 1 -- 172.21.15.8:0/2616306809 learned_addr learned my addr 172.21.15.8:0/2616306809 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f084affd640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f084c101f70 0x7f084c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.668+0000 7f084a7fc640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f084c1048a0 0x7f084c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f084b7fe640 1 -- 172.21.15.8:0/2616306809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f084c1048a0 msgr2=0x7f084c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f084b7fe640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f084c1048a0 0x7f084c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f084b7fe640 1 -- 172.21.15.8:0/2616306809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f084c101f70 msgr2=0x7f084c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f084b7fe640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f084c101f70 0x7f084c197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f084b7fe640 1 -- 172.21.15.8:0/2616306809 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844009560 con 0x7f084c1071d0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f084b7fe640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 0x7f084c19ebb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f083c00eb10 tx=0x7f083c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c004b20 con 0x7f084c1071d0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f083c004c80 con 0x7f084c1071d0 2024-02-19T16:22:30.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c19f700 con 0x7f084c1071d0 2024-02-19T16:22:30.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c010650 con 0x7f084c1071d0 2024-02-19T16:22:30.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.669+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084c19fbc0 con 0x7f084c1071d0 2024-02-19T16:22:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.670+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c0fd470 con 0x7f084c1071d0 2024-02-19T16:22:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.671+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f083c005020 con 0x7f084c1071d0 2024-02-19T16:22:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.672+0000 7f0843fff640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f08380a20c0 0x7f08380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.672+0000 7f084affd640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f08380a20c0 0x7f08380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.672+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f083c014070 con 0x7f084c1071d0 2024-02-19T16:22:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.672+0000 7f084affd640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f08380a20c0 0x7f08380a4580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0844009230 tx=0x7f0844005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.676+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f083c097230 con 0x7f084c1071d0 2024-02-19T16:22:30.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:30 smithi191 ceph-mon[37203]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:30 smithi191 ceph-mon[37203]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-02-19T16:22:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:30 smithi191 ceph-mon[37203]: Cluster is now healthy 2024-02-19T16:22:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:30 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2489063147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:30 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2128436249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:30 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/245677959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:30 smithi156 ceph-mon[36502]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:22:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:30 smithi156 ceph-mon[36502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-02-19T16:22:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:30 smithi156 ceph-mon[36502]: Cluster is now healthy 2024-02-19T16:22:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:30 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2489063147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:30 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2128436249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:30 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/245677959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:30.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.859+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f084c02cf60 con 0x7f084c1071d0 2024-02-19T16:22:30.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.859+0000 7f0843fff640 1 -- 172.21.15.8:0/2616306809 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f083c09a8f0 con 0x7f084c1071d0 2024-02-19T16:22:30.860 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:30.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.861+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f08380a20c0 msgr2=0x7f08380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.861+0000 7f0851348640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f08380a20c0 0x7f08380a4580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0844009230 tx=0x7f0844005fd0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.861+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 msgr2=0x7f084c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.861+0000 7f0851348640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 0x7f084c19ebb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f083c00eb10 tx=0x7f083c00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 shutdown_connections 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f08380a20c0 0x7f08380a4580 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f084c1071d0 0x7f084c19ebb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f084c1048a0 0x7f084c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 --2- 172.21.15.8:0/2616306809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f084c101f70 0x7f084c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 >> 172.21.15.8:0/2616306809 conn(0x7f084c0fb980 msgr2=0x7f084c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 shutdown_connections 2024-02-19T16:22:30.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.862+0000 7f0851348640 1 -- 172.21.15.8:0/2616306809 wait complete. 2024-02-19T16:22:30.874 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 -- 172.21.15.8:0/2896934336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 msgr2=0x7f56481088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 --2- 172.21.15.8:0/2896934336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f56481088d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5640009ca0 tx=0x7f564002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 -- 172.21.15.8:0/2896934336 shutdown_connections 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 --2- 172.21.15.8:0/2896934336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f56481088d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 --2- 172.21.15.8:0/2896934336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648101680 0x7f5648101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 --2- 172.21.15.8:0/2896934336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 0x7f56481008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 -- 172.21.15.8:0/2896934336 >> 172.21.15.8:0/2896934336 conn(0x7f56480fba20 msgr2=0x7f56480fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 -- 172.21.15.8:0/2896934336 shutdown_connections 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.975+0000 7f564cf08640 1 -- 172.21.15.8:0/2896934336 wait complete. 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 Processor -- start 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 -- start start 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 0x7f56481975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648101680 0x7f5648197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f564819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564819f0e0 con 0x7f5648102040 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564819f250 con 0x7f5648101680 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564cf08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564819f410 con 0x7f5648100490 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f5646d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f564819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 0x7f56481975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f5646d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f564819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34856/0 (socket says 172.21.15.8:34856) 2024-02-19T16:22:30.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f5646d9d640 1 -- 172.21.15.8:0/3908257036 learned_addr learned my addr 172.21.15.8:0/3908257036 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f563dd9b640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648101680 0x7f5648197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564659c640 1 -- 172.21.15.8:0/3908257036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648101680 msgr2=0x7f5648197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564659c640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648101680 0x7f5648197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564659c640 1 -- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 msgr2=0x7f564819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564659c640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f564819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f564659c640 1 -- 172.21.15.8:0/3908257036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638009490 con 0x7f5648100490 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.976+0000 7f5646d9d640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f564819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.977+0000 7f564659c640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 0x7f56481975f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f563800e9c0 tx=0x7f563800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.977+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800cd80 con 0x7f5648100490 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.977+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5640009900 con 0x7f5648100490 2024-02-19T16:22:30.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.977+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5638010910 con 0x7f5648100490 2024-02-19T16:22:30.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.977+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638018900 con 0x7f5648100490 2024-02-19T16:22:30.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.977+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564819fa70 con 0x7f5648100490 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.978+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564810be00 con 0x7f5648100490 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.979+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f5638010a80 con 0x7f5648100490 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.979+0000 7f563ffff640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f56340a20c0 0x7f56340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.980+0000 7f563dd9b640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f56340a20c0 0x7f56340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.980+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f5638014070 con 0x7f5648100490 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.980+0000 7f563dd9b640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f56340a20c0 0x7f56340a4580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5648198b00 tx=0x7f5630005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:30.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:30.983+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5638076230 con 0x7f5648100490 2024-02-19T16:22:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.206+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f564810bf90 con 0x7f5648100490 2024-02-19T16:22:31.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.207+0000 7f563ffff640 1 -- 172.21.15.8:0/3908257036 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f563809acf0 con 0x7f5648100490 2024-02-19T16:22:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:31.208 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:20:41.315237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:31.208 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-02-19T16:22:31.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f56340a20c0 msgr2=0x7f56340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f56340a20c0 0x7f56340a4580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5648198b00 tx=0x7f5630005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 msgr2=0x7f56481975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 0x7f56481975f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f563800e9c0 tx=0x7f563800ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 shutdown_connections 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f56340a20c0 0x7f56340a4580 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5648102040 0x7f564819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648101680 0x7f5648197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 --2- 172.21.15.8:0/3908257036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5648100490 0x7f56481975f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 >> 172.21.15.8:0/3908257036 conn(0x7f56480fba20 msgr2=0x7f56480fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 shutdown_connections 2024-02-19T16:22:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.210+0000 7f564cf08640 1 -- 172.21.15.8:0/3908257036 wait complete. 2024-02-19T16:22:31.222 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 -- 172.21.15.8:0/2034465214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101fe0 msgr2=0x7fda48108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 --2- 172.21.15.8:0/2034465214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101fe0 0x7fda48108870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ca0 tx=0x7fda3802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 -- 172.21.15.8:0/2034465214 shutdown_connections 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 --2- 172.21.15.8:0/2034465214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101fe0 0x7fda48108870 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 --2- 172.21.15.8:0/2034465214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda48101620 0x7fda48101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 --2- 172.21.15.8:0/2034465214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda48100430 0x7fda48100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 -- 172.21.15.8:0/2034465214 >> 172.21.15.8:0/2034465214 conn(0x7fda480fb9a0 msgr2=0x7fda480fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 -- 172.21.15.8:0/2034465214 shutdown_connections 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 -- 172.21.15.8:0/2034465214 wait complete. 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 Processor -- start 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.323+0000 7fda5015c640 1 -- start start 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda48100430 0x7fda481975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 0x7fda48197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda48101fe0 0x7fda4819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4819f090 con 0x7fda48101620 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4819f200 con 0x7fda48101fe0 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4819f3c0 con 0x7fda48100430 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 0x7fda48197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 0x7fda48197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34880/0 (socket says 172.21.15.8:34880) 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 -- 172.21.15.8:0/2592868927 learned_addr learned my addr 172.21.15.8:0/2592868927 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4def8640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda48100430 0x7fda481975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4e6f9640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda48101fe0 0x7fda4819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 -- 172.21.15.8:0/2592868927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda48100430 msgr2=0x7fda481975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda48100430 0x7fda481975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 -- 172.21.15.8:0/2592868927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda48101fe0 msgr2=0x7fda4819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda48101fe0 0x7fda4819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 -- 172.21.15.8:0/2592868927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009490 con 0x7fda48101620 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda4d6f7640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 0x7fda48197ae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00ea60 tx=0x7fda3c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00cde0 con 0x7fda48101620 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda3c0035f0 con 0x7fda48101620 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda38009900 con 0x7fda48101620 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c010690 con 0x7fda48101620 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.324+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4819fa20 con 0x7fda48101620 2024-02-19T16:22:31.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.325+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda481076f0 con 0x7fda48101620 2024-02-19T16:22:31.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.326+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7fda3c003af0 con 0x7fda48101620 2024-02-19T16:22:31.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.327+0000 7fda46ffd640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda300a20c0 0x7fda300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.327+0000 7fda4def8640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda300a20c0 0x7fda300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.327+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fda3c014070 con 0x7fda48101620 2024-02-19T16:22:31.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.328+0000 7fda4def8640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda300a20c0 0x7fda300a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda48101480 tx=0x7fda40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.331+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fda3c097100 con 0x7fda48101620 2024-02-19T16:22:31.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:31 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1377498687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:31.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:31 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2616306809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:31.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:31 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3908257036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:31.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.520+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda4810cf40 con 0x7fda48101620 2024-02-19T16:22:31.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.522+0000 7fda46ffd640 1 -- 172.21.15.8:0/2592868927 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7fda3c09a7c0 con 0x7fda48101620 2024-02-19T16:22:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:31.523 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda300a20c0 msgr2=0x7fda300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda300a20c0 0x7fda300a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda48101480 tx=0x7fda40005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 msgr2=0x7fda48197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 0x7fda48197ae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00ea60 tx=0x7fda3c00ef20 comp rx=0 tx=0).stop 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 shutdown_connections 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fda300a20c0 0x7fda300a4580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda48101fe0 0x7fda4819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda48101620 0x7fda48197ae0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 --2- 172.21.15.8:0/2592868927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda48100430 0x7fda481975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 >> 172.21.15.8:0/2592868927 conn(0x7fda480fb9a0 msgr2=0x7fda481048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 shutdown_connections 2024-02-19T16:22:31.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.524+0000 7fda5015c640 1 -- 172.21.15.8:0/2592868927 wait complete. 2024-02-19T16:22:31.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-19T16:20:21.002558+0000', 'modified': '2024-02-19T16:20:41.315237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24502, 'mds_1': 24488, 'mds_2': 24475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6833/2702528663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 2702528663}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 2702528663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/3993132637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3993132637}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3993132637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/562441805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 562441805}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 562441805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.191:6837/1523568014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 1523568014}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1523568014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6839/3057541491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 3057541491}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 3057541491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.191:6839/2229091821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2229091821}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2229091821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-19T16:22:31.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-19T16:22:31.537 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 -- 172.21.15.8:0/999030646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3590101500 msgr2=0x7f3590101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 --2- 172.21.15.8:0/999030646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3590101500 0x7f3590101920 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f358c009900 tx=0x7f358c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 -- 172.21.15.8:0/999030646 shutdown_connections 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 --2- 172.21.15.8:0/999030646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35901030b0 0x7f3590109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 --2- 172.21.15.8:0/999030646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35901026f0 0x7f3590102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 --2- 172.21.15.8:0/999030646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3590101500 0x7f3590101920 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 -- 172.21.15.8:0/999030646 >> 172.21.15.8:0/999030646 conn(0x7f35900fca90 msgr2=0x7f35900feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 -- 172.21.15.8:0/999030646 shutdown_connections 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.640+0000 7f3597283640 1 -- 172.21.15.8:0/999030646 wait complete. 2024-02-19T16:22:31.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 Processor -- start 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 -- start start 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3590101500 0x7f35901975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 0x7f3590197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35901030b0 0x7f359019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359019f0d0 con 0x7f35901026f0 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359019f240 con 0x7f3590101500 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3597283640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359019f400 con 0x7f35901030b0 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 0x7f3590197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 0x7f3590197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34896/0 (socket says 172.21.15.8:34896) 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 -- 172.21.15.8:0/1807931358 learned_addr learned my addr 172.21.15.8:0/1807931358 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359501f640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3590101500 0x7f35901975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f3595820640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35901030b0 0x7f359019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 -- 172.21.15.8:0/1807931358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35901030b0 msgr2=0x7f359019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35901030b0 0x7f359019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 -- 172.21.15.8:0/1807931358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3590101500 msgr2=0x7f35901975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3590101500 0x7f35901975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 -- 172.21.15.8:0/1807931358 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c009560 con 0x7f35901026f0 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.641+0000 7f359481e640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 0x7f3590197b20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f358000e9f0 tx=0x7f358000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.642+0000 7f35867fc640 1 -- 172.21.15.8:0/1807931358 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580009700 con 0x7f35901026f0 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.642+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359019f6e0 con 0x7f35901026f0 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.642+0000 7f35867fc640 1 -- 172.21.15.8:0/1807931358 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f358000cd80 con 0x7f35901026f0 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.642+0000 7f35867fc640 1 -- 172.21.15.8:0/1807931358 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580018920 con 0x7f35901026f0 2024-02-19T16:22:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.642+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359019fbf0 con 0x7f35901026f0 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f35867fc640 1 -- 172.21.15.8:0/1807931358 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f3580018a80 con 0x7f35901026f0 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 Processor -- start 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 -- start start 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7ac970 0x559c7a7b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a767cd0 con 0x559c7a7b1ae0 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a71af20 con 0x559c7a7abf80 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3599e56740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a718450 con 0x559c7a7ac970 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359481e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359481e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34906/0 (socket says 172.21.15.8:34906) 2024-02-19T16:22:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359481e640 1 -- 172.21.15.8:0/1050276719 learned_addr learned my addr 172.21.15.8:0/1050276719 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359501f640 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7ac970 0x559c7a7b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3595820640 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f35867fc640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f357c0a20c0 0x7f357c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f3595820640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f357c0a20c0 0x7f357c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359501f640 1 -- 172.21.15.8:0/1050276719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 msgr2=0x559c7a7ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359501f640 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359501f640 1 -- 172.21.15.8:0/1050276719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 msgr2=0x559c7a7b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359501f640 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f35867fc640 1 -- 172.21.15.8:0/1807931358 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f3580014070 con 0x7f35901026f0 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359501f640 1 -- 172.21.15.8:0/1050276719 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7a6c8f20 con 0x559c7a7ac970 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.645+0000 7f359481e640 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f359501f640 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7ac970 0x559c7a7b1470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f358c036040 tx=0x7f358c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bf95e65317a6af server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f3595820640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f357c0a20c0 0x7f357c0a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x559c7a7ad1d0 tx=0x7f3588004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f357b7fe640 1 -- 172.21.15.8:0/1050276719 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c02ca20 con 0x559c7a7ac970 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f357b7fe640 1 -- 172.21.15.8:0/1050276719 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f358c02cb80 con 0x559c7a7ac970 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f357b7fe640 1 -- 172.21.15.8:0/1050276719 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c005210 con 0x559c7a7ac970 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f3599e56740 1 -- 172.21.15.8:0/1050276719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7ac970 msgr2=0x559c7a7b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.646+0000 7f3599e56740 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7ac970 0x559c7a7b1470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f358c036040 tx=0x7f358c005d50 comp rx=0 tx=0).stop 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- 172.21.15.8:0/1050276719 shutdown_connections 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7ac970 0x559c7a7b1470 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 --2- 172.21.15.8:0/1050276719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- 172.21.15.8:0/1050276719 >> 172.21.15.8:0/1050276719 conn(0x559c7a7a74f0 msgr2=0x559c7a7a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- 172.21.15.8:0/1050276719 shutdown_connections 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- 172.21.15.8:0/1050276719 wait complete. 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- start start 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 Processor -- start 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7bd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a713c30 con 0x559c7a7abf80 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a6b8d30 con 0x559c7a7ac970 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a3e7310 con 0x559c7a7b1ae0 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f359501f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.647+0000 7f3599e56740 10 client.0 _ll_register_callbacks cb 0x559c7a6c0d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34908/0 (socket says 172.21.15.8:34908) 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 -- 172.21.15.8:0/1832854700 learned_addr learned my addr 172.21.15.8:0/1832854700 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3595820640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7bd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359481e640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 -- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 msgr2=0x559c7a7bd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7bd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 -- 172.21.15.8:0/1832854700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 msgr2=0x559c7a7b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 -- 172.21.15.8:0/1832854700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359010a240 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f359501f640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7b6110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f358c009900 tx=0x7f358c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78e9af4e886e2e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3578ff9640 1 -- 172.21.15.8:0/1832854700 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c02ca20 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3578ff9640 1 -- 172.21.15.8:0/1832854700 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f358c04d070 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c7a4136c0 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3578ff9640 1 -- 172.21.15.8:0/1832854700 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f358c004ae0 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3599e56740 10 client.15387 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.648+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559c7a7122b0 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3578ff9640 1 -- 172.21.15.8:0/1832854700 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559c7a4136c0 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3578ff9640 1 -- 172.21.15.8:0/1832854700 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f358c03c050 con 0x559c7a7abf80 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3599e56740 10 client.15387 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3599e56740 10 client.15387 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-19T16:22:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3599e56740 20 client.15387 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3599e56740 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a7c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3599e56740 4 client.15387 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c7a464270 con 0x559c7a7c2630 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f359481e640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a7c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f3578ff9640 10 client.15387 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.649+0000 7f359481e640 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a7c6ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d7880fa2ed5d7cb server_cookie=9426d201b0b21e17 in_seq=0 out_seq=0 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3578ff9640 1 -- 172.21.15.8:0/1832854700 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559c7a464270 con 0x559c7a7c2630 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3578ff9640 10 client.15387 handle_command_reply: tid=0 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 client.15387 shutdown 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 msgr2=0x559c7a7b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7b6110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f358c009900 tx=0x7f358c0048f0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 shutdown_connections 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a7c6ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7bd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3578ff9640 0 client.15387 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 --2- 172.21.15.8:0/1832854700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7b6110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 >> 172.21.15.8:0/1832854700 conn(0x559c7a7a74f0 msgr2=0x559c7a7a67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 shutdown_connections 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 1 -- 172.21.15.8:0/1832854700 wait complete. 2024-02-19T16:22:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.650+0000 7f3599e56740 20 client.15387 trim_cache size 0 max 16384 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 Processor -- start 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 -- start start 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 0x559c7a7bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a699490 con 0x559c7a7ac970 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a713c30 con 0x559c7a7abf80 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3599e56740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a7b4eb0 con 0x559c7a7b1ae0 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f359481e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 0x559c7a7bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f359481e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 0x559c7a7bd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34912/0 (socket says 172.21.15.8:34912) 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f359481e640 1 -- 172.21.15.8:0/1805912879 learned_addr learned my addr 172.21.15.8:0/1805912879 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.667+0000 7f3595820640 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359501f640 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359481e640 1 -- 172.21.15.8:0/1805912879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 msgr2=0x559c7a7b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359481e640 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359481e640 1 -- 172.21.15.8:0/1805912879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 msgr2=0x559c7a7aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359481e640 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359481e640 1 -- 172.21.15.8:0/1805912879 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7a6aea20 con 0x559c7a7ac970 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f359481e640 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 0x559c7a7bd650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f35800977e0 tx=0x7f35800d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2450725329c22d8a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f357affd640 1 -- 172.21.15.8:0/1805912879 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358009a770 con 0x559c7a7ac970 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f357affd640 1 -- 172.21.15.8:0/1805912879 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f358009ae10 con 0x559c7a7ac970 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.668+0000 7f357affd640 1 -- 172.21.15.8:0/1805912879 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580097480 con 0x559c7a7ac970 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- 172.21.15.8:0/1805912879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 msgr2=0x559c7a7bd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 0x559c7a7bd650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f35800977e0 tx=0x7f35800d0040 comp rx=0 tx=0).stop 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- 172.21.15.8:0/1805912879 shutdown_connections 2024-02-19T16:22:31.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a7b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7ac970 0x559c7a7bd650 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- 172.21.15.8:0/1805912879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7abf80 0x559c7a7aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- 172.21.15.8:0/1805912879 >> 172.21.15.8:0/1805912879 conn(0x559c7a7a74f0 msgr2=0x559c7a7aa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- 172.21.15.8:0/1805912879 shutdown_connections 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- 172.21.15.8:0/1805912879 wait complete. 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- start start 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 Processor -- start 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7ba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a81dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a7be310 con 0x559c7a7abf80 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a7b16c0 con 0x559c7a7ac970 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7a7c2200 con 0x559c7a7b1ae0 2024-02-19T16:22:31.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.669+0000 7f3599e56740 10 client.0 _ll_register_callbacks cb 0x7f3580097130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7ba2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34924/0 (socket says 172.21.15.8:34924) 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 -- 172.21.15.8:0/2656226469 learned_addr learned my addr 172.21.15.8:0/2656226469 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359481e640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7ba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3595820640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a81dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 -- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 msgr2=0x559c7a81dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a81dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 -- 172.21.15.8:0/2656226469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 msgr2=0x559c7a7ba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7ba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 -- 172.21.15.8:0/2656226469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359019f6e0 con 0x559c7a7abf80 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359481e640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f359501f640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7ba2a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x559c7a7a8430 tx=0x7f358c03f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8671255cc2d577af server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3578ff9640 1 -- 172.21.15.8:0/2656226469 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0053a0 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3578ff9640 1 -- 172.21.15.8:0/2656226469 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f358c04d040 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c7a7c2340 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3578ff9640 1 -- 172.21.15.8:0/2656226469 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f358c004df0 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3599e56740 10 client.15393 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559c7a6d4780 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.670+0000 7f3578ff9640 1 -- 172.21.15.8:0/2656226469 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559c7a7c2340 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3578ff9640 1 -- 172.21.15.8:0/2656226469 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f358c03c040 con 0x559c7a7abf80 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 10 client.15393 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 10 client.15393 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 20 client.15393 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a81ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 4 client.15393 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f358c02cb80 con 0x559c7a7c2630 2024-02-19T16:22:31.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f359481e640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a81ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f359481e640 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a81ef50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99bd05d42aa1d62b server_cookie=be3edbd928aff20 in_seq=0 out_seq=0 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3578ff9640 10 client.15393 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3578ff9640 1 -- 172.21.15.8:0/2656226469 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f358c02cb80 con 0x559c7a7c2630 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3578ff9640 10 client.15393 handle_command_reply: tid=0 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 1 client.15393 shutdown 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 msgr2=0x559c7a7ba2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.671+0000 7f3599e56740 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7ba2a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x559c7a7a8430 tx=0x7f358c03f4c0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 shutdown_connections 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x559c7a7c2630 0x559c7a81ef50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c7a7b1ae0 0x559c7a81dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3578ff9640 0 client.15393 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c7a7ac970 0x559c7a7ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 --2- 172.21.15.8:0/2656226469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c7a7abf80 0x559c7a7ba2a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 >> 172.21.15.8:0/2656226469 conn(0x559c7a7a74f0 msgr2=0x559c7a7aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 shutdown_connections 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 1 -- 172.21.15.8:0/2656226469 wait complete. 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3599e56740 20 client.15393 trim_cache size 0 max 16384 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 2, 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24475, 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:22:31.680 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:22:31.681 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 110.87271940700001, 2024-02-19T16:22:31.681 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 23, 2024-02-19T16:22:31.681 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 95, 2024-02-19T16:22:31.681 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 91, 2024-02-19T16:22:31.682 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 148.293315552 2024-02-19T16:22:31.682 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:31.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f357c0a20c0 msgr2=0x7f357c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f357c0a20c0 0x7f357c0a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x559c7a7ad1d0 tx=0x7f3588004ab0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 msgr2=0x7f3590197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 0x7f3590197b20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f358000e9f0 tx=0x7f358000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 shutdown_connections 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f357c0a20c0 0x7f357c0a4580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35901030b0 0x7f359019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35901026f0 0x7f3590197b20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 --2- 172.21.15.8:0/1807931358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3590101500 0x7f35901975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 >> 172.21.15.8:0/1807931358 conn(0x7f35900fca90 msgr2=0x7f3590105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 shutdown_connections 2024-02-19T16:22:31.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.672+0000 7f3597283640 1 -- 172.21.15.8:0/1807931358 wait complete. 2024-02-19T16:22:31.684 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-19T16:22:31.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:31 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1377498687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:31.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:31 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2616306809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:31.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:31 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3908257036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:31.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 -- 172.21.15.8:0/129801271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c100490 msgr2=0x7f417c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 --2- 172.21.15.8:0/129801271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c100490 0x7f417c1008b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4178009900 tx=0x7f417802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 -- 172.21.15.8:0/129801271 shutdown_connections 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 --2- 172.21.15.8:0/129801271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c102040 0x7f417c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 --2- 172.21.15.8:0/129801271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f417c101680 0x7f417c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 --2- 172.21.15.8:0/129801271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c100490 0x7f417c1008b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 -- 172.21.15.8:0/129801271 >> 172.21.15.8:0/129801271 conn(0x7f417c0fba00 msgr2=0x7f417c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 -- 172.21.15.8:0/129801271 shutdown_connections 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 -- 172.21.15.8:0/129801271 wait complete. 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 Processor -- start 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.787+0000 7f4183b25640 1 -- start start 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c100490 0x7f417c10ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f417c101680 0x7f417c109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 0x7f417c1098d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c10a030 con 0x7f417c102040 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c10a1a0 con 0x7f417c100490 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c1a1690 con 0x7f417c101680 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 0x7f417c1098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 0x7f417c1098d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34944/0 (socket says 172.21.15.8:34944) 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41810c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f417c101680 0x7f417c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 -- 172.21.15.8:0/1002535677 learned_addr learned my addr 172.21.15.8:0/1002535677 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41818c1640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c100490 0x7f417c10ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 -- 172.21.15.8:0/1002535677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f417c101680 msgr2=0x7f417c109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f417c101680 0x7f417c109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 -- 172.21.15.8:0/1002535677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c100490 msgr2=0x7f417c10ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c100490 0x7f417c10ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 -- 172.21.15.8:0/1002535677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178009560 con 0x7f417c102040 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f41820c2640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 0x7f417c1098d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f417400eb40 tx=0x7f417400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4172ffd640 1 -- 172.21.15.8:0/1002535677 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174004b20 con 0x7f417c102040 2024-02-19T16:22:31.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4172ffd640 1 -- 172.21.15.8:0/1002535677 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4174004c80 con 0x7f417c102040 2024-02-19T16:22:31.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c1a1940 con 0x7f417c102040 2024-02-19T16:22:31.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4172ffd640 1 -- 172.21.15.8:0/1002535677 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174010670 con 0x7f417c102040 2024-02-19T16:22:31.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.788+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c1a1e50 con 0x7f417c102040 2024-02-19T16:22:31.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.790+0000 7f4172ffd640 1 -- 172.21.15.8:0/1002535677 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f4174005020 con 0x7f417c102040 2024-02-19T16:22:31.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 Processor -- start 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 -- start start 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f9c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fa1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83fa3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83f57cd0 con 0x55be83fa1ae0 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83f0af20 con 0x55be83f9bf80 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41867da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83f08450 con 0x55be83f9c970 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f4172ffd640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41680a2140 0x7f41680a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41810c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f9c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41810c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f9c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:44850/0 (socket says 172.21.15.8:44850) 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41810c0640 1 -- 172.21.15.8:0/4258974073 learned_addr learned my addr 172.21.15.8:0/4258974073 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41818c1640 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fa1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.791+0000 7f41810c0640 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83fa3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41820c2640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41680a2140 0x7f41680a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41810c0640 1 -- 172.21.15.8:0/4258974073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 msgr2=0x55be83fa1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41810c0640 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fa1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f4172ffd640 1 -- 172.21.15.8:0/1002535677 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f4174014070 con 0x7f417c102040 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41810c0640 1 -- 172.21.15.8:0/4258974073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 msgr2=0x55be83fa3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41810c0640 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83fa3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41810c0640 1 -- 172.21.15.8:0/4258974073 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be83eb8f20 con 0x55be83f9bf80 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41810c0640 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f9c3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f416c011810 tx=0x7f416c0097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580958803afeb78c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f41820c2640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41680a2140 0x7f41680a4600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f417409abe0 tx=0x7f417401b6d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f4167fff640 1 -- 172.21.15.8:0/4258974073 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004030 con 0x55be83f9bf80 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f4167fff640 1 -- 172.21.15.8:0/4258974073 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f416c00f7c0 con 0x55be83f9bf80 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.792+0000 7f4167fff640 1 -- 172.21.15.8:0/4258974073 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c003910 con 0x55be83f9bf80 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 -- 172.21.15.8:0/4258974073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 msgr2=0x55be83f9c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f9c3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f416c011810 tx=0x7f416c0097d0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 -- 172.21.15.8:0/4258974073 shutdown_connections 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83fa3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fa1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 --2- 172.21.15.8:0/4258974073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f9c3a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 -- 172.21.15.8:0/4258974073 >> 172.21.15.8:0/4258974073 conn(0x55be83f974f0 msgr2=0x55be83f978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 -- 172.21.15.8:0/4258974073 shutdown_connections 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 -- 172.21.15.8:0/4258974073 wait complete. 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 -- start start 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.793+0000 7f41867da740 1 Processor -- start 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 0x55be83fa6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9c970 0x55be83fa6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83fa1ae0 0x55be83fad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83f03c30 con 0x55be83f9bf80 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83ea8d30 con 0x55be83f9c970 2024-02-19T16:22:31.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83bd7310 con 0x55be83fa1ae0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 10 client.0 _ll_register_callbacks cb 0x55be83eb0d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 0x55be83fa6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 0x55be83fa6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34960/0 (socket says 172.21.15.8:34960) 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 -- 172.21.15.8:0/3186785454 learned_addr learned my addr 172.21.15.8:0/3186785454 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41818c1640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9c970 0x55be83fa6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83fa1ae0 0x55be83fad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 -- 172.21.15.8:0/3186785454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83fa1ae0 msgr2=0x55be83fad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83fa1ae0 0x55be83fad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 -- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9c970 msgr2=0x55be83fa6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9c970 0x55be83fa6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 -- 172.21.15.8:0/3186785454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c019040 con 0x55be83f9bf80 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41818c1640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9c970 0x55be83fa6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41810c0640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 0x55be83fa6130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f416c0117e0 tx=0x7f416c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85f29583dadab84 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41657fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:31.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41657fa640 1 -- 172.21.15.8:0/3186785454 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00fda0 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41657fa640 1 -- 172.21.15.8:0/3186785454 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f416c005540 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.794+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be83c036c0 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41657fa640 1 -- 172.21.15.8:0/3186785454 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f416c0046c0 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 10 client.15405 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55be83f022b0 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41657fa640 1 -- 172.21.15.8:0/3186785454 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55be83c036c0 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41657fa640 1 -- 172.21.15.8:0/3186785454 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f416c02f8c0 con 0x55be83f9bf80 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 10 client.15405 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 10 client.15405 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 20 client.15405 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be83fb2380 0x55be83fb6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 4 client.15405 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be83c54270 con 0x55be83fb2380 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.795+0000 7f41818c1640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be83fb2380 0x55be83fb6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41818c1640 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be83fb2380 0x55be83fb6e00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a86d8b543df8fc3c server_cookie=c737dd24ac52f6e3 in_seq=0 out_seq=0 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41657fa640 10 client.15405 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41657fa640 1 -- 172.21.15.8:0/3186785454 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be83c54270 con 0x55be83fb2380 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41657fa640 10 client.15405 handle_command_reply: tid=0 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41867da740 1 client.15405 shutdown 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 msgr2=0x55be83fa6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41867da740 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 0x55be83fa6130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f416c0117e0 tx=0x7f416c009620 comp rx=0 tx=0).stop 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 shutdown_connections 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41867da740 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be83fb2380 0x55be83fb6e00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.796+0000 7f41867da740 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83fa1ae0 0x55be83fad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41657fa640 0 client.15405 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41867da740 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9c970 0x55be83fa6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41867da740 1 --2- 172.21.15.8:0/3186785454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9bf80 0x55be83fa6130 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 >> 172.21.15.8:0/3186785454 conn(0x55be83f974f0 msgr2=0x55be83f967d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 shutdown_connections 2024-02-19T16:22:31.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41867da740 1 -- 172.21.15.8:0/3186785454 wait complete. 2024-02-19T16:22:31.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.797+0000 7f41867da740 20 client.15405 trim_cache size 0 max 16384 2024-02-19T16:22:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.813+0000 7f41867da740 1 Processor -- start 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.813+0000 7f41867da740 1 -- start start 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41867da740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f95f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41867da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41867da740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83f96480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41867da740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83e89490 con 0x55be83fa1ae0 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41867da740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83c54270 con 0x55be83f9bf80 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41867da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83f6d8c0 con 0x55be83f9c970 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83f96480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83f96480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34970/0 (socket says 172.21.15.8:34970) 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 -- 172.21.15.8:0/2631082156 learned_addr learned my addr 172.21.15.8:0/2631082156 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41818c1640 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f95f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 -- 172.21.15.8:0/2631082156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 msgr2=0x55be83fad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 -- 172.21.15.8:0/2631082156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 msgr2=0x55be83f95f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f95f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 -- 172.21.15.8:0/2631082156 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be83fb2070 con 0x55be83fa1ae0 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41810c0640 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83f96480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f416c005480 tx=0x7f416c0115b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193b82503ee3384e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41677fe640 1 -- 172.21.15.8:0/2631082156 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c02fe90 con 0x55be83fa1ae0 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41677fe640 1 -- 172.21.15.8:0/2631082156 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f416c00fd80 con 0x55be83fa1ae0 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.814+0000 7f41677fe640 1 -- 172.21.15.8:0/2631082156 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c02f6e0 con 0x55be83fa1ae0 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 -- 172.21.15.8:0/2631082156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 msgr2=0x55be83f96480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83f96480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f416c005480 tx=0x7f416c0115b0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 -- 172.21.15.8:0/2631082156 shutdown_connections 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83fa1ae0 0x55be83f96480 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9c970 0x55be83fad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 --2- 172.21.15.8:0/2631082156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83f9bf80 0x55be83f95f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 -- 172.21.15.8:0/2631082156 >> 172.21.15.8:0/2631082156 conn(0x55be83f974f0 msgr2=0x55be83f9de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 -- 172.21.15.8:0/2631082156 shutdown_connections 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 -- 172.21.15.8:0/2631082156 wait complete. 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 -- start start 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.815+0000 7f41867da740 1 Processor -- start 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9bf80 0x55be83fa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 0x55be83fa9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83fa1ae0 0x55be83fac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83fac5a0 con 0x55be83f9c970 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83fac710 con 0x55be83fa1ae0 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83fac8d0 con 0x55be83f9bf80 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 10 client.0 _ll_register_callbacks cb 0x7f416c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 0x55be83fa9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 0x55be83fa9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34982/0 (socket says 172.21.15.8:34982) 2024-02-19T16:22:31.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41810c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9bf80 0x55be83fa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 -- 172.21.15.8:0/2493529375 learned_addr learned my addr 172.21.15.8:0/2493529375 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41810c0640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83fa1ae0 0x55be83fac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 -- 172.21.15.8:0/2493529375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9bf80 msgr2=0x55be83fa8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9bf80 0x55be83fa8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 -- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83fa1ae0 msgr2=0x55be83fac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83fa1ae0 0x55be83fac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 -- 172.21.15.8:0/2493529375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be83f022b0 con 0x55be83f9c970 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41810c0640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83fa1ae0 0x55be83fac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41818c1640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 0x55be83fa9040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f417802bc70 tx=0x7f41780092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeea72f468d165e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41657fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41657fa640 1 -- 172.21.15.8:0/2493529375 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178004040 con 0x55be83f9c970 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41657fa640 1 -- 172.21.15.8:0/2493529375 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f4178046040 con 0x55be83f9c970 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.816+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f416c00f7c0 con 0x55be83f9c970 2024-02-19T16:22:31.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41657fa640 1 -- 172.21.15.8:0/2493529375 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41780041a0 con 0x55be83f9c970 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 10 client.15411 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55be83e9ea20 con 0x55be83f9c970 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41657fa640 1 -- 172.21.15.8:0/2493529375 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f416c00f7c0 con 0x55be83f9c970 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41657fa640 1 -- 172.21.15.8:0/2493529375 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f417803c040 con 0x55be83f9c970 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 10 client.15411 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 10 client.15411 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 20 client.15411 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be84011f90 0x55be8400e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 4 client.15411 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be83c40610 con 0x55be84011f90 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41810c0640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be84011f90 0x55be8400e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41657fa640 10 client.15411 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.817+0000 7f41810c0640 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be84011f90 0x55be8400e3f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f691e9a9f0cee26 server_cookie=26249eefc3eb2a27 in_seq=0 out_seq=0 2024-02-19T16:22:31.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41657fa640 1 -- 172.21.15.8:0/2493529375 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55be83c40610 con 0x55be84011f90 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41657fa640 10 client.15411 handle_command_reply: tid=0 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 client.15411 shutdown 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 msgr2=0x55be83fa9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 0x55be83fa9040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f417802bc70 tx=0x7f41780092d0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 shutdown_connections 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55be84011f90 0x55be8400e3f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55be83fa1ae0 0x55be83fac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55be83f9c970 0x55be83fa9040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 --2- 172.21.15.8:0/2493529375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be83f9bf80 0x55be83fa8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41657fa640 0 client.15411 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 >> 172.21.15.8:0/2493529375 conn(0x55be83f974f0 msgr2=0x55be83f9afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 shutdown_connections 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 1 -- 172.21.15.8:0/2493529375 wait complete. 2024-02-19T16:22:31.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f41867da740 20 client.15411 trim_cache size 0 max 16384 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 1, 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24488, 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 112.02680870899999, 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 23, 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 95, 2024-02-19T16:22:31.827 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 91, 2024-02-19T16:22:31.828 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 134.31742008200001 2024-02-19T16:22:31.828 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:31.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41680a2140 msgr2=0x7f41680a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f4183b25640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41680a2140 0x7f41680a4600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f417409abe0 tx=0x7f417401b6d0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 msgr2=0x7f417c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.818+0000 7f4183b25640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 0x7f417c1098d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f417400eb40 tx=0x7f417400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 shutdown_connections 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41680a2140 0x7f41680a4600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f417c102040 0x7f417c1098d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f417c101680 0x7f417c109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 --2- 172.21.15.8:0/1002535677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c100490 0x7f417c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 >> 172.21.15.8:0/1002535677 conn(0x7f417c0fba00 msgr2=0x7f417c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 shutdown_connections 2024-02-19T16:22:31.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.819+0000 7f4183b25640 1 -- 172.21.15.8:0/1002535677 wait complete. 2024-02-19T16:22:31.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:31 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1377498687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:31.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:31 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2616306809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:31.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:31 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3908257036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:31.831 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-19T16:22:31.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 -- 172.21.15.8:0/886940740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c100400 msgr2=0x7faa9c100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 --2- 172.21.15.8:0/886940740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c100400 0x7faa9c100820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faa94009900 tx=0x7faa9402bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:31.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 -- 172.21.15.8:0/886940740 shutdown_connections 2024-02-19T16:22:31.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 --2- 172.21.15.8:0/886940740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa9c101fb0 0x7faa9c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 --2- 172.21.15.8:0/886940740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa9c1015f0 0x7faa9c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 --2- 172.21.15.8:0/886940740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c100400 0x7faa9c100820 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 -- 172.21.15.8:0/886940740 >> 172.21.15.8:0/886940740 conn(0x7faa9c0fb990 msgr2=0x7faa9c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 -- 172.21.15.8:0/886940740 shutdown_connections 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.934+0000 7faaa3360640 1 -- 172.21.15.8:0/886940740 wait complete. 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 Processor -- start 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 -- start start 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa9c100400 0x7faa9c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 0x7faa9c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa9c101fb0 0x7faa9c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c19f0d0 con 0x7faa9c1015f0 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c19f240 con 0x7faa9c101fb0 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c19f400 con 0x7faa9c100400 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 0x7faa9c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 0x7faa9c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35004/0 (socket says 172.21.15.8:35004) 2024-02-19T16:22:31.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 -- 172.21.15.8:0/2924691140 learned_addr learned my addr 172.21.15.8:0/2924691140 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa10fc640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa9c100400 0x7faa9c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa18fd640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa9c101fb0 0x7faa9c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 -- 172.21.15.8:0/2924691140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa9c100400 msgr2=0x7faa9c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa9c100400 0x7faa9c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 -- 172.21.15.8:0/2924691140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa9c101fb0 msgr2=0x7faa9c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa9c101fb0 0x7faa9c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 -- 172.21.15.8:0/2924691140 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90009490 con 0x7faa9c1015f0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa08fb640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 0x7faa9c197b20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa9000ea60 tx=0x7faa9000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faa9a7fc640 1 -- 172.21.15.8:0/2924691140 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000cde0 con 0x7faa9c1015f0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faa9a7fc640 1 -- 172.21.15.8:0/2924691140 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa900035f0 con 0x7faa9c1015f0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.935+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa94009560 con 0x7faa9c1015f0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.936+0000 7faa9a7fc640 1 -- 172.21.15.8:0/2924691140 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90010690 con 0x7faa9c1015f0 2024-02-19T16:22:31.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.936+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c19fab0 con 0x7faa9c1015f0 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faa9a7fc640 1 -- 172.21.15.8:0/2924691140 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7faa90003af0 con 0x7faa9c1015f0 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 Processor -- start 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 -- start start 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x5578056943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x557805699470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780569bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780564fcd0 con 0x557805693f80 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557805602f20 con 0x557805694970 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faaa600f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557805600450 con 0x557805699ae0 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.938+0000 7faa9a7fc640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa840a2140 0x7faa840a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa18fd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x5578056943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x557805699470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x557805699470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:44942/0 (socket says 172.21.15.8:44942) 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa18fd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x5578056943a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35020/0 (socket says 172.21.15.8:35020) 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 -- 172.21.15.8:0/3133504024 learned_addr learned my addr 172.21.15.8:0/3133504024 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa08fb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780569bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa18fd640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa840a2140 0x7faa840a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faa9a7fc640 1 -- 172.21.15.8:0/2924691140 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7faa90014070 con 0x7faa9c1015f0 2024-02-19T16:22:31.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 -- 172.21.15.8:0/3133504024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 msgr2=0x55780569bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780569bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 -- 172.21.15.8:0/3133504024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 msgr2=0x5578056943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x5578056943a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 -- 172.21.15.8:0/3133504024 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578055b0f20 con 0x557805694970 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa18fd640 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x5578056943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa18fd640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa840a2140 0x7faa840a4600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5578056951d0 tx=0x7faa8c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faaa10fc640 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x557805699470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa94009900 tx=0x7faa94005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e8572c67c768f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faa837fe640 1 -- 172.21.15.8:0/3133504024 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9402c960 con 0x557805694970 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faa837fe640 1 -- 172.21.15.8:0/3133504024 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa9402cac0 con 0x557805694970 2024-02-19T16:22:31.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.939+0000 7faa837fe640 1 -- 172.21.15.8:0/3133504024 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94005010 con 0x557805694970 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 -- 172.21.15.8:0/3133504024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 msgr2=0x557805699470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x557805699470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa94009900 tx=0x7faa94005d50 comp rx=0 tx=0).stop 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 -- 172.21.15.8:0/3133504024 shutdown_connections 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780569bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x557805699470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 --2- 172.21.15.8:0/3133504024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x5578056943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 -- 172.21.15.8:0/3133504024 >> 172.21.15.8:0/3133504024 conn(0x55780568f4f0 msgr2=0x55780568f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 -- 172.21.15.8:0/3133504024 shutdown_connections 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 -- 172.21.15.8:0/3133504024 wait complete. 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 -- start start 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.940+0000 7faaa600f740 1 Processor -- start 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780569e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805694970 0x55780569e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805699ae0 0x5578056a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578055fbc30 con 0x557805693f80 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578055a0d30 con 0x557805699ae0 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578052cf310 con 0x557805694970 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa600f740 10 client.0 _ll_register_callbacks cb 0x5578055a8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780569e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780569e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35024/0 (socket says 172.21.15.8:35024) 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 -- 172.21.15.8:0/735072035 learned_addr learned my addr 172.21.15.8:0/735072035 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa18fd640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805699ae0 0x5578056a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa08fb640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805694970 0x55780569e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 -- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805694970 msgr2=0x55780569e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805694970 0x55780569e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 -- 172.21.15.8:0/735072035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805699ae0 msgr2=0x5578056a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805699ae0 0x5578056a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 -- 172.21.15.8:0/735072035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c069e10 con 0x557805693f80 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faaa10fc640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780569e0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faa94000c00 tx=0x7faa940050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1e6e7a87566885 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.941+0000 7faa80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faa80ff9640 1 -- 172.21.15.8:0/735072035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9402c960 con 0x557805693f80 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faa80ff9640 1 -- 172.21.15.8:0/735072035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7faa9404d070 con 0x557805693f80 2024-02-19T16:22:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578052fb6c0 con 0x557805693f80 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faa80ff9640 1 -- 172.21.15.8:0/735072035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa9402cc30 con 0x557805693f80 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 10 client.15423 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5578055fa2b0 con 0x557805693f80 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faa80ff9640 1 -- 172.21.15.8:0/735072035 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5578052fb6c0 con 0x557805693f80 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faa80ff9640 1 -- 172.21.15.8:0/735072035 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa9403c050 con 0x557805693f80 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 10 client.15423 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 10 client.15423 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 20 client.15423 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5578056aa6d0 0x5578056aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 4 client.15423 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55780534c270 con 0x5578056aa6d0 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.942+0000 7faaa08fb640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5578056aa6d0 0x5578056aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faaa08fb640 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5578056aa6d0 0x5578056aeb50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e30c71c0ffe38e5 server_cookie=c5350e22a795183f in_seq=0 out_seq=0 2024-02-19T16:22:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faa80ff9640 10 client.15423 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faa80ff9640 1 -- 172.21.15.8:0/735072035 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55780534c270 con 0x5578056aa6d0 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faa80ff9640 10 client.15423 handle_command_reply: tid=0 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faaa600f740 1 client.15423 shutdown 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 msgr2=0x55780569e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.943+0000 7faaa600f740 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780569e0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faa94000c00 tx=0x7faa940050a0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 shutdown_connections 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805699ae0 0x5578056a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5578056aa6d0 0x5578056aeb50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805694970 0x55780569e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 --2- 172.21.15.8:0/735072035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780569e0d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faa80ff9640 0 client.15423 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 >> 172.21.15.8:0/735072035 conn(0x55780568f4f0 msgr2=0x55780568e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 shutdown_connections 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 1 -- 172.21.15.8:0/735072035 wait complete. 2024-02-19T16:22:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.944+0000 7faaa600f740 20 client.15423 trim_cache size 0 max 16384 2024-02-19T16:22:31.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.960+0000 7faaa600f740 1 Processor -- start 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.960+0000 7faaa600f740 1 -- start start 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa600f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780568df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa600f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa600f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780568e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa600f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557805581490 con 0x557805693f80 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa600f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780568ffe0 con 0x557805694970 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa600f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578052fb6c0 con 0x557805699ae0 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780568df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780568df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35028/0 (socket says 172.21.15.8:35028) 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 -- 172.21.15.8:0/90808155 learned_addr learned my addr 172.21.15.8:0/90808155 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa18fd640 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780568e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa08fb640 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 -- 172.21.15.8:0/90808155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 msgr2=0x55780568e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780568e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 -- 172.21.15.8:0/90808155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 msgr2=0x5578056a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 -- 172.21.15.8:0/90808155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557805596a20 con 0x557805693f80 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa08fb640 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faaa10fc640 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780568df40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faa94004b10 tx=0x7faa9402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0682ccb0d0c39f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faa82ffd640 1 -- 172.21.15.8:0/90808155 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94005110 con 0x557805693f80 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faa82ffd640 1 -- 172.21.15.8:0/90808155 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa94004510 con 0x557805693f80 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.961+0000 7faa82ffd640 1 -- 172.21.15.8:0/90808155 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94034d00 con 0x557805693f80 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 -- 172.21.15.8:0/90808155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 msgr2=0x55780568df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780568df40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faa94004b10 tx=0x7faa9402c330 comp rx=0 tx=0).stop 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 -- 172.21.15.8:0/90808155 shutdown_connections 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805699ae0 0x55780568e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 --2- 172.21.15.8:0/90808155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805693f80 0x55780568df40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 -- 172.21.15.8:0/90808155 >> 172.21.15.8:0/90808155 conn(0x55780568f4f0 msgr2=0x55780568f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 -- 172.21.15.8:0/90808155 shutdown_connections 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 -- 172.21.15.8:0/90808155 wait complete. 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 -- start start 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.962+0000 7faaa600f740 1 Processor -- start 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805693f80 0x5578056a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 0x5578056a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578056a4ee0 con 0x557805699ae0 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578056a5050 con 0x557805694970 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578056a5210 con 0x557805693f80 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 10 client.0 _ll_register_callbacks cb 0x7faa9009ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 0x5578056a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 0x5578056a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35038/0 (socket says 172.21.15.8:35038) 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 -- 172.21.15.8:0/3662535242 learned_addr learned my addr 172.21.15.8:0/3662535242 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa10fc640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805693f80 0x5578056a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa08fb640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 -- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805693f80 msgr2=0x5578056a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805693f80 0x5578056a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 -- 172.21.15.8:0/3662535242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 msgr2=0x5578056a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 -- 172.21.15.8:0/3662535242 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578055fa2b0 con 0x557805699ae0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa18fd640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 0x5578056a49a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0039a0 tx=0x7faa8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e70c0107ee15dbd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faa80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faa80ff9640 1 -- 172.21.15.8:0/3662535242 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c004b20 con 0x557805699ae0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faa80ff9640 1 -- 172.21.15.8:0/3662535242 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7faa8c025070 con 0x557805699ae0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578056aa3e0 con 0x557805699ae0 2024-02-19T16:22:31.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.963+0000 7faa80ff9640 1 -- 172.21.15.8:0/3662535242 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa8c004c80 con 0x557805699ae0 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 10 client.15429 fetch_fsmap learned FSMap version 23 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5578055bc780 con 0x557805699ae0 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faa80ff9640 1 -- 172.21.15.8:0/3662535242 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5578056aa3e0 con 0x557805699ae0 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faa80ff9640 1 -- 172.21.15.8:0/3662535242 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa8c029470 con 0x557805699ae0 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 10 client.15429 fetch_fsmap finished waiting for FSMap version 23 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 10 client.15429 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 20 client.15429 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55780570a750 0x557805706b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 4 client.15429 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7faa9402cac0 con 0x55780570a750 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa10fc640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55780570a750 0x557805706b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faaa10fc640 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55780570a750 0x557805706b70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420e1564cd05e6e7 server_cookie=52bad03f82920c50 in_seq=0 out_seq=0 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.964+0000 7faa80ff9640 10 client.15429 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faa80ff9640 1 -- 172.21.15.8:0/3662535242 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x7faa9402cac0 con 0x55780570a750 2024-02-19T16:22:31.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faa80ff9640 10 client.15429 handle_command_reply: tid=0 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 client.15429 shutdown 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 msgr2=0x5578056a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 0x5578056a49a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0039a0 tx=0x7faa8c004a40 comp rx=0 tx=0).stop 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 shutdown_connections 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557805699ae0 0x5578056a49a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557805694970 0x5578056a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55780570a750 0x557805706b70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 --2- 172.21.15.8:0/3662535242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557805693f80 0x5578056a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faa80ff9640 0 client.15429 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 >> 172.21.15.8:0/3662535242 conn(0x55780568f4f0 msgr2=0x557805692750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 shutdown_connections 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 1 -- 172.21.15.8:0/3662535242 wait complete. 2024-02-19T16:22:31.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa600f740 20 client.15429 trim_cache size 0 max 16384 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "81b9a938-cf41-11ee-95bb-87774f69a715", 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24502, 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 130.23773466, 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 23, 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 95, 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 91, 2024-02-19T16:22:31.976 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 142.985645764 2024-02-19T16:22:31.977 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa840a2140 msgr2=0x7faa840a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa840a2140 0x7faa840a4600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5578056951d0 tx=0x7faa8c009500 comp rx=0 tx=0).stop 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 msgr2=0x7faa9c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 0x7faa9c197b20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa9000ea60 tx=0x7faa9000ef20 comp rx=0 tx=0).stop 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 shutdown_connections 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa840a2140 0x7faa840a4600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa9c101fb0 0x7faa9c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa9c1015f0 0x7faa9c197b20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 --2- 172.21.15.8:0/2924691140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa9c100400 0x7faa9c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:31.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.965+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 >> 172.21.15.8:0/2924691140 conn(0x7faa9c0fb990 msgr2=0x7faa9c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:31.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.966+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 shutdown_connections 2024-02-19T16:22:31.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:31.966+0000 7faaa3360640 1 -- 172.21.15.8:0/2924691140 wait complete. 2024-02-19T16:22:31.979 INFO:teuthology.run_tasks:Running task kclient... 2024-02-19T16:22:31.995 INFO:tasks.kclient:Mounting kernel clients... 2024-02-19T16:22:31.995 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-19T16:22:31.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:31.996 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2024-02-19T16:22:32.012 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-19T16:22:32.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:32.013 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-19T16:22:32.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:32.080 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2024-02-19T16:22:32.187 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-02-19T16:22:32.187 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-19T16:22:32.187 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:32.187 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph fs ls 2024-02-19T16:22:32.433 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:22:32.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:32 smithi008 ceph-mon[32573]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-02-19T16:22:32.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:32 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2592868927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:32.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:32 smithi191 ceph-mon[37203]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-02-19T16:22:32.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:32 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2592868927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:32 smithi156 ceph-mon[36502]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-02-19T16:22:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:32 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2592868927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:34.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 -- 172.21.15.8:0/2357254384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06b130 msgr2=0x7f41fc106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 --2- 172.21.15.8:0/2357254384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06b130 0x7f41fc106c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009990 tx=0x7f41f0009ca0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 -- 172.21.15.8:0/2357254384 shutdown_connections 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 --2- 172.21.15.8:0/2357254384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc107360 0x7f41fc10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 --2- 172.21.15.8:0/2357254384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc06a780 0x7f41fc06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 --2- 172.21.15.8:0/2357254384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06b130 0x7f41fc106c30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 -- 172.21.15.8:0/2357254384 >> 172.21.15.8:0/2357254384 conn(0x7f41fc076f30 msgr2=0x7f41fc077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 -- 172.21.15.8:0/2357254384 shutdown_connections 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.133+0000 7f4203dce700 1 -- 172.21.15.8:0/2357254384 wait complete. 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 Processor -- start 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 -- start start 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 0x7f41fc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc06b130 0x7f41fc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:34.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc107360 0x7f41fc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc19d3b0 con 0x7f41fc06a780 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc19d520 con 0x7f41fc107360 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4203dce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc19d6e0 con 0x7f41fc06b130 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 0x7f41fc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 0x7f41fc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35054/0 (socket says 172.21.15.8:35054) 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 -- 172.21.15.8:0/1673462735 learned_addr learned my addr 172.21.15.8:0/1673462735 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f420236b700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc107360 0x7f41fc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201369700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc06b130 0x7f41fc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 -- 172.21.15.8:0/1673462735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc06b130 msgr2=0x7f41fc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc06b130 0x7f41fc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 -- 172.21.15.8:0/1673462735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc107360 msgr2=0x7f41fc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc107360 0x7f41fc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.134+0000 7f4201b6a700 1 -- 172.21.15.8:0/1673462735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f00095f0 con 0x7f41fc06a780 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.135+0000 7f4201b6a700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 0x7f41fc1984d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f41f400ef10 tx=0x7f41f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:34.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.135+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400c9c0 con 0x7f41fc06a780 2024-02-19T16:22:34.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.135+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41f400f040 con 0x7f41fc06a780 2024-02-19T16:22:34.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.135+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014480 con 0x7f41fc06a780 2024-02-19T16:22:34.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.135+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc19d960 con 0x7f41fc06a780 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.135+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc19dc10 con 0x7f41fc06a780 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.136+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc10a550 con 0x7f41fc06a780 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.137+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f41f4007720 con 0x7f41fc06a780 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.138+0000 7f41faffd700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41e40a21d0 0x7f41e40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.138+0000 7f4201369700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41e40a21d0 0x7f41e40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.138+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f41f40ce730 con 0x7f41fc06a780 2024-02-19T16:22:34.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.138+0000 7f4201369700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41e40a21d0 0x7f41e40a4690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f41f00059d0 tx=0x7f41f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:34.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.141+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f41f4097360 con 0x7f41fc06a780 2024-02-19T16:22:34.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.401+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f41fc068ca0 con 0x7f41fc06a780 2024-02-19T16:22:34.403 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:34 smithi008 ceph-mon[32573]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:22:34.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.402+0000 7f41faffd700 1 -- 172.21.15.8:0/1673462735 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f41f409aa20 con 0x7f41fc06a780 2024-02-19T16:22:34.404 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-19T16:22:34.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41e40a21d0 msgr2=0x7f41e40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:34.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41e40a21d0 0x7f41e40a4690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f41f00059d0 tx=0x7f41f0005960 comp rx=0 tx=0).stop 2024-02-19T16:22:34.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 msgr2=0x7f41fc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:34.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 0x7f41fc1984d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f41f400ef10 tx=0x7f41f400c3a0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 shutdown_connections 2024-02-19T16:22:34.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f41e40a21d0 0x7f41e40a4690 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc107360 0x7f41fc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41fc06a780 0x7f41fc1984d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 --2- 172.21.15.8:0/1673462735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc06b130 0x7f41fc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:34.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 >> 172.21.15.8:0/1673462735 conn(0x7f41fc076f30 msgr2=0x7f41fc0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:34.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 shutdown_connections 2024-02-19T16:22:34.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:34.404+0000 7f4203dce700 1 -- 172.21.15.8:0/1673462735 wait complete. 2024-02-19T16:22:34.631 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:34 smithi156 ceph-mon[36502]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:22:34.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:34 smithi191 ceph-mon[37203]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-19T16:22:35.085 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi008.front.sepia.ceph.com 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-19T16:22:35.086 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-19T16:22:35.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.087 DEBUG:teuthology.orchestra.run.smithi008:> ip addr 2024-02-19T16:22:35.104 INFO:teuthology.orchestra.run.smithi008.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-19T16:22:35.104 INFO:teuthology.orchestra.run.smithi008.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-19T16:22:35.104 INFO:teuthology.orchestra.run.smithi008.stdout: inet 127.0.0.1/8 scope host lo 2024-02-19T16:22:35.104 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 ::1/128 scope host 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2e brd ff:ff:ff:ff:ff:ff 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f0 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2f brd ff:ff:ff:ff:ff:ff 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f1 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:fe brd ff:ff:ff:ff:ff:ff 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f0 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.21.15.8/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft 41837sec preferred_lft 41837sec 2024-02-19T16:22:35.105 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link noprefixroute 2024-02-19T16:22:35.106 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-02-19T16:22:35.106 INFO:teuthology.orchestra.run.smithi008.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T16:22:35.106 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:ff brd ff:ff:ff:ff:ff:ff 2024-02-19T16:22:35.106 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f1 2024-02-19T16:22:35.106 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-19T16:22:35.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.106 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:22:35.106 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:22:35.106 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add name ceph-brx type bridge 2024-02-19T16:22:35.106 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr flush dev ceph-brx 2024-02-19T16:22:35.107 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set ceph-brx up 2024-02-19T16:22:35.107 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-19T16:22:35.107 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:22:35.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.218 DEBUG:teuthology.orchestra.run.smithi008:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-19T16:22:35.245 INFO:teuthology.orchestra.run.smithi008.stdout:1 2024-02-19T16:22:35.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.247 DEBUG:teuthology.orchestra.run.smithi008:> route 2024-02-19T16:22:35.302 INFO:teuthology.orchestra.run.smithi008.stdout:Kernel IP routing table 2024-02-19T16:22:35.302 INFO:teuthology.orchestra.run.smithi008.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-19T16:22:35.302 INFO:teuthology.orchestra.run.smithi008.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-19T16:22:35.302 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-19T16:22:35.303 INFO:teuthology.orchestra.run.smithi008.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-19T16:22:35.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.303 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:22:35.303 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:22:35.303 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-19T16:22:35.303 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-19T16:22:35.303 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-19T16:22:35.303 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:22:35.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.399 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2024-02-19T16:22:35.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.415 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list-id 2024-02-19T16:22:35.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.470 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:22:35.470 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:22:35.470 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T16:22:35.470 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-19T16:22:35.470 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:22:35.551 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-19T16:22:35.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-19T16:22:35.552 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:22:35.617 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:35 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1673462735' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T16:22:35.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.681 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-02-19T16:22:35.681 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-02-19T16:22:35.682 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set brx.0 up 2024-02-19T16:22:35.682 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set dev brx.0 master ceph-brx 2024-02-19T16:22:35.682 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-02-19T16:22:35.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T16:22:35.763 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:35.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:35 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1673462735' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T16:22:35.822 INFO:teuthology.orchestra.run.smithi008.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-19T16:22:35.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T16:22:35.822 DEBUG:teuthology.orchestra.run.smithi008:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:35.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:35 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1673462735' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T16:22:35.878 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-19T16:22:35.878 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-19T16:22:35.878 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/86171f7c-f2df-4e31-a388-f917828b18e5 2024-02-19T16:22:35.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:35.878 DEBUG:teuthology.orchestra.run.smithi008:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/86171f7c-f2df-4e31-a388-f917828b18e5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-19T16:22:35.961 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-19T16:22:35.962 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-02-19T16:22:36.089 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-19T16:22:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-02-19T16:22:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: resolved to: "172.21.15.8:6789,172.21.15.156:6789,172.21.15.191:6789" 2024-02-19T16:22:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: trying mount with new device syntax: 0@81b9a938-cf41-11ee-95bb-87774f69a715.cephfs=/volumes/qa/sv_1/86171f7c-f2df-4e31-a388-f917828b18e5 2024-02-19T16:22:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.8:6789/172.21.15.156:6789/172.21.15.191:6789" will pass to kernel 2024-02-19T16:22:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-19T16:22:36.091 INFO:teuthology.orchestra.run.smithi008.stdout: You just mounted a file system that supports labels which does not 2024-02-19T16:22:36.091 INFO:teuthology.orchestra.run.smithi008.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-19T16:22:36.091 INFO:teuthology.orchestra.run.smithi008.stdout: applications will generate AVC messages and not be allowed access to 2024-02-19T16:22:36.091 INFO:teuthology.orchestra.run.smithi008.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-19T16:22:36.093 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-19T16:22:36.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:36.093 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:36.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:36.168 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-19T16:22:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:client25315 2024-02-19T16:22:36.224 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:22:36.224 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T16:22:36.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:36.280 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-19T16:22:36.335 INFO:teuthology.orchestra.run.smithi008.stdout:client25315 2024-02-19T16:22:36.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:36.336 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-19T16:22:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:client25315 2024-02-19T16:22:36.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:36.392 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/81b9a938-cf41-11ee-95bb-87774f69a715.client25315/status') 2024-02-19T16:22:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:instance: client.25315 (3)172.21.15.8:0/1445213621 2024-02-19T16:22:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:blocklisted: false 2024-02-19T16:22:36.459 INFO:teuthology.orchestra.run.smithi008.stderr:0+1 records in 2024-02-19T16:22:36.459 INFO:teuthology.orchestra.run.smithi008.stderr:0+1 records out 2024-02-19T16:22:36.459 INFO:teuthology.orchestra.run.smithi008.stderr:70 bytes copied, 2.0165e-05 s, 3.5 MB/s 2024-02-19T16:22:36.460 INFO:teuthology.run_tasks:Running task exec... 2024-02-19T16:22:36.473 INFO:teuthology.task.exec:Executing custom commands... 2024-02-19T16:22:36.473 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:22:36.473 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-02-19T16:22:36.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 -- 172.21.15.8:0/3164094836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 msgr2=0x7f8a741043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 --2- 172.21.15.8:0/3164094836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a741043e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009900 tx=0x7f8a6c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 -- 172.21.15.8:0/3164094836 shutdown_connections 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 --2- 172.21.15.8:0/3164094836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74107250 0x7f8a74109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 --2- 172.21.15.8:0/3164094836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a74104920 0x7f8a74106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 --2- 172.21.15.8:0/3164094836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a741043e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 -- 172.21.15.8:0/3164094836 >> 172.21.15.8:0/3164094836 conn(0x7f8a740fb980 msgr2=0x7f8a740fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 -- 172.21.15.8:0/3164094836 shutdown_connections 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 -- 172.21.15.8:0/3164094836 wait complete. 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 Processor -- start 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 -- start start 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a74197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.603+0000 7f8a7930d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104920 0x7f8a74197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:36.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a7930d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a74107250 0x7f8a7419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a7930d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7419f040 con 0x7f8a74101ff0 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a7930d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7419f1b0 con 0x7f8a74107250 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a7930d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7419f370 con 0x7f8a74104920 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a74197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a74197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49306/0 (socket says 172.21.15.8:49306) 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 -- 172.21.15.8:0/1117095609 learned_addr learned my addr 172.21.15.8:0/1117095609 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a737fe640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a74107250 0x7f8a7419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a727fc640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104920 0x7f8a74197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 -- 172.21.15.8:0/1117095609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104920 msgr2=0x7f8a74197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104920 0x7f8a74197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 -- 172.21.15.8:0/1117095609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a74107250 msgr2=0x7f8a7419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a74107250 0x7f8a7419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 -- 172.21.15.8:0/1117095609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009560 con 0x7f8a74101ff0 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a727fc640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104920 0x7f8a74197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:22:36.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a72ffd640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a74197550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c006010 tx=0x7f8a6c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:36.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0043d0 con 0x7f8a74101ff0 2024-02-19T16:22:36.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a6c0052d0 con 0x7f8a74101ff0 2024-02-19T16:22:36.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7419f5f0 con 0x7f8a74101ff0 2024-02-19T16:22:36.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c003cc0 con 0x7f8a74101ff0 2024-02-19T16:22:36.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.604+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7419fb00 con 0x7f8a74101ff0 2024-02-19T16:22:36.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.605+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a740fd470 con 0x7f8a74101ff0 2024-02-19T16:22:36.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.606+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8a6c005440 con 0x7f8a74101ff0 2024-02-19T16:22:36.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.607+0000 7f8a5bfff640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8a5c0a2160 0x7f8a5c0a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:36.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.607+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f8a6c0f0310 con 0x7f8a74101ff0 2024-02-19T16:22:36.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.607+0000 7f8a727fc640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8a5c0a2160 0x7f8a5c0a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:36.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.608+0000 7f8a727fc640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8a5c0a2160 0x7f8a5c0a4620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a74198a60 tx=0x7f8a68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:36.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.612+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8a6c0b8f40 con 0x7f8a74101ff0 2024-02-19T16:22:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:36 smithi008 ceph-mon[32573]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:36.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:36 smithi191 ceph-mon[37203]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:36 smithi156 ceph-mon[36502]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-19T16:22:36.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:36.841+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f8a7402cf90 con 0x7f8a74101ff0 2024-02-19T16:22:37.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.391+0000 7f8a5bfff640 1 -- 172.21.15.8:0/1117095609 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8a6c0bc600 con 0x7f8a74101ff0 2024-02-19T16:22:37.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8a5c0a2160 msgr2=0x7f8a5c0a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8a5c0a2160 0x7f8a5c0a4620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a74198a60 tx=0x7f8a68005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 msgr2=0x7f8a74197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a74197550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c006010 tx=0x7f8a6c005e80 comp rx=0 tx=0).stop 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 shutdown_connections 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8a5c0a2160 0x7f8a5c0a4620 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a74107250 0x7f8a7419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104920 0x7f8a74197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 --2- 172.21.15.8:0/1117095609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a74101ff0 0x7f8a74197550 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 >> 172.21.15.8:0/1117095609 conn(0x7f8a740fb980 msgr2=0x7f8a74102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 shutdown_connections 2024-02-19T16:22:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.395+0000 7f8a7930d640 1 -- 172.21.15.8:0/1117095609 wait complete. 2024-02-19T16:22:37.409 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-02-19T16:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 -- 172.21.15.8:0/1215672894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101fe0 msgr2=0x7f2fb8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/1215672894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101fe0 0x7f2fb8108870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009ca0 tx=0x7f2fb002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 -- 172.21.15.8:0/1215672894 shutdown_connections 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/1215672894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101fe0 0x7f2fb8108870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/1215672894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101620 0x7f2fb8101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/1215672894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb8100430 0x7f2fb8100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 -- 172.21.15.8:0/1215672894 >> 172.21.15.8:0/1215672894 conn(0x7f2fb80fb9a0 msgr2=0x7f2fb80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 -- 172.21.15.8:0/1215672894 shutdown_connections 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 -- 172.21.15.8:0/1215672894 wait complete. 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 Processor -- start 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.520+0000 7f2fbe90d640 1 -- start start 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbe90d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb8100430 0x7f2fb8197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbe90d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 0x7f2fb81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbe90d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101fe0 0x7f2fb819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbe90d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb819ef70 con 0x7f2fb8101620 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbe90d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb819f0e0 con 0x7f2fb8100430 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbe90d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb819f2a0 con 0x7f2fb8101fe0 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 0x7f2fb81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 0x7f2fb81979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49322/0 (socket says 172.21.15.8:49322) 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101fe0 0x7f2fb819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fb77fe640 1 -- 172.21.15.8:0/2622091594 learned_addr learned my addr 172.21.15.8:0/2622091594 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fb7fff640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb8100430 0x7f2fb8197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 -- 172.21.15.8:0/2622091594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb8100430 msgr2=0x7f2fb8197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb8100430 0x7f2fb8197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 -- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 msgr2=0x7f2fb81979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 0x7f2fb81979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 -- 172.21.15.8:0/2622091594 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0009900 con 0x7f2fb8101fe0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fb77fe640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 0x7f2fb81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fbceaa640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101fe0 0x7f2fb819ea30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fb002c630 tx=0x7f2fb0034a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.521+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00329b0 con 0x7f2fb8101fe0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.522+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fb0003e50 con 0x7f2fb8101fe0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.522+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb819f520 con 0x7f2fb8101fe0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.522+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0040450 con 0x7f2fb8101fe0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.522+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb819f9a0 con 0x7f2fb8101fe0 2024-02-19T16:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.523+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb810be40 con 0x7f2fb8101fe0 2024-02-19T16:22:37.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.523+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f2fb000a040 con 0x7f2fb8101fe0 2024-02-19T16:22:37.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.524+0000 7f2fb57fa640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2fa00a2160 0x7f2fa00a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.524+0000 7f2fb7fff640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2fa00a2160 0x7f2fa00a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.524+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f2fb00f0cd0 con 0x7f2fb8101fe0 2024-02-19T16:22:37.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.524+0000 7f2fb7fff640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2fa00a2160 0x7f2fa00a4620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8101480 tx=0x7f2fa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:37.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.528+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2fb00b9900 con 0x7f2fb8101fe0 2024-02-19T16:22:37.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.741+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fb80fdb00 con 0x7f2fa00a2160 2024-02-19T16:22:37.747 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:37 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1117095609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-19T16:22:37.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.756+0000 7f2fb57fa640 1 -- 172.21.15.8:0/2622091594 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f2fb80fdb00 con 0x7f2fa00a2160 2024-02-19T16:22:37.757 INFO:teuthology.orchestra.run.smithi008.stdout:{} 2024-02-19T16:22:37.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2fa00a2160 msgr2=0x7f2fa00a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2fa00a2160 0x7f2fa00a4620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8101480 tx=0x7f2fa8005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101fe0 msgr2=0x7f2fb819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101fe0 0x7f2fb819ea30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fb002c630 tx=0x7f2fb0034a90 comp rx=0 tx=0).stop 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 shutdown_connections 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f2fa00a2160 0x7f2fa00a4620 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb8101fe0 0x7f2fb819ea30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fb8101620 0x7f2fb81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 --2- 172.21.15.8:0/2622091594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb8100430 0x7f2fb8197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 >> 172.21.15.8:0/2622091594 conn(0x7f2fb80fb9a0 msgr2=0x7f2fb80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 shutdown_connections 2024-02-19T16:22:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.759+0000 7f2fbe90d640 1 -- 172.21.15.8:0/2622091594 wait complete. 2024-02-19T16:22:37.770 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-19T16:22:37.798 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-19T16:22:37.866 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:37.867 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:37 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1117095609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-19T16:22:37.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:37 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1117095609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.969+0000 7f67bcd65640 1 -- 172.21.15.8:0/3850840258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8104920 msgr2=0x7f67b8106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.969+0000 7f67bcd65640 1 --2- 172.21.15.8:0/3850840258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8104920 0x7f67b8106d10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009900 tx=0x7f67ac02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 -- 172.21.15.8:0/3850840258 shutdown_connections 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 --2- 172.21.15.8:0/3850840258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b8107250 0x7f67b8109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 --2- 172.21.15.8:0/3850840258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8104920 0x7f67b8106d10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 --2- 172.21.15.8:0/3850840258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67b8101ff0 0x7f67b81043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 -- 172.21.15.8:0/3850840258 >> 172.21.15.8:0/3850840258 conn(0x7f67b80fb980 msgr2=0x7f67b80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 -- 172.21.15.8:0/3850840258 shutdown_connections 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 -- 172.21.15.8:0/3850840258 wait complete. 2024-02-19T16:22:37.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 Processor -- start 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.970+0000 7f67bcd65640 1 -- start start 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b8101ff0 0x7f67b81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67b8104920 0x7f67b8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 0x7f67b819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819f0a0 con 0x7f67b8107250 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819f210 con 0x7f67b8104920 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819f3d0 con 0x7f67b8101ff0 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 0x7f67b819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 0x7f67b819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49358/0 (socket says 172.21.15.8:49358) 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 -- 172.21.15.8:0/1098402780 learned_addr learned my addr 172.21.15.8:0/1098402780 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b659c640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b8101ff0 0x7f67b81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b5d9b640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67b8104920 0x7f67b8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 -- 172.21.15.8:0/1098402780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b8101ff0 msgr2=0x7f67b81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b8101ff0 0x7f67b81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 -- 172.21.15.8:0/1098402780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67b8104920 msgr2=0x7f67b8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67b8104920 0x7f67b8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 -- 172.21.15.8:0/1098402780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009560 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67b6d9d640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 0x7f67b819eb60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f67b000eb10 tx=0x7f67b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004b20 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b819f6b0 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67b0004c80 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0010670 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.971+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b819fbe0 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.972+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b80fd470 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.973+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f67b0005020 con 0x7f67b8107250 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.974+0000 7f67a77fe640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f67a00a20c0 0x7f67a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.974+0000 7f67b659c640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f67a00a20c0 0x7f67a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:37.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.974+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f67b0014070 con 0x7f67b8107250 2024-02-19T16:22:37.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.975+0000 7f67b659c640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f67a00a20c0 0x7f67a00a4580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009bd0 tx=0x7f67a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:37.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:37.978+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f67b00971b0 con 0x7f67b8107250 2024-02-19T16:22:38.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:37.745+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:38.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:37.745+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:38.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:37.745+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:38.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:37.745+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:38.020 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:37.745+0000 7f1b1b07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:38.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.163+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67b802cf90 con 0x7f67b8107250 2024-02-19T16:22:38.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.164+0000 7f67a77fe640 1 -- 172.21.15.8:0/1098402780 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7f67b009a870 con 0x7f67b8107250 2024-02-19T16:22:38.189 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:38.189 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:38.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.166+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f67a00a20c0 msgr2=0x7f67a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.166+0000 7f67bcd65640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f67a00a20c0 0x7f67a00a4580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009bd0 tx=0x7f67a8005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.166+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 msgr2=0x7f67b819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.166+0000 7f67bcd65640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 0x7f67b819eb60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f67b000eb10 tx=0x7f67b000c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 shutdown_connections 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f67a00a20c0 0x7f67a00a4580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b8107250 0x7f67b819eb60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67b8104920 0x7f67b8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 --2- 172.21.15.8:0/1098402780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b8101ff0 0x7f67b81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 >> 172.21.15.8:0/1098402780 conn(0x7f67b80fb980 msgr2=0x7f67b8102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 shutdown_connections 2024-02-19T16:22:38.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.167+0000 7f67bcd65640 1 -- 172.21.15.8:0/1098402780 wait complete. 2024-02-19T16:22:38.192 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:38.193 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 -- 172.21.15.8:0/3324913695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 msgr2=0x7f8adc104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/3324913695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc104370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009900 tx=0x7f8acc02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 -- 172.21.15.8:0/3324913695 shutdown_connections 2024-02-19T16:22:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/3324913695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc1071e0 0x7f8adc109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/3324913695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1048b0 0x7f8adc106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/3324913695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc104370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 -- 172.21.15.8:0/3324913695 >> 172.21.15.8:0/3324913695 conn(0x7f8adc0fb990 msgr2=0x7f8adc0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 -- 172.21.15.8:0/3324913695 shutdown_connections 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.298+0000 7f8ae10fa640 1 -- 172.21.15.8:0/3324913695 wait complete. 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 Processor -- start 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 -- start start 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc1048b0 0x7f8adc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 0x7f8adc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f0e0 con 0x7f8adc1071e0 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f250 con 0x7f8adc1048b0 2024-02-19T16:22:38.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ae10fa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f410 con 0x7f8adc101f80 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8adad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8adb59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 0x7f8adc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8adad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:57382/0 (socket says 172.21.15.8:57382) 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8adb59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 0x7f8adc19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49366/0 (socket says 172.21.15.8:49366) 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8adad9d640 1 -- 172.21.15.8:0/988479163 learned_addr learned my addr 172.21.15.8:0/988479163 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.299+0000 7f8ad259c640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc1048b0 0x7f8adc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8adb59e640 1 -- 172.21.15.8:0/988479163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 msgr2=0x7f8adc1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8adb59e640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8adb59e640 1 -- 172.21.15.8:0/988479163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc1048b0 msgr2=0x7f8adc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8adb59e640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc1048b0 0x7f8adc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8adb59e640 1 -- 172.21.15.8:0/988479163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4009490 con 0x7f8adc1071e0 2024-02-19T16:22:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8adb59e640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 0x7f8adc19eba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400ec50 tx=0x7f8ad400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4003b70 con 0x7f8adc1071e0 2024-02-19T16:22:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad4009840 con 0x7f8adc1071e0 2024-02-19T16:22:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad400ce70 con 0x7f8adc1071e0 2024-02-19T16:22:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4010830 con 0x7f8adc1071e0 2024-02-19T16:22:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.300+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc009560 con 0x7f8adc1071e0 2024-02-19T16:22:38.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.301+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc0fd480 con 0x7f8adc1071e0 2024-02-19T16:22:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.302+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 136767+0+0 (secure 0 0 0) 0x7f8ad4004020 con 0x7f8adc1071e0 2024-02-19T16:22:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.303+0000 7f8ad8d99640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8ac80a20c0 0x7f8ac80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.303+0000 7f8adad9d640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8ac80a20c0 0x7f8ac80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.303+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f8ad4014070 con 0x7f8adc1071e0 2024-02-19T16:22:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.303+0000 7f8adad9d640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8ac80a20c0 0x7f8ac80a4580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000c00 tx=0x7f8acc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.307+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8ad4097380 con 0x7f8adc1071e0 2024-02-19T16:22:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.407+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f8ad40042d0 con 0x7f8adc1071e0 2024-02-19T16:22:38.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.495+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8adc02cf60 con 0x7f8adc1071e0 2024-02-19T16:22:38.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.495+0000 7f8ad8d99640 1 -- 172.21.15.8:0/988479163 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ad4055910 con 0x7f8adc1071e0 2024-02-19T16:22:38.496 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:38.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8ac80a20c0 msgr2=0x7f8ac80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8ac80a20c0 0x7f8ac80a4580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000c00 tx=0x7f8acc02c7e0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 msgr2=0x7f8adc19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 0x7f8adc19eba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400ec50 tx=0x7f8ad400c6a0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 shutdown_connections 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8ac80a20c0 0x7f8ac80a4580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8adc1071e0 0x7f8adc19eba0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc1048b0 0x7f8adc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 --2- 172.21.15.8:0/988479163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8adc101f80 0x7f8adc1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 >> 172.21.15.8:0/988479163 conn(0x7f8adc0fb990 msgr2=0x7f8adc102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 shutdown_connections 2024-02-19T16:22:38.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.497+0000 7f8ae10fa640 1 -- 172.21.15.8:0/988479163 wait complete. 2024-02-19T16:22:38.508 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:38 smithi008 ceph-mon[32573]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:22:38.511 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:38 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1117095609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-19T16:22:38.511 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:38 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:22:38.511 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:38 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1098402780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:38.513 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:38.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- 172.21.15.8:0/2670083796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 msgr2=0x7f8814101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- 172.21.15.8:0/2670083796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f8814101a70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- 172.21.15.8:0/2670083796 shutdown_connections 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- 172.21.15.8:0/2670083796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8814101fb0 0x7f8814108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- 172.21.15.8:0/2670083796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f8814101a70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- 172.21.15.8:0/2670083796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8814100400 0x7f8814100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- 172.21.15.8:0/2670083796 >> 172.21.15.8:0/2670083796 conn(0x7f88140fb990 msgr2=0x7f88140fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- 172.21.15.8:0/2670083796 shutdown_connections 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- 172.21.15.8:0/2670083796 wait complete. 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 Processor -- start 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- start start 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8814100400 0x7f881406a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f881419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8814101fb0 0x7f881419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419f0d0 con 0x7f88141015f0 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419f240 con 0x7f8814100400 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.618+0000 7f881bb48640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419f400 con 0x7f8814101fb0 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f881419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f881419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49388/0 (socket says 172.21.15.8:49388) 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88198e4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8814100400 0x7f881406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 -- 172.21.15.8:0/3597207164 learned_addr learned my addr 172.21.15.8:0/3597207164 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f881a0e5640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8814101fb0 0x7f881419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 -- 172.21.15.8:0/3597207164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8814101fb0 msgr2=0x7f881419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8814101fb0 0x7f881419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 -- 172.21.15.8:0/3597207164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8814100400 msgr2=0x7f881406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8814100400 0x7f881406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 -- 172.21.15.8:0/3597207164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8808009560 con 0x7f88141015f0 2024-02-19T16:22:38.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f88190e3640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f881419c350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f8808005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:38.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88080043d0 con 0x7f88141015f0 2024-02-19T16:22:38.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881419f680 con 0x7f88141015f0 2024-02-19T16:22:38.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88080052d0 con 0x7f88141015f0 2024-02-19T16:22:38.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808003cc0 con 0x7f88141015f0 2024-02-19T16:22:38.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.619+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881419fb90 con 0x7f88141015f0 2024-02-19T16:22:38.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.620+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8814107720 con 0x7f88141015f0 2024-02-19T16:22:38.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.621+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f8808005440 con 0x7f88141015f0 2024-02-19T16:22:38.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.622+0000 7f8812ffd640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87fc0a2150 0x7f87fc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.622+0000 7f88198e4640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87fc0a2150 0x7f87fc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.622+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f88080f0350 con 0x7f88141015f0 2024-02-19T16:22:38.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.622+0000 7f88198e4640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87fc0a2150 0x7f87fc0a4610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8814101450 tx=0x7f880c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:38.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.626+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f88080b8ed0 con 0x7f88141015f0 2024-02-19T16:22:38.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:38 smithi191 ceph-mon[37203]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:22:38.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:38 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1117095609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-19T16:22:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:38 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:22:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:38 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1098402780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:38.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.809+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f881410bfc0 con 0x7f88141015f0 2024-02-19T16:22:38.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.809+0000 7f8812ffd640 1 -- 172.21.15.8:0/3597207164 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88080bc590 con 0x7f88141015f0 2024-02-19T16:22:38.810 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:38.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87fc0a2150 msgr2=0x7f87fc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87fc0a2150 0x7f87fc0a4610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8814101450 tx=0x7f880c005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:38.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 msgr2=0x7f881419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f881419c350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f8808005e80 comp rx=0 tx=0).stop 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 shutdown_connections 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f87fc0a2150 0x7f87fc0a4610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8814101fb0 0x7f881419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f88141015f0 0x7f881419c350 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 --2- 172.21.15.8:0/3597207164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8814100400 0x7f881406a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 >> 172.21.15.8:0/3597207164 conn(0x7f88140fb990 msgr2=0x7f88141048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 shutdown_connections 2024-02-19T16:22:38.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.811+0000 7f881bb48640 1 -- 172.21.15.8:0/3597207164 wait complete. 2024-02-19T16:22:38.823 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:38 smithi156 ceph-mon[36502]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-19T16:22:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:38 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1117095609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-19T16:22:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:38 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:22:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:38 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1098402780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 -- 172.21.15.8:0/3683904153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a41030b0 msgr2=0x7f98a4109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 --2- 172.21.15.8:0/3683904153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a41030b0 0x7f98a4109940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f989c009ca0 tx=0x7f989c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 -- 172.21.15.8:0/3683904153 shutdown_connections 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 --2- 172.21.15.8:0/3683904153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a41030b0 0x7f98a4109940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 --2- 172.21.15.8:0/3683904153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98a41026f0 0x7f98a4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 --2- 172.21.15.8:0/3683904153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a4101500 0x7f98a4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 -- 172.21.15.8:0/3683904153 >> 172.21.15.8:0/3683904153 conn(0x7f98a40fca90 msgr2=0x7f98a40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:38.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 -- 172.21.15.8:0/3683904153 shutdown_connections 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 -- 172.21.15.8:0/3683904153 wait complete. 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 Processor -- start 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.929+0000 7f98a8b13640 1 -- start start 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 0x7f98a4197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a41026f0 0x7f98a4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98a41030b0 0x7f98a419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a419f050 con 0x7f98a4101500 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a419f1c0 con 0x7f98a41030b0 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a419f380 con 0x7f98a41026f0 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 0x7f98a4197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 0x7f98a4197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49406/0 (socket says 172.21.15.8:49406) 2024-02-19T16:22:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 -- 172.21.15.8:0/946534985 learned_addr learned my addr 172.21.15.8:0/946534985 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f989bfff640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a41026f0 0x7f98a4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a2d9d640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98a41030b0 0x7f98a419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 -- 172.21.15.8:0/946534985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a41026f0 msgr2=0x7f98a4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a41026f0 0x7f98a4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 -- 172.21.15.8:0/946534985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98a41030b0 msgr2=0x7f98a419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98a41030b0 0x7f98a419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 -- 172.21.15.8:0/946534985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009900 con 0x7f98a4101500 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a259c640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 0x7f98a4197560 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9c0 tx=0x7f989400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940096e0 con 0x7f98a4101500 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f989400cd80 con 0x7f98a4101500 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a419f660 con 0x7f98a4101500 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894010430 con 0x7f98a4101500 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.930+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a419fb20 con 0x7f98a4101500 2024-02-19T16:22:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.931+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a4108820 con 0x7f98a4101500 2024-02-19T16:22:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.932+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f9894003680 con 0x7f98a4101500 2024-02-19T16:22:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.933+0000 7f989b7fe640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98900a2150 0x7f98900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.933+0000 7f989bfff640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98900a2150 0x7f98900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.933+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f9894014070 con 0x7f98a4101500 2024-02-19T16:22:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.933+0000 7f989bfff640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98900a2150 0x7f98900a4610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98a4198a70 tx=0x7f988c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:38.937+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9894097040 con 0x7f98a4101500 2024-02-19T16:22:39.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.120+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98a410cf10 con 0x7f98a4101500 2024-02-19T16:22:39.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.121+0000 7f989b7fe640 1 -- 172.21.15.8:0/946534985 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f989409a700 con 0x7f98a4101500 2024-02-19T16:22:39.121 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:39.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.122+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98900a2150 msgr2=0x7f98900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.122+0000 7f98a8b13640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98900a2150 0x7f98900a4610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98a4198a70 tx=0x7f988c005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.122+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 msgr2=0x7f98a4197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.122+0000 7f98a8b13640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 0x7f98a4197560 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9c0 tx=0x7f989400ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 shutdown_connections 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f98900a2150 0x7f98900a4610 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98a41030b0 0x7f98a419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a41026f0 0x7f98a4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 --2- 172.21.15.8:0/946534985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98a4101500 0x7f98a4197560 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 >> 172.21.15.8:0/946534985 conn(0x7f98a40fca90 msgr2=0x7f98a41059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 shutdown_connections 2024-02-19T16:22:39.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.123+0000 7f98a8b13640 1 -- 172.21.15.8:0/946534985 wait complete. 2024-02-19T16:22:39.134 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:39.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 -- 172.21.15.8:0/3089786538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 msgr2=0x7fc920104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 --2- 172.21.15.8:0/3089786538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc920104380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc918009860 tx=0x7fc91802bb70 comp rx=0 tx=0).stop 2024-02-19T16:22:39.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 -- 172.21.15.8:0/3089786538 shutdown_connections 2024-02-19T16:22:39.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 --2- 172.21.15.8:0/3089786538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9201071f0 0x7fc920109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 --2- 172.21.15.8:0/3089786538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201048c0 0x7fc920106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 --2- 172.21.15.8:0/3089786538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc920104380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 -- 172.21.15.8:0/3089786538 >> 172.21.15.8:0/3089786538 conn(0x7fc9200fb9a0 msgr2=0x7fc9200fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 -- 172.21.15.8:0/3089786538 shutdown_connections 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.240+0000 7fc9253b5640 1 -- 172.21.15.8:0/3089786538 wait complete. 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 Processor -- start 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 -- start start 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc9201975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9201048c0 0x7fc920197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 0x7fc92019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92019f0c0 con 0x7fc9201071f0 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92019f230 con 0x7fc920101f90 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc9253b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92019f3f0 con 0x7fc9201048c0 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc91f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 0x7fc92019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc91effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc9201975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc91effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc9201975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:32770/0 (socket says 172.21.15.8:32770) 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc91f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 0x7fc92019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49416/0 (socket says 172.21.15.8:49416) 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc91effd640 1 -- 172.21.15.8:0/7114739 learned_addr learned my addr 172.21.15.8:0/7114739 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.241+0000 7fc91e7fc640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9201048c0 0x7fc920197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91effd640 1 -- 172.21.15.8:0/7114739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9201048c0 msgr2=0x7fc920197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91effd640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9201048c0 0x7fc920197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91effd640 1 -- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 msgr2=0x7fc92019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91effd640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 0x7fc92019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91effd640 1 -- 172.21.15.8:0/7114739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc908009560 con 0x7fc920101f90 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91f7fe640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 0x7fc92019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc91effd640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc9201975d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc918005940 tx=0x7fc9180045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91802c510 con 0x7fc920101f90 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc91802cb50 con 0x7fc920101f90 2024-02-19T16:22:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918032860 con 0x7fc920101f90 2024-02-19T16:22:39.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc918009490 con 0x7fc920101f90 2024-02-19T16:22:39.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.242+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92019f850 con 0x7fc920101f90 2024-02-19T16:22:39.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.243+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9200fd490 con 0x7fc920101f90 2024-02-19T16:22:39.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.244+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fc91803b040 con 0x7fc920101f90 2024-02-19T16:22:39.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.245+0000 7fc917fff640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc90c0a21d0 0x7fc90c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.245+0000 7fc91e7fc640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc90c0a21d0 0x7fc90c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.245+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fc9180efc80 con 0x7fc920101f90 2024-02-19T16:22:39.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.245+0000 7fc91e7fc640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc90c0a21d0 0x7fc90c0a4690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc920198ae0 tx=0x7fc908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:39.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.248+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fc9180b8990 con 0x7fc920101f90 2024-02-19T16:22:39.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.432+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc92010cf20 con 0x7fc920101f90 2024-02-19T16:22:39.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.432+0000 7fc917fff640 1 -- 172.21.15.8:0/7114739 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc9180390a0 con 0x7fc920101f90 2024-02-19T16:22:39.434 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:39.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc90c0a21d0 msgr2=0x7fc90c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc90c0a21d0 0x7fc90c0a4690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc920198ae0 tx=0x7fc908009260 comp rx=0 tx=0).stop 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 msgr2=0x7fc9201975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc9201975d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc918005940 tx=0x7fc9180045e0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 shutdown_connections 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc90c0a21d0 0x7fc90c0a4690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9201071f0 0x7fc92019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9201048c0 0x7fc920197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 --2- 172.21.15.8:0/7114739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc920101f90 0x7fc9201975d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 >> 172.21.15.8:0/7114739 conn(0x7fc9200fb9a0 msgr2=0x7fc9201028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:39.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 shutdown_connections 2024-02-19T16:22:39.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.434+0000 7fc9253b5640 1 -- 172.21.15.8:0/7114739 wait complete. 2024-02-19T16:22:39.447 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:39.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/1472116874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 msgr2=0x7fa6c8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/1472116874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8102b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009900 tx=0x7fa6b802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/1472116874 shutdown_connections 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/1472116874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c81030d0 0x7fa6c8109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/1472116874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8102b90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/1472116874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c8101520 0x7fa6c8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/1472116874 >> 172.21.15.8:0/1472116874 conn(0x7fa6c80fcad0 msgr2=0x7fa6c80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/1472116874 shutdown_connections 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.550+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/1472116874 wait complete. 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 Processor -- start 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 -- start start 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c8101520 0x7fa6c81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c81030d0 0x7fa6c819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c819f0b0 con 0x7fa6c8102710 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c819f220 con 0x7fa6c8101520 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ce9bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c819f3e0 con 0x7fa6c81030d0 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49430/0 (socket says 172.21.15.8:49430) 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 -- 172.21.15.8:0/392655188 learned_addr learned my addr 172.21.15.8:0/392655188 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c7fff640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c8101520 0x7fa6c81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6ccf5a640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c81030d0 0x7fa6c819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 -- 172.21.15.8:0/392655188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c81030d0 msgr2=0x7fa6c819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c81030d0 0x7fa6c819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 -- 172.21.15.8:0/392655188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c8101520 msgr2=0x7fa6c81975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c8101520 0x7fa6c81975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.551+0000 7fa6c77fe640 1 -- 172.21.15.8:0/392655188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b8009560 con 0x7fa6c8102710 2024-02-19T16:22:39.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.552+0000 7fa6c77fe640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8197b00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8006010 tx=0x7fa6b8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.552+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80043d0 con 0x7fa6c8102710 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.552+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c819f660 con 0x7fa6c8102710 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.552+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b80052d0 con 0x7fa6c8102710 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.552+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8003cc0 con 0x7fa6c8102710 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.552+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c819fbc0 con 0x7fa6c8102710 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.553+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8108840 con 0x7fa6c8102710 2024-02-19T16:22:39.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.554+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fa6b8005440 con 0x7fa6c8102710 2024-02-19T16:22:39.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.554+0000 7fa6c57fa640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa6b00a2150 0x7fa6b00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.554+0000 7fa6c7fff640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa6b00a2150 0x7fa6b00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.555+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fa6b80f02c0 con 0x7fa6c8102710 2024-02-19T16:22:39.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.555+0000 7fa6c7fff640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa6b00a2150 0x7fa6b00a4610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8102570 tx=0x7fa6bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:39.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.559+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fa6b80b8ea0 con 0x7fa6c8102710 2024-02-19T16:22:39.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.742+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c810d0e0 con 0x7fa6c8102710 2024-02-19T16:22:39.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.744+0000 7fa6c57fa640 1 -- 172.21.15.8:0/392655188 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7fa6b80bc560 con 0x7fa6c8102710 2024-02-19T16:22:39.745 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:39.745 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:39.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa6b00a2150 msgr2=0x7fa6b00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa6b00a2150 0x7fa6b00a4610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8102570 tx=0x7fa6bc005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:39.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 msgr2=0x7fa6c8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8197b00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8006010 tx=0x7fa6b8005e80 comp rx=0 tx=0).stop 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 shutdown_connections 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fa6b00a2150 0x7fa6b00a4610 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c81030d0 0x7fa6c819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6c8102710 0x7fa6c8197b00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 --2- 172.21.15.8:0/392655188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6c8101520 0x7fa6c81975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.746+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 >> 172.21.15.8:0/392655188 conn(0x7fa6c80fcad0 msgr2=0x7fa6c8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.747+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 shutdown_connections 2024-02-19T16:22:39.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.747+0000 7fa6ce9bd640 1 -- 172.21.15.8:0/392655188 wait complete. 2024-02-19T16:22:39.757 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:39 smithi008 ceph-mon[32573]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:39.757 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:39 smithi008 ceph-mon[32573]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-02-19T16:22:39.758 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/988479163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.758 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3597207164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.758 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/946534985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.760 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:39.761 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:39.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:39 smithi191 ceph-mon[37203]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:39 smithi191 ceph-mon[37203]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-02-19T16:22:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/988479163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3597207164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/946534985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:39 smithi156 ceph-mon[36502]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:39.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:39 smithi156 ceph-mon[36502]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-02-19T16:22:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/988479163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3597207164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/946534985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:39.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/1732568043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d341030b0 msgr2=0x7f4d34109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/1732568043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d341030b0 0x7f4d34109940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009ca0 tx=0x7f4d2c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/1732568043 shutdown_connections 2024-02-19T16:22:39.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/1732568043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d341030b0 0x7f4d34109940 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/1732568043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d341026f0 0x7f4d34102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/1732568043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d34101500 0x7f4d34101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/1732568043 >> 172.21.15.8:0/1732568043 conn(0x7f4d340fca90 msgr2=0x7f4d340feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/1732568043 shutdown_connections 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.866+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/1732568043 wait complete. 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 Processor -- start 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 -- start start 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 0x7f4d340752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d341026f0 0x7f4d340738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d341030b0 0x7f4d34073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d34074580 con 0x7f4d34101500 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340746f0 con 0x7f4d341026f0 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d3ab0c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341a15b0 con 0x7f4d341030b0 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 0x7f4d340752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 0x7f4d340752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49452/0 (socket says 172.21.15.8:49452) 2024-02-19T16:22:39.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 -- 172.21.15.8:0/143013666 learned_addr learned my addr 172.21.15.8:0/143013666 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d390a9640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d341030b0 0x7f4d34073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d33fff640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d341026f0 0x7f4d340738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 -- 172.21.15.8:0/143013666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d341030b0 msgr2=0x7f4d34073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d341030b0 0x7f4d34073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 -- 172.21.15.8:0/143013666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d341026f0 msgr2=0x7f4d340738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d341026f0 0x7f4d340738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 -- 172.21.15.8:0/143013666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28009490 con 0x7f4d34101500 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.867+0000 7f4d388a8640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 0x7f4d340752a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800e9f0 tx=0x7f4d2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.868+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800cde0 con 0x7f4d34101500 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.868+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d28010910 con 0x7f4d34101500 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.868+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c009900 con 0x7f4d34101500 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.868+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28018950 con 0x7f4d34101500 2024-02-19T16:22:39.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.868+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d341a1be0 con 0x7f4d34101500 2024-02-19T16:22:39.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.869+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d34108820 con 0x7f4d34101500 2024-02-19T16:22:39.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.870+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f4d28010a80 con 0x7f4d34101500 2024-02-19T16:22:39.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.871+0000 7f4d31ffb640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4d240a2150 0x7f4d240a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:39.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.871+0000 7f4d33fff640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4d240a2150 0x7f4d240a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:39.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.871+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f4d28014070 con 0x7f4d34101500 2024-02-19T16:22:39.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.871+0000 7f4d33fff640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4d240a2150 0x7f4d240a4610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4d34075010 tx=0x7f4d20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:39.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:39.875+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f4d28097210 con 0x7f4d34101500 2024-02-19T16:22:40.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.060+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d3410cf10 con 0x7f4d34101500 2024-02-19T16:22:40.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.061+0000 7f4d31ffb640 1 -- 172.21.15.8:0/143013666 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d2809a8d0 con 0x7f4d34101500 2024-02-19T16:22:40.062 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:40.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4d240a2150 msgr2=0x7f4d240a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4d240a2150 0x7f4d240a4610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4d34075010 tx=0x7f4d20005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:40.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 msgr2=0x7f4d340752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 0x7f4d340752a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800e9f0 tx=0x7f4d2800eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 shutdown_connections 2024-02-19T16:22:40.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f4d240a2150 0x7f4d240a4610 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d341030b0 0x7f4d34073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d341026f0 0x7f4d340738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 --2- 172.21.15.8:0/143013666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d34101500 0x7f4d340752a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 >> 172.21.15.8:0/143013666 conn(0x7f4d340fca90 msgr2=0x7f4d341059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 shutdown_connections 2024-02-19T16:22:40.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.063+0000 7f4d3ab0c640 1 -- 172.21.15.8:0/143013666 wait complete. 2024-02-19T16:22:40.075 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 -- 172.21.15.8:0/1384114469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 msgr2=0x7febb8101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 --2- 172.21.15.8:0/1384114469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb8101a60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7febac009900 tx=0x7febac02bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 -- 172.21.15.8:0/1384114469 shutdown_connections 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 --2- 172.21.15.8:0/1384114469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb8101fa0 0x7febb8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 --2- 172.21.15.8:0/1384114469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb8101a60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 --2- 172.21.15.8:0/1384114469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb81003f0 0x7febb8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 -- 172.21.15.8:0/1384114469 >> 172.21.15.8:0/1384114469 conn(0x7febb80fb980 msgr2=0x7febb80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 -- 172.21.15.8:0/1384114469 shutdown_connections 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 -- 172.21.15.8:0/1384114469 wait complete. 2024-02-19T16:22:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 Processor -- start 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 -- start start 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.180+0000 7febbf5d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb81003f0 0x7febb81964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb81969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb8101fa0 0x7febb819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819df90 con 0x7febb81015e0 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819e100 con 0x7febb81003f0 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819e2c0 con 0x7febb8101fa0 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb81969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb81969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49478/0 (socket says 172.21.15.8:49478) 2024-02-19T16:22:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 -- 172.21.15.8:0/3849024473 learned_addr learned my addr 172.21.15.8:0/3849024473 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbdb73640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb8101fa0 0x7febb819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbd372640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb81003f0 0x7febb81964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 -- 172.21.15.8:0/3849024473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb8101fa0 msgr2=0x7febb819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb8101fa0 0x7febb819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 -- 172.21.15.8:0/3849024473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb81003f0 msgr2=0x7febb81964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb81003f0 0x7febb81964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 -- 172.21.15.8:0/3849024473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8009840 con 0x7febb81015e0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbcb71640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb81969e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7febac006010 tx=0x7febac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac004df0 con 0x7febb81015e0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febac040070 con 0x7febb81015e0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac009560 con 0x7febb81015e0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac003d80 con 0x7febb81015e0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.181+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb8068a20 con 0x7febb81015e0 2024-02-19T16:22:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.182+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb8069400 con 0x7febb81015e0 2024-02-19T16:22:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.183+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7febac004f50 con 0x7febb81015e0 2024-02-19T16:22:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.184+0000 7febb67fc640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feba00a21d0 0x7feba00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.184+0000 7febbd372640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feba00a21d0 0x7feba00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.184+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7febac042070 con 0x7febb81015e0 2024-02-19T16:22:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.185+0000 7febbd372640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feba00a21d0 0x7feba00a4690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febb8101440 tx=0x7feba8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:40.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.189+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7febac0bb010 con 0x7febb81015e0 2024-02-19T16:22:40.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.370+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7febb810be00 con 0x7febb81015e0 2024-02-19T16:22:40.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.371+0000 7febb67fc640 1 -- 172.21.15.8:0/3849024473 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7febac0be6d0 con 0x7febb81015e0 2024-02-19T16:22:40.372 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:40.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feba00a21d0 msgr2=0x7feba00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feba00a21d0 0x7feba00a4690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febb8101440 tx=0x7feba8009500 comp rx=0 tx=0).stop 2024-02-19T16:22:40.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 msgr2=0x7febb81969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb81969e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7febac006010 tx=0x7febac005e80 comp rx=0 tx=0).stop 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 shutdown_connections 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7feba00a21d0 0x7feba00a4690 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb8101fa0 0x7febb819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb81015e0 0x7febb81969e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 --2- 172.21.15.8:0/3849024473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febb81003f0 0x7febb81964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 >> 172.21.15.8:0/3849024473 conn(0x7febb80fb980 msgr2=0x7febb8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 shutdown_connections 2024-02-19T16:22:40.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.373+0000 7febbf5d6640 1 -- 172.21.15.8:0/3849024473 wait complete. 2024-02-19T16:22:40.385 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 -- 172.21.15.8:0/725191384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 msgr2=0x7f3840109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 --2- 172.21.15.8:0/725191384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f3840109ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3838009ca0 tx=0x7f383802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 -- 172.21.15.8:0/725191384 shutdown_connections 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 --2- 172.21.15.8:0/725191384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f3840109ab0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 --2- 172.21.15.8:0/725191384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3840102860 0x7f3840102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 --2- 172.21.15.8:0/725191384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38400ff7f0 0x7f38400ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 -- 172.21.15.8:0/725191384 >> 172.21.15.8:0/725191384 conn(0x7f38400fb100 msgr2=0x7f38400fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 -- 172.21.15.8:0/725191384 shutdown_connections 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 -- 172.21.15.8:0/725191384 wait complete. 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.490+0000 7f38458c3640 1 Processor -- start 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 -- start start 2024-02-19T16:22:40.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38400ff7f0 0x7f3840197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3840102860 0x7f3840197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f384019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019f040 con 0x7f3840103220 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019f1b0 con 0x7f38400ff7f0 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38458c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019f370 con 0x7f3840102860 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f384019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3840102860 0x7f3840197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f384019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49496/0 (socket says 172.21.15.8:49496) 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 -- 172.21.15.8:0/1679133695 learned_addr learned my addr 172.21.15.8:0/1679133695 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383effd640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38400ff7f0 0x7f3840197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 -- 172.21.15.8:0/1679133695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3840102860 msgr2=0x7f3840197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3840102860 0x7f3840197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 -- 172.21.15.8:0/1679133695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38400ff7f0 msgr2=0x7f3840197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38400ff7f0 0x7f3840197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 -- 172.21.15.8:0/1679133695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838009900 con 0x7f3840103220 2024-02-19T16:22:40.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f383f7fe640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f384019eb00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3838037040 tx=0x7f3838009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:40.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.491+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38380329b0 con 0x7f3840103220 2024-02-19T16:22:40.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.492+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3838003e50 con 0x7f3840103220 2024-02-19T16:22:40.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.492+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384019f5f0 con 0x7f3840103220 2024-02-19T16:22:40.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.492+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38380048a0 con 0x7f3840103220 2024-02-19T16:22:40.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.492+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384019faa0 con 0x7f3840103220 2024-02-19T16:22:40.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.492+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3840108930 con 0x7f3840103220 2024-02-19T16:22:40.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.494+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f383800a040 con 0x7f3840103220 2024-02-19T16:22:40.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.494+0000 7f38448c1640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f38280a2150 0x7f38280a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.494+0000 7f383effd640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f38280a2150 0x7f38280a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.494+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f38380f0e40 con 0x7f3840103220 2024-02-19T16:22:40.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.495+0000 7f383effd640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f38280a2150 0x7f38280a4610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3840069820 tx=0x7f3830005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:40.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.499+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f38380b9ab0 con 0x7f3840103220 2024-02-19T16:22:40.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.682+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f384002d000 con 0x7f3840103220 2024-02-19T16:22:40.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.682+0000 7f38448c1640 1 -- 172.21.15.8:0/1679133695 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3838034e60 con 0x7f3840103220 2024-02-19T16:22:40.683 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f38280a2150 msgr2=0x7f38280a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f38280a2150 0x7f38280a4610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3840069820 tx=0x7f3830005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 msgr2=0x7f384019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f384019eb00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3838037040 tx=0x7f3838009360 comp rx=0 tx=0).stop 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 shutdown_connections 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f38280a2150 0x7f38280a4610 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3840103220 0x7f384019eb00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3840102860 0x7f3840197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 --2- 172.21.15.8:0/1679133695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38400ff7f0 0x7f3840197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 >> 172.21.15.8:0/1679133695 conn(0x7f38400fb100 msgr2=0x7f3840105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 shutdown_connections 2024-02-19T16:22:40.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.684+0000 7f38458c3640 1 -- 172.21.15.8:0/1679133695 wait complete. 2024-02-19T16:22:40.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:40 smithi008 ceph-mon[32573]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:22:40.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/7114739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/392655188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:40.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/143013666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3849024473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.696 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:40.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:40 smithi191 ceph-mon[37203]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:22:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/7114739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/392655188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/143013666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3849024473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 -- 172.21.15.8:0/1569660680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 msgr2=0x7fd8a8109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/1569660680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 0x7fd8a8109ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009ca0 tx=0x7fd8a002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 -- 172.21.15.8:0/1569660680 shutdown_connections 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/1569660680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 0x7fd8a8109ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/1569660680 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a8102860 0x7fd8a8102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/1569660680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a80ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 -- 172.21.15.8:0/1569660680 >> 172.21.15.8:0/1569660680 conn(0x7fd8a80fb100 msgr2=0x7fd8a80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 -- 172.21.15.8:0/1569660680 shutdown_connections 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.801+0000 7fd8aec4e640 1 -- 172.21.15.8:0/1569660680 wait complete. 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 Processor -- start 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 -- start start 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8102860 0x7fd8a8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 0x7fd8a819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a819f040 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a819f1b0 con 0x7fd8a8103220 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8aec4e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a819f370 con 0x7fd8a8102860 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a8197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a8197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49514/0 (socket says 172.21.15.8:49514) 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8a7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8102860 0x7fd8a8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 -- 172.21.15.8:0/675690730 learned_addr learned my addr 172.21.15.8:0/675690730 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ad1eb640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 0x7fd8a819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 -- 172.21.15.8:0/675690730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8102860 msgr2=0x7fd8a8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8102860 0x7fd8a8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 -- 172.21.15.8:0/675690730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 msgr2=0x7fd8a819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 0x7fd8a819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.802+0000 7fd8ac9ea640 1 -- 172.21.15.8:0/675690730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd894009490 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.803+0000 7fd8ac9ea640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a8197550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e9c0 tx=0x7fd89c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.803+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0096e0 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.803+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd89c00cd80 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.803+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a0009900 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.803+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c010430 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.803+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a819f9d0 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.804+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a8108990 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.805+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fd89c003680 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.806+0000 7fd8a5ffb640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a21d0 0x7fd8980a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:40.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.806+0000 7fd8a7fff640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a21d0 0x7fd8980a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:40.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.806+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fd89c014070 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.806+0000 7fd8a7fff640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a21d0 0x7fd8980a4690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8198a60 tx=0x7fd894005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:40.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.810+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd89c096fe0 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:40 smithi156 ceph-mon[36502]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:22:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/7114739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/392655188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/143013666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3849024473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:40.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.992+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8a810cf30 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.992+0000 7fd8a5ffb640 1 -- 172.21.15.8:0/675690730 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd89c09a6a0 con 0x7fd8a80ff7f0 2024-02-19T16:22:40.993 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a21d0 msgr2=0x7fd8980a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a21d0 0x7fd8980a4690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8198a60 tx=0x7fd894005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 msgr2=0x7fd8a8197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a8197550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e9c0 tx=0x7fd89c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 shutdown_connections 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd8980a21d0 0x7fd8980a4690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8103220 0x7fd8a819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8102860 0x7fd8a8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 --2- 172.21.15.8:0/675690730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8a80ff7f0 0x7fd8a8197550 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:40.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 >> 172.21.15.8:0/675690730 conn(0x7fd8a80fb100 msgr2=0x7fd8a8105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:40.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 shutdown_connections 2024-02-19T16:22:40.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:40.994+0000 7fd8aec4e640 1 -- 172.21.15.8:0/675690730 wait complete. 2024-02-19T16:22:41.006 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-19T16:22:41.006 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:41.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.108+0000 7fbb6728f640 1 -- 172.21.15.8:0/421818217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101600 msgr2=0x7fbb60101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.108+0000 7fbb6728f640 1 --2- 172.21.15.8:0/421818217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101600 0x7fbb60101a80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009900 tx=0x7fbb5402bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- 172.21.15.8:0/421818217 shutdown_connections 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 --2- 172.21.15.8:0/421818217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb60101fc0 0x7fbb60108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 --2- 172.21.15.8:0/421818217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101600 0x7fbb60101a80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 --2- 172.21.15.8:0/421818217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100410 0x7fbb60100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- 172.21.15.8:0/421818217 >> 172.21.15.8:0/421818217 conn(0x7fbb600fb980 msgr2=0x7fbb600fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- 172.21.15.8:0/421818217 shutdown_connections 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- 172.21.15.8:0/421818217 wait complete. 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 Processor -- start 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- start start 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100410 0x7fbb601964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb60101600 0x7fbb601969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 0x7fbb6019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6019df90 con 0x7fbb60101fc0 2024-02-19T16:22:41.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6019e100 con 0x7fbb60100410 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6728f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6019e2c0 con 0x7fbb60101600 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6582c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 0x7fbb6019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.109+0000 7fbb6582c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 0x7fbb6019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49534/0 (socket says 172.21.15.8:49534) 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 -- 172.21.15.8:0/2642237061 learned_addr learned my addr 172.21.15.8:0/2642237061 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6502b640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100410 0x7fbb601964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6482a640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb60101600 0x7fbb601969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 -- 172.21.15.8:0/2642237061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb60101600 msgr2=0x7fbb601969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb60101600 0x7fbb601969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 -- 172.21.15.8:0/2642237061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100410 msgr2=0x7fbb601964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100410 0x7fbb601964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 -- 172.21.15.8:0/2642237061 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb54009560 con 0x7fbb60101fc0 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6582c640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 0x7fbb6019da50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e970 tx=0x7fbb5000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50009720 con 0x7fbb60101fc0 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb50005500 con 0x7fbb60101fc0 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6019e5a0 con 0x7fbb60101fc0 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50010430 con 0x7fbb60101fc0 2024-02-19T16:22:41.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.110+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6019ead0 con 0x7fbb60101fc0 2024-02-19T16:22:41.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.111+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb60107730 con 0x7fbb60101fc0 2024-02-19T16:22:41.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.112+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fbb50004ff0 con 0x7fbb60101fc0 2024-02-19T16:22:41.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.113+0000 7fbb5e7fc640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb480a21d0 0x7fbb480a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.113+0000 7fbb6502b640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb480a21d0 0x7fbb480a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.113+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fbb50014070 con 0x7fbb60101fc0 2024-02-19T16:22:41.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.113+0000 7fbb6502b640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb480a21d0 0x7fbb480a4690 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbb60101460 tx=0x7fbb58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:41.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.117+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbb50096f70 con 0x7fbb60101fc0 2024-02-19T16:22:41.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.339+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb6010be20 con 0x7fbb60101fc0 2024-02-19T16:22:41.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.339+0000 7fbb5e7fc640 1 -- 172.21.15.8:0/2642237061 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fbb5009a630 con 0x7fbb60101fc0 2024-02-19T16:22:41.340 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:41.340 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:22:36.842920+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:41.341 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2024-02-19T16:22:41.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.341+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb480a21d0 msgr2=0x7fbb480a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.341+0000 7fbb6728f640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb480a21d0 0x7fbb480a4690 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbb60101460 tx=0x7fbb58009500 comp rx=0 tx=0).stop 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.341+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 msgr2=0x7fbb6019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.341+0000 7fbb6728f640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 0x7fbb6019da50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e970 tx=0x7fbb5000ed70 comp rx=0 tx=0).stop 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 shutdown_connections 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbb480a21d0 0x7fbb480a4690 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb60101fc0 0x7fbb6019da50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb60101600 0x7fbb601969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 --2- 172.21.15.8:0/2642237061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100410 0x7fbb601964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 >> 172.21.15.8:0/2642237061 conn(0x7fbb600fb980 msgr2=0x7fbb60104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 shutdown_connections 2024-02-19T16:22:41.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.342+0000 7fbb6728f640 1 -- 172.21.15.8:0/2642237061 wait complete. 2024-02-19T16:22:41.353 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-19T16:22:41.354 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-19T16:22:41.354 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-19T16:22:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 -- 172.21.15.8:0/2560114087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8100410 msgr2=0x7f55d8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 --2- 172.21.15.8:0/2560114087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8100410 0x7f55d8100830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009900 tx=0x7f55d002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 -- 172.21.15.8:0/2560114087 shutdown_connections 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 --2- 172.21.15.8:0/2560114087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55d8101fc0 0x7f55d8108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 --2- 172.21.15.8:0/2560114087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8101600 0x7f55d8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 --2- 172.21.15.8:0/2560114087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8100410 0x7f55d8100830 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 -- 172.21.15.8:0/2560114087 >> 172.21.15.8:0/2560114087 conn(0x7f55d80fb980 msgr2=0x7f55d80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 -- 172.21.15.8:0/2560114087 shutdown_connections 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.456+0000 7f55dec8f640 1 -- 172.21.15.8:0/2560114087 wait complete. 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 Processor -- start 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 -- start start 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8100410 0x7f55d81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55d8101600 0x7f55d8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 0x7f55d819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d819f090 con 0x7f55d8101fc0 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d819f200 con 0x7f55d8100410 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dec8f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d819f3c0 con 0x7f55d8101600 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dd22c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 0x7f55d819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dca2b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8100410 0x7f55d81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dd22c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 0x7f55d819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49560/0 (socket says 172.21.15.8:49560) 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55dd22c640 1 -- 172.21.15.8:0/3213525594 learned_addr learned my addr 172.21.15.8:0/3213525594 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.457+0000 7f55d7fff640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55d8101600 0x7f55d8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dd22c640 1 -- 172.21.15.8:0/3213525594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55d8101600 msgr2=0x7f55d8197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dd22c640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55d8101600 0x7f55d8197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dd22c640 1 -- 172.21.15.8:0/3213525594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8100410 msgr2=0x7f55d81975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dd22c640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8100410 0x7f55d81975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dd22c640 1 -- 172.21.15.8:0/3213525594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d0009560 con 0x7f55d8101fc0 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dd22c640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 0x7f55d819eb50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e970 tx=0x7f55c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0009720 con 0x7f55d8101fc0 2024-02-19T16:22:41.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55c0005500 con 0x7f55d8101fc0 2024-02-19T16:22:41.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d819f6a0 con 0x7f55d8101fc0 2024-02-19T16:22:41.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0010430 con 0x7f55d8101fc0 2024-02-19T16:22:41.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.458+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d819fb60 con 0x7f55d8101fc0 2024-02-19T16:22:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.459+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d810be70 con 0x7f55d8101fc0 2024-02-19T16:22:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.460+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f55c0005020 con 0x7f55d8101fc0 2024-02-19T16:22:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.461+0000 7f55d5ffb640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f55c40a2150 0x7f55c40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.461+0000 7f55dca2b640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f55c40a2150 0x7f55c40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.461+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f55c0014070 con 0x7f55d8101fc0 2024-02-19T16:22:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.461+0000 7f55dca2b640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f55c40a2150 0x7f55c40a4610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55d002c240 tx=0x7f55d002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:41.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.465+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f55c0097040 con 0x7f55d8101fc0 2024-02-19T16:22:41.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.647+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55d80fdb30 con 0x7f55c40a2150 2024-02-19T16:22:41.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.660+0000 7f55d5ffb640 1 -- 172.21.15.8:0/3213525594 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428108 (secure 0 0 0) 0x7f55d80fdb30 con 0x7f55c40a2150 2024-02-19T16:22:41.666 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:41.669 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:22:41.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f55c40a2150 msgr2=0x7f55c40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f55c40a2150 0x7f55c40a4610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55d002c240 tx=0x7f55d002c7e0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 msgr2=0x7f55d819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 0x7f55d819eb50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e970 tx=0x7f55c000ed70 comp rx=0 tx=0).stop 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 shutdown_connections 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f55c40a2150 0x7f55c40a4610 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55d8101fc0 0x7f55d819eb50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55d8101600 0x7f55d8197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 --2- 172.21.15.8:0/3213525594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8100410 0x7f55d81975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 >> 172.21.15.8:0/3213525594 conn(0x7f55d80fb980 msgr2=0x7f55d80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 shutdown_connections 2024-02-19T16:22:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.664+0000 7f55dec8f640 1 -- 172.21.15.8:0/3213525594 wait complete. 2024-02-19T16:22:41.675 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1679133695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:41.675 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/675690730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:41.675 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2642237061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:41.676 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-02-19T16:22:40.748176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":868,"num_read_kb":561,"num_write":356,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378548,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563276,"statfs":{"total":1151940427776,"available":1151552794624,"internally_reserved":0,"allocated":12992512,"data_stored":8345637,"data_compressed":468924,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003373"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450074+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:22:22.450074+0000","last_peered":"2024-02-19T16:22:22.450074+0000","last_clean":"2024-02-19T16:22:22.450074+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:22:22.450074+0000","last_undegraded":"2024-02-19T16:22:22.450074+0000","last_fullsized":"2024-02-19T16:22:22.450074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450435+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:22:22.450435+0000","last_peered":"2024-02-19T16:22:22.450435+0000","last_clean":"2024-02-19T16:22:22.450435+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:22:22.450435+0000","last_undegraded":"2024-02-19T16:22:22.450435+0000","last_fullsized":"2024-02-19T16:22:22.450435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450286+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:22:22.450286+0000","last_peered":"2024-02-19T16:22:22.450286+0000","last_clean":"2024-02-19T16:22:22.450286+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:22:22.450286+0000","last_undegraded":"2024-02-19T16:22:22.450286+0000","last_fullsized":"2024-02-19T16:22:22.450286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349393+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:22:22.349393+0000","last_peered":"2024-02-19T16:22:22.349393+0000","last_clean":"2024-02-19T16:22:22.349393+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:22:22.349393+0000","last_undegraded":"2024-02-19T16:22:22.349393+0000","last_fullsized":"2024-02-19T16:22:22.349393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351320+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:22:22.351320+0000","last_peered":"2024-02-19T16:22:22.351320+0000","last_clean":"2024-02-19T16:22:22.351320+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:22:22.351320+0000","last_undegraded":"2024-02-19T16:22:22.351320+0000","last_fullsized":"2024-02-19T16:22:22.351320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428710+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:22:22.428710+0000","last_peered":"2024-02-19T16:22:22.428710+0000","last_clean":"2024-02-19T16:22:22.428710+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:22:22.428710+0000","last_undegraded":"2024-02-19T16:22:22.428710+0000","last_fullsized":"2024-02-19T16:22:22.428710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350749+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:22:22.350749+0000","last_peered":"2024-02-19T16:22:22.350749+0000","last_clean":"2024-02-19T16:22:22.350749+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:22:22.350749+0000","last_undegraded":"2024-02-19T16:22:22.350749+0000","last_fullsized":"2024-02-19T16:22:22.350749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348892+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:22:22.348892+0000","last_peered":"2024-02-19T16:22:22.348892+0000","last_clean":"2024-02-19T16:22:22.348892+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:22:22.348892+0000","last_undegraded":"2024-02-19T16:22:22.348892+0000","last_fullsized":"2024-02-19T16:22:22.348892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449821+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:22:22.449821+0000","last_peered":"2024-02-19T16:22:22.449821+0000","last_clean":"2024-02-19T16:22:22.449821+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:22:22.449821+0000","last_undegraded":"2024-02-19T16:22:22.449821+0000","last_fullsized":"2024-02-19T16:22:22.449821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428995+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:22:22.428995+0000","last_peered":"2024-02-19T16:22:22.428995+0000","last_clean":"2024-02-19T16:22:22.428995+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:22:22.428995+0000","last_undegraded":"2024-02-19T16:22:22.428995+0000","last_fullsized":"2024-02-19T16:22:22.428995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351456+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:22:22.351456+0000","last_peered":"2024-02-19T16:22:22.351456+0000","last_clean":"2024-02-19T16:22:22.351456+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:22:22.351456+0000","last_undegraded":"2024-02-19T16:22:22.351456+0000","last_fullsized":"2024-02-19T16:22:22.351456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349352+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:22:22.349352+0000","last_peered":"2024-02-19T16:22:22.349352+0000","last_clean":"2024-02-19T16:22:22.349352+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:22:22.349352+0000","last_undegraded":"2024-02-19T16:22:22.349352+0000","last_fullsized":"2024-02-19T16:22:22.349352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354017+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:22:22.354017+0000","last_peered":"2024-02-19T16:22:22.354017+0000","last_clean":"2024-02-19T16:22:22.354017+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:22:22.354017+0000","last_undegraded":"2024-02-19T16:22:22.354017+0000","last_fullsized":"2024-02-19T16:22:22.354017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450028+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:22:22.450028+0000","last_peered":"2024-02-19T16:22:22.450028+0000","last_clean":"2024-02-19T16:22:22.450028+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:22:22.450028+0000","last_undegraded":"2024-02-19T16:22:22.450028+0000","last_fullsized":"2024-02-19T16:22:22.450028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450061+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:22:22.450061+0000","last_peered":"2024-02-19T16:22:22.450061+0000","last_clean":"2024-02-19T16:22:22.450061+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:22:22.450061+0000","last_undegraded":"2024-02-19T16:22:22.450061+0000","last_fullsized":"2024-02-19T16:22:22.450061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449363+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:22:22.449363+0000","last_peered":"2024-02-19T16:22:22.449363+0000","last_clean":"2024-02-19T16:22:22.449363+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:22:22.449363+0000","last_undegraded":"2024-02-19T16:22:22.449363+0000","last_fullsized":"2024-02-19T16:22:22.449363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449854+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:22:22.449854+0000","last_peered":"2024-02-19T16:22:22.449854+0000","last_clean":"2024-02-19T16:22:22.449854+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:22:22.449854+0000","last_undegraded":"2024-02-19T16:22:22.449854+0000","last_fullsized":"2024-02-19T16:22:22.449854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351299+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:22:22.351299+0000","last_peered":"2024-02-19T16:22:22.351299+0000","last_clean":"2024-02-19T16:22:22.351299+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:22:22.351299+0000","last_undegraded":"2024-02-19T16:22:22.351299+0000","last_fullsized":"2024-02-19T16:22:22.351299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353320+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:22:22.353320+0000","last_peered":"2024-02-19T16:22:22.353320+0000","last_clean":"2024-02-19T16:22:22.353320+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:22:22.353320+0000","last_undegraded":"2024-02-19T16:22:22.353320+0000","last_fullsized":"2024-02-19T16:22:22.353320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352554+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:22:22.352554+0000","last_peered":"2024-02-19T16:22:22.352554+0000","last_clean":"2024-02-19T16:22:22.352554+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:22:22.352554+0000","last_undegraded":"2024-02-19T16:22:22.352554+0000","last_fullsized":"2024-02-19T16:22:22.352554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351551+0000","last_change":"2024-02-19T16:21:21.892098+0000","last_active":"2024-02-19T16:22:22.351551+0000","last_peered":"2024-02-19T16:22:22.351551+0000","last_clean":"2024-02-19T16:22:22.351551+0000","last_became_active":"2024-02-19T16:21:21.884753+0000","last_became_peered":"2024-02-19T16:21:21.884753+0000","last_unstale":"2024-02-19T16:22:22.351551+0000","last_undegraded":"2024-02-19T16:22:22.351551+0000","last_fullsized":"2024-02-19T16:22:22.351551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:24.360019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355648+0000","last_change":"2024-02-19T16:22:22.355648+0000","last_active":"2024-02-19T16:22:22.355648+0000","last_peered":"2024-02-19T16:22:22.355648+0000","last_clean":"2024-02-19T16:22:22.355648+0000","last_became_active":"2024-02-19T16:22:22.355213+0000","last_became_peered":"2024-02-19T16:22:22.355213+0000","last_unstale":"2024-02-19T16:22:22.355648+0000","last_undegraded":"2024-02-19T16:22:22.355648+0000","last_fullsized":"2024-02-19T16:22:22.355648+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:49:24.360051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348894+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:22:22.348894+0000","last_peered":"2024-02-19T16:22:22.348894+0000","last_clean":"2024-02-19T16:22:22.348894+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:22:22.348894+0000","last_undegraded":"2024-02-19T16:22:22.348894+0000","last_fullsized":"2024-02-19T16:22:22.348894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350052+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:22:22.350052+0000","last_peered":"2024-02-19T16:22:22.350052+0000","last_clean":"2024-02-19T16:22:22.350052+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:22:22.350052+0000","last_undegraded":"2024-02-19T16:22:22.350052+0000","last_fullsized":"2024-02-19T16:22:22.350052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450474+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:22:22.450474+0000","last_peered":"2024-02-19T16:22:22.450474+0000","last_clean":"2024-02-19T16:22:22.450474+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:22:22.450474+0000","last_undegraded":"2024-02-19T16:22:22.450474+0000","last_fullsized":"2024-02-19T16:22:22.450474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349058+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:22:22.349058+0000","last_peered":"2024-02-19T16:22:22.349058+0000","last_clean":"2024-02-19T16:22:22.349058+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:22:22.349058+0000","last_undegraded":"2024-02-19T16:22:22.349058+0000","last_fullsized":"2024-02-19T16:22:22.349058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351817+0000","last_change":"2024-02-19T16:21:21.889760+0000","last_active":"2024-02-19T16:22:22.351817+0000","last_peered":"2024-02-19T16:22:22.351817+0000","last_clean":"2024-02-19T16:22:22.351817+0000","last_became_active":"2024-02-19T16:21:21.884230+0000","last_became_peered":"2024-02-19T16:21:21.884230+0000","last_unstale":"2024-02-19T16:22:22.351817+0000","last_undegraded":"2024-02-19T16:22:22.351817+0000","last_fullsized":"2024-02-19T16:22:22.351817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:48:45.580976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449511+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:22:22.449511+0000","last_peered":"2024-02-19T16:22:22.449511+0000","last_clean":"2024-02-19T16:22:22.449511+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:22:22.449511+0000","last_undegraded":"2024-02-19T16:22:22.449511+0000","last_fullsized":"2024-02-19T16:22:22.449511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349008+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:22:22.349008+0000","last_peered":"2024-02-19T16:22:22.349008+0000","last_clean":"2024-02-19T16:22:22.349008+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:22:22.349008+0000","last_undegraded":"2024-02-19T16:22:22.349008+0000","last_fullsized":"2024-02-19T16:22:22.349008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351467+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:22:22.351467+0000","last_peered":"2024-02-19T16:22:22.351467+0000","last_clean":"2024-02-19T16:22:22.351467+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:22:22.351467+0000","last_undegraded":"2024-02-19T16:22:22.351467+0000","last_fullsized":"2024-02-19T16:22:22.351467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354355+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:22:22.354355+0000","last_peered":"2024-02-19T16:22:22.354355+0000","last_clean":"2024-02-19T16:22:22.354355+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:22:22.354355+0000","last_undegraded":"2024-02-19T16:22:22.354355+0000","last_fullsized":"2024-02-19T16:22:22.354355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353349+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:22:22.353349+0000","last_peered":"2024-02-19T16:22:22.353349+0000","last_clean":"2024-02-19T16:22:22.353349+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:22:22.353349+0000","last_undegraded":"2024-02-19T16:22:22.353349+0000","last_fullsized":"2024-02-19T16:22:22.353349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":159,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:38.445988+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:22:38.445988+0000","last_peered":"2024-02-19T16:22:38.445988+0000","last_clean":"2024-02-19T16:22:38.445988+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:22:38.445988+0000","last_undegraded":"2024-02-19T16:22:38.445988+0000","last_fullsized":"2024-02-19T16:22:38.445988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351396+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:22:22.351396+0000","last_peered":"2024-02-19T16:22:22.351396+0000","last_clean":"2024-02-19T16:22:22.351396+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:22:22.351396+0000","last_undegraded":"2024-02-19T16:22:22.351396+0000","last_fullsized":"2024-02-19T16:22:22.351396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352924+0000","last_change":"2024-02-19T16:21:21.885536+0000","last_active":"2024-02-19T16:22:22.352924+0000","last_peered":"2024-02-19T16:22:22.352924+0000","last_clean":"2024-02-19T16:22:22.352924+0000","last_became_active":"2024-02-19T16:21:21.885125+0000","last_became_peered":"2024-02-19T16:21:21.885125+0000","last_unstale":"2024-02-19T16:22:22.352924+0000","last_undegraded":"2024-02-19T16:22:22.352924+0000","last_fullsized":"2024-02-19T16:22:22.352924+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:23:23.180070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428691+0000","last_change":"2024-02-19T16:21:22.344872+0000","last_active":"2024-02-19T16:22:22.428691+0000","last_peered":"2024-02-19T16:22:22.428691+0000","last_clean":"2024-02-19T16:22:22.428691+0000","last_became_active":"2024-02-19T16:21:22.342317+0000","last_became_peered":"2024-02-19T16:21:22.342317+0000","last_unstale":"2024-02-19T16:22:22.428691+0000","last_undegraded":"2024-02-19T16:22:22.428691+0000","last_fullsized":"2024-02-19T16:22:22.428691+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:53:45.080091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354273+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:22:22.354273+0000","last_peered":"2024-02-19T16:22:22.354273+0000","last_clean":"2024-02-19T16:22:22.354273+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:22:22.354273+0000","last_undegraded":"2024-02-19T16:22:22.354273+0000","last_fullsized":"2024-02-19T16:22:22.354273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449894+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:22:22.449894+0000","last_peered":"2024-02-19T16:22:22.449894+0000","last_clean":"2024-02-19T16:22:22.449894+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:22:22.449894+0000","last_undegraded":"2024-02-19T16:22:22.449894+0000","last_fullsized":"2024-02-19T16:22:22.449894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353391+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:22:22.353391+0000","last_peered":"2024-02-19T16:22:22.353391+0000","last_clean":"2024-02-19T16:22:22.353391+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:22:22.353391+0000","last_undegraded":"2024-02-19T16:22:22.353391+0000","last_fullsized":"2024-02-19T16:22:22.353391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349154+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:22:22.349154+0000","last_peered":"2024-02-19T16:22:22.349154+0000","last_clean":"2024-02-19T16:22:22.349154+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:22:22.349154+0000","last_undegraded":"2024-02-19T16:22:22.349154+0000","last_fullsized":"2024-02-19T16:22:22.349154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450308+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:22:22.450308+0000","last_peered":"2024-02-19T16:22:22.450308+0000","last_clean":"2024-02-19T16:22:22.450308+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:22:22.450308+0000","last_undegraded":"2024-02-19T16:22:22.450308+0000","last_fullsized":"2024-02-19T16:22:22.450308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353726+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:22:22.353726+0000","last_peered":"2024-02-19T16:22:22.353726+0000","last_clean":"2024-02-19T16:22:22.353726+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:22:22.353726+0000","last_undegraded":"2024-02-19T16:22:22.353726+0000","last_fullsized":"2024-02-19T16:22:22.353726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350686+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:22:22.350686+0000","last_peered":"2024-02-19T16:22:22.350686+0000","last_clean":"2024-02-19T16:22:22.350686+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:22:22.350686+0000","last_undegraded":"2024-02-19T16:22:22.350686+0000","last_fullsized":"2024-02-19T16:22:22.350686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349021+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:22:22.349021+0000","last_peered":"2024-02-19T16:22:22.349021+0000","last_clean":"2024-02-19T16:22:22.349021+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:22:22.349021+0000","last_undegraded":"2024-02-19T16:22:22.349021+0000","last_fullsized":"2024-02-19T16:22:22.349021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:37.444275+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:22:37.444275+0000","last_peered":"2024-02-19T16:22:37.444275+0000","last_clean":"2024-02-19T16:22:37.444275+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:22:37.444275+0000","last_undegraded":"2024-02-19T16:22:37.444275+0000","last_fullsized":"2024-02-19T16:22:37.444275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449813+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:22:22.449813+0000","last_peered":"2024-02-19T16:22:22.449813+0000","last_clean":"2024-02-19T16:22:22.449813+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:22:22.449813+0000","last_undegraded":"2024-02-19T16:22:22.449813+0000","last_fullsized":"2024-02-19T16:22:22.449813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351146+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:22:22.351146+0000","last_peered":"2024-02-19T16:22:22.351146+0000","last_clean":"2024-02-19T16:22:22.351146+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:22:22.351146+0000","last_undegraded":"2024-02-19T16:22:22.351146+0000","last_fullsized":"2024-02-19T16:22:22.351146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351218+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:22:22.351218+0000","last_peered":"2024-02-19T16:22:22.351218+0000","last_clean":"2024-02-19T16:22:22.351218+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:22:22.351218+0000","last_undegraded":"2024-02-19T16:22:22.351218+0000","last_fullsized":"2024-02-19T16:22:22.351218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450461+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:22:22.450461+0000","last_peered":"2024-02-19T16:22:22.450461+0000","last_clean":"2024-02-19T16:22:22.450461+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:22:22.450461+0000","last_undegraded":"2024-02-19T16:22:22.450461+0000","last_fullsized":"2024-02-19T16:22:22.450461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349162+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:22:22.349162+0000","last_peered":"2024-02-19T16:22:22.349162+0000","last_clean":"2024-02-19T16:22:22.349162+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:22:22.349162+0000","last_undegraded":"2024-02-19T16:22:22.349162+0000","last_fullsized":"2024-02-19T16:22:22.349162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354277+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:22:22.354277+0000","last_peered":"2024-02-19T16:22:22.354277+0000","last_clean":"2024-02-19T16:22:22.354277+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:22:22.354277+0000","last_undegraded":"2024-02-19T16:22:22.354277+0000","last_fullsized":"2024-02-19T16:22:22.354277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449706+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:22:22.449706+0000","last_peered":"2024-02-19T16:22:22.449706+0000","last_clean":"2024-02-19T16:22:22.449706+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:22:22.449706+0000","last_undegraded":"2024-02-19T16:22:22.449706+0000","last_fullsized":"2024-02-19T16:22:22.449706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428529+0000","last_change":"2024-02-19T16:21:21.885002+0000","last_active":"2024-02-19T16:22:22.428529+0000","last_peered":"2024-02-19T16:22:22.428529+0000","last_clean":"2024-02-19T16:22:22.428529+0000","last_became_active":"2024-02-19T16:21:21.884557+0000","last_became_peered":"2024-02-19T16:21:21.884557+0000","last_unstale":"2024-02-19T16:22:22.428529+0000","last_undegraded":"2024-02-19T16:22:22.428529+0000","last_fullsized":"2024-02-19T16:22:22.428529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:08:31.859577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'37","reported_seq":315,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:38.445992+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:22:38.445992+0000","last_peered":"2024-02-19T16:22:38.445992+0000","last_clean":"2024-02-19T16:22:38.445992+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:22:38.445992+0000","last_undegraded":"2024-02-19T16:22:38.445992+0000","last_fullsized":"2024-02-19T16:22:38.445992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":190,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351523+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:22:22.351523+0000","last_peered":"2024-02-19T16:22:22.351523+0000","last_clean":"2024-02-19T16:22:22.351523+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:22:22.351523+0000","last_undegraded":"2024-02-19T16:22:22.351523+0000","last_fullsized":"2024-02-19T16:22:22.351523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354566+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:22:22.354566+0000","last_peered":"2024-02-19T16:22:22.354566+0000","last_clean":"2024-02-19T16:22:22.354566+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:22:22.354566+0000","last_undegraded":"2024-02-19T16:22:22.354566+0000","last_fullsized":"2024-02-19T16:22:22.354566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352980+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:22:22.352980+0000","last_peered":"2024-02-19T16:22:22.352980+0000","last_clean":"2024-02-19T16:22:22.352980+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:22:22.352980+0000","last_undegraded":"2024-02-19T16:22:22.352980+0000","last_fullsized":"2024-02-19T16:22:22.352980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354094+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:22:22.354094+0000","last_peered":"2024-02-19T16:22:22.354094+0000","last_clean":"2024-02-19T16:22:22.354094+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:22:22.354094+0000","last_undegraded":"2024-02-19T16:22:22.354094+0000","last_fullsized":"2024-02-19T16:22:22.354094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449654+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:22:22.449654+0000","last_peered":"2024-02-19T16:22:22.449654+0000","last_clean":"2024-02-19T16:22:22.449654+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:22:22.449654+0000","last_undegraded":"2024-02-19T16:22:22.449654+0000","last_fullsized":"2024-02-19T16:22:22.449654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354252+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:22:22.354252+0000","last_peered":"2024-02-19T16:22:22.354252+0000","last_clean":"2024-02-19T16:22:22.354252+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:22:22.354252+0000","last_undegraded":"2024-02-19T16:22:22.354252+0000","last_fullsized":"2024-02-19T16:22:22.354252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355544+0000","last_change":"2024-02-19T16:22:22.355544+0000","last_active":"2024-02-19T16:22:22.355544+0000","last_peered":"2024-02-19T16:22:22.355544+0000","last_clean":"2024-02-19T16:22:22.355544+0000","last_became_active":"2024-02-19T16:22:22.355184+0000","last_became_peered":"2024-02-19T16:22:22.355184+0000","last_unstale":"2024-02-19T16:22:22.355544+0000","last_undegraded":"2024-02-19T16:22:22.355544+0000","last_fullsized":"2024-02-19T16:22:22.355544+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:48:26.236286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351943+0000","last_change":"2024-02-19T16:21:22.216357+0000","last_active":"2024-02-19T16:22:22.351943+0000","last_peered":"2024-02-19T16:22:22.351943+0000","last_clean":"2024-02-19T16:22:22.351943+0000","last_became_active":"2024-02-19T16:21:22.215883+0000","last_became_peered":"2024-02-19T16:21:22.215883+0000","last_unstale":"2024-02-19T16:22:22.351943+0000","last_undegraded":"2024-02-19T16:22:22.351943+0000","last_fullsized":"2024-02-19T16:22:22.351943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:32:43.235515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":157,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:34.460388+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:22:34.460388+0000","last_peered":"2024-02-19T16:22:34.460388+0000","last_clean":"2024-02-19T16:22:34.460388+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:22:34.460388+0000","last_undegraded":"2024-02-19T16:22:34.460388+0000","last_fullsized":"2024-02-19T16:22:34.460388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350839+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:22:22.350839+0000","last_peered":"2024-02-19T16:22:22.350839+0000","last_clean":"2024-02-19T16:22:22.350839+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:22:22.350839+0000","last_undegraded":"2024-02-19T16:22:22.350839+0000","last_fullsized":"2024-02-19T16:22:22.350839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354823+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:22:22.354823+0000","last_peered":"2024-02-19T16:22:22.354823+0000","last_clean":"2024-02-19T16:22:22.354823+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:22:22.354823+0000","last_undegraded":"2024-02-19T16:22:22.354823+0000","last_fullsized":"2024-02-19T16:22:22.354823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351234+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:22:22.351234+0000","last_peered":"2024-02-19T16:22:22.351234+0000","last_clean":"2024-02-19T16:22:22.351234+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:22:22.351234+0000","last_undegraded":"2024-02-19T16:22:22.351234+0000","last_fullsized":"2024-02-19T16:22:22.351234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353983+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:22:22.353983+0000","last_peered":"2024-02-19T16:22:22.353983+0000","last_clean":"2024-02-19T16:22:22.353983+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:22:22.353983+0000","last_undegraded":"2024-02-19T16:22:22.353983+0000","last_fullsized":"2024-02-19T16:22:22.353983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353999+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:22:22.353999+0000","last_peered":"2024-02-19T16:22:22.353999+0000","last_clean":"2024-02-19T16:22:22.353999+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:22:22.353999+0000","last_undegraded":"2024-02-19T16:22:22.353999+0000","last_fullsized":"2024-02-19T16:22:22.353999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352102+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:22:22.352102+0000","last_peered":"2024-02-19T16:22:22.352102+0000","last_clean":"2024-02-19T16:22:22.352102+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:22:22.352102+0000","last_undegraded":"2024-02-19T16:22:22.352102+0000","last_fullsized":"2024-02-19T16:22:22.352102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449849+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:22:22.449849+0000","last_peered":"2024-02-19T16:22:22.449849+0000","last_clean":"2024-02-19T16:22:22.449849+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:22:22.449849+0000","last_undegraded":"2024-02-19T16:22:22.449849+0000","last_fullsized":"2024-02-19T16:22:22.449849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351345+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:22:22.351345+0000","last_peered":"2024-02-19T16:22:22.351345+0000","last_clean":"2024-02-19T16:22:22.351345+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:22:22.351345+0000","last_undegraded":"2024-02-19T16:22:22.351345+0000","last_fullsized":"2024-02-19T16:22:22.351345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351655+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:22:22.351655+0000","last_peered":"2024-02-19T16:22:22.351655+0000","last_clean":"2024-02-19T16:22:22.351655+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:22:22.351655+0000","last_undegraded":"2024-02-19T16:22:22.351655+0000","last_fullsized":"2024-02-19T16:22:22.351655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428974+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:22:22.428974+0000","last_peered":"2024-02-19T16:22:22.428974+0000","last_clean":"2024-02-19T16:22:22.428974+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:22:22.428974+0000","last_undegraded":"2024-02-19T16:22:22.428974+0000","last_fullsized":"2024-02-19T16:22:22.428974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354592+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:22:22.354592+0000","last_peered":"2024-02-19T16:22:22.354592+0000","last_clean":"2024-02-19T16:22:22.354592+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:22:22.354592+0000","last_undegraded":"2024-02-19T16:22:22.354592+0000","last_fullsized":"2024-02-19T16:22:22.354592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350901+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:22:22.350901+0000","last_peered":"2024-02-19T16:22:22.350901+0000","last_clean":"2024-02-19T16:22:22.350901+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:22:22.350901+0000","last_undegraded":"2024-02-19T16:22:22.350901+0000","last_fullsized":"2024-02-19T16:22:22.350901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428794+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:22:22.428794+0000","last_peered":"2024-02-19T16:22:22.428794+0000","last_clean":"2024-02-19T16:22:22.428794+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:22:22.428794+0000","last_undegraded":"2024-02-19T16:22:22.428794+0000","last_fullsized":"2024-02-19T16:22:22.428794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355307+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:22:22.355307+0000","last_peered":"2024-02-19T16:22:22.355307+0000","last_clean":"2024-02-19T16:22:22.355307+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:22:22.355307+0000","last_undegraded":"2024-02-19T16:22:22.355307+0000","last_fullsized":"2024-02-19T16:22:22.355307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353828+0000","last_change":"2024-02-19T16:22:22.353828+0000","last_active":"2024-02-19T16:22:22.353828+0000","last_peered":"2024-02-19T16:22:22.353828+0000","last_clean":"2024-02-19T16:22:22.353828+0000","last_became_active":"2024-02-19T16:22:22.353369+0000","last_became_peered":"2024-02-19T16:22:22.353369+0000","last_unstale":"2024-02-19T16:22:22.353828+0000","last_undegraded":"2024-02-19T16:22:22.353828+0000","last_fullsized":"2024-02-19T16:22:22.353828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:26:32.976202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":155,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:35.461056+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:22:35.461056+0000","last_peered":"2024-02-19T16:22:35.461056+0000","last_clean":"2024-02-19T16:22:35.461056+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:22:35.461056+0000","last_undegraded":"2024-02-19T16:22:35.461056+0000","last_fullsized":"2024-02-19T16:22:35.461056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351122+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:22:22.351122+0000","last_peered":"2024-02-19T16:22:22.351122+0000","last_clean":"2024-02-19T16:22:22.351122+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:22:22.351122+0000","last_undegraded":"2024-02-19T16:22:22.351122+0000","last_fullsized":"2024-02-19T16:22:22.351122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349313+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:22:22.349313+0000","last_peered":"2024-02-19T16:22:22.349313+0000","last_clean":"2024-02-19T16:22:22.349313+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:22:22.349313+0000","last_undegraded":"2024-02-19T16:22:22.349313+0000","last_fullsized":"2024-02-19T16:22:22.349313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349176+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:22:22.349176+0000","last_peered":"2024-02-19T16:22:22.349176+0000","last_clean":"2024-02-19T16:22:22.349176+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:22:22.349176+0000","last_undegraded":"2024-02-19T16:22:22.349176+0000","last_fullsized":"2024-02-19T16:22:22.349176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350666+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:22:22.350666+0000","last_peered":"2024-02-19T16:22:22.350666+0000","last_clean":"2024-02-19T16:22:22.350666+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:22:22.350666+0000","last_undegraded":"2024-02-19T16:22:22.350666+0000","last_fullsized":"2024-02-19T16:22:22.350666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354508+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:22:22.354508+0000","last_peered":"2024-02-19T16:22:22.354508+0000","last_clean":"2024-02-19T16:22:22.354508+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:22:22.354508+0000","last_undegraded":"2024-02-19T16:22:22.354508+0000","last_fullsized":"2024-02-19T16:22:22.354508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353185+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:22:22.353185+0000","last_peered":"2024-02-19T16:22:22.353185+0000","last_clean":"2024-02-19T16:22:22.353185+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:22:22.353185+0000","last_undegraded":"2024-02-19T16:22:22.353185+0000","last_fullsized":"2024-02-19T16:22:22.353185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350730+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:22:22.350730+0000","last_peered":"2024-02-19T16:22:22.350730+0000","last_clean":"2024-02-19T16:22:22.350730+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:22:22.350730+0000","last_undegraded":"2024-02-19T16:22:22.350730+0000","last_fullsized":"2024-02-19T16:22:22.350730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449675+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:22:22.449675+0000","last_peered":"2024-02-19T16:22:22.449675+0000","last_clean":"2024-02-19T16:22:22.449675+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:22:22.449675+0000","last_undegraded":"2024-02-19T16:22:22.449675+0000","last_fullsized":"2024-02-19T16:22:22.449675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351689+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:22:22.351689+0000","last_peered":"2024-02-19T16:22:22.351689+0000","last_clean":"2024-02-19T16:22:22.351689+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:22:22.351689+0000","last_undegraded":"2024-02-19T16:22:22.351689+0000","last_fullsized":"2024-02-19T16:22:22.351689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354591+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:22:22.354591+0000","last_peered":"2024-02-19T16:22:22.354591+0000","last_clean":"2024-02-19T16:22:22.354591+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:22:22.354591+0000","last_undegraded":"2024-02-19T16:22:22.354591+0000","last_fullsized":"2024-02-19T16:22:22.354591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348967+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:22:22.348967+0000","last_peered":"2024-02-19T16:22:22.348967+0000","last_clean":"2024-02-19T16:22:22.348967+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:22:22.348967+0000","last_undegraded":"2024-02-19T16:22:22.348967+0000","last_fullsized":"2024-02-19T16:22:22.348967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:32.339521+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:22:32.339521+0000","last_peered":"2024-02-19T16:22:32.339521+0000","last_clean":"2024-02-19T16:22:32.339521+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:22:32.339521+0000","last_undegraded":"2024-02-19T16:22:32.339521+0000","last_fullsized":"2024-02-19T16:22:32.339521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349548+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:22:22.349548+0000","last_peered":"2024-02-19T16:22:22.349548+0000","last_clean":"2024-02-19T16:22:22.349548+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:22:22.349548+0000","last_undegraded":"2024-02-19T16:22:22.349548+0000","last_fullsized":"2024-02-19T16:22:22.349548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352447+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:22:22.352447+0000","last_peered":"2024-02-19T16:22:22.352447+0000","last_clean":"2024-02-19T16:22:22.352447+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:22:22.352447+0000","last_undegraded":"2024-02-19T16:22:22.352447+0000","last_fullsized":"2024-02-19T16:22:22.352447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349728+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:22:22.349728+0000","last_peered":"2024-02-19T16:22:22.349728+0000","last_clean":"2024-02-19T16:22:22.349728+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:22:22.349728+0000","last_undegraded":"2024-02-19T16:22:22.349728+0000","last_fullsized":"2024-02-19T16:22:22.349728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450488+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:22:22.450488+0000","last_peered":"2024-02-19T16:22:22.450488+0000","last_clean":"2024-02-19T16:22:22.450488+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:22:22.450488+0000","last_undegraded":"2024-02-19T16:22:22.450488+0000","last_fullsized":"2024-02-19T16:22:22.450488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349405+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:22:22.349405+0000","last_peered":"2024-02-19T16:22:22.349405+0000","last_clean":"2024-02-19T16:22:22.349405+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:22:22.349405+0000","last_undegraded":"2024-02-19T16:22:22.349405+0000","last_fullsized":"2024-02-19T16:22:22.349405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449451+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:22:22.449451+0000","last_peered":"2024-02-19T16:22:22.449451+0000","last_clean":"2024-02-19T16:22:22.449451+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:22:22.449451+0000","last_undegraded":"2024-02-19T16:22:22.449451+0000","last_fullsized":"2024-02-19T16:22:22.449451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351520+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:22:22.351520+0000","last_peered":"2024-02-19T16:22:22.351520+0000","last_clean":"2024-02-19T16:22:22.351520+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:22:22.351520+0000","last_undegraded":"2024-02-19T16:22:22.351520+0000","last_fullsized":"2024-02-19T16:22:22.351520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351365+0000","last_change":"2024-02-19T16:21:22.216182+0000","last_active":"2024-02-19T16:22:22.351365+0000","last_peered":"2024-02-19T16:22:22.351365+0000","last_clean":"2024-02-19T16:22:22.351365+0000","last_became_active":"2024-02-19T16:21:22.215705+0000","last_became_peered":"2024-02-19T16:21:22.215705+0000","last_unstale":"2024-02-19T16:22:22.351365+0000","last_undegraded":"2024-02-19T16:22:22.351365+0000","last_fullsized":"2024-02-19T16:22:22.351365+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:29:29.574422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349162+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:22:22.349162+0000","last_peered":"2024-02-19T16:22:22.349162+0000","last_clean":"2024-02-19T16:22:22.349162+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:22:22.349162+0000","last_undegraded":"2024-02-19T16:22:22.349162+0000","last_fullsized":"2024-02-19T16:22:22.349162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449565+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:22:22.449565+0000","last_peered":"2024-02-19T16:22:22.449565+0000","last_clean":"2024-02-19T16:22:22.449565+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:22:22.449565+0000","last_undegraded":"2024-02-19T16:22:22.449565+0000","last_fullsized":"2024-02-19T16:22:22.449565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349272+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:22:22.349272+0000","last_peered":"2024-02-19T16:22:22.349272+0000","last_clean":"2024-02-19T16:22:22.349272+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:22:22.349272+0000","last_undegraded":"2024-02-19T16:22:22.349272+0000","last_fullsized":"2024-02-19T16:22:22.349272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353013+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:22:22.353013+0000","last_peered":"2024-02-19T16:22:22.353013+0000","last_clean":"2024-02-19T16:22:22.353013+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:22:22.353013+0000","last_undegraded":"2024-02-19T16:22:22.353013+0000","last_fullsized":"2024-02-19T16:22:22.353013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349632+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:22:22.349632+0000","last_peered":"2024-02-19T16:22:22.349632+0000","last_clean":"2024-02-19T16:22:22.349632+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:22:22.349632+0000","last_undegraded":"2024-02-19T16:22:22.349632+0000","last_fullsized":"2024-02-19T16:22:22.349632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349434+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:22:22.349434+0000","last_peered":"2024-02-19T16:22:22.349434+0000","last_clean":"2024-02-19T16:22:22.349434+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:22:22.349434+0000","last_undegraded":"2024-02-19T16:22:22.349434+0000","last_fullsized":"2024-02-19T16:22:22.349434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449599+0000","last_change":"2024-02-19T16:21:21.894342+0000","last_active":"2024-02-19T16:22:22.449599+0000","last_peered":"2024-02-19T16:22:22.449599+0000","last_clean":"2024-02-19T16:22:22.449599+0000","last_became_active":"2024-02-19T16:21:21.885142+0000","last_became_peered":"2024-02-19T16:21:21.885142+0000","last_unstale":"2024-02-19T16:22:22.449599+0000","last_undegraded":"2024-02-19T16:22:22.449599+0000","last_fullsized":"2024-02-19T16:22:22.449599+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:19:30.846573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351023+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:22:22.351023+0000","last_peered":"2024-02-19T16:22:22.351023+0000","last_clean":"2024-02-19T16:22:22.351023+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:22:22.351023+0000","last_undegraded":"2024-02-19T16:22:22.351023+0000","last_fullsized":"2024-02-19T16:22:22.351023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449534+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:22:22.449534+0000","last_peered":"2024-02-19T16:22:22.449534+0000","last_clean":"2024-02-19T16:22:22.449534+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:22:22.449534+0000","last_undegraded":"2024-02-19T16:22:22.449534+0000","last_fullsized":"2024-02-19T16:22:22.449534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353273+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:22:22.353273+0000","last_peered":"2024-02-19T16:22:22.353273+0000","last_clean":"2024-02-19T16:22:22.353273+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:22:22.353273+0000","last_undegraded":"2024-02-19T16:22:22.353273+0000","last_fullsized":"2024-02-19T16:22:22.353273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351249+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:22:22.351249+0000","last_peered":"2024-02-19T16:22:22.351249+0000","last_clean":"2024-02-19T16:22:22.351249+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:22:22.351249+0000","last_undegraded":"2024-02-19T16:22:22.351249+0000","last_fullsized":"2024-02-19T16:22:22.351249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450177+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:22:22.450177+0000","last_peered":"2024-02-19T16:22:22.450177+0000","last_clean":"2024-02-19T16:22:22.450177+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:22:22.450177+0000","last_undegraded":"2024-02-19T16:22:22.450177+0000","last_fullsized":"2024-02-19T16:22:22.450177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354439+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:22:22.354439+0000","last_peered":"2024-02-19T16:22:22.354439+0000","last_clean":"2024-02-19T16:22:22.354439+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:22:22.354439+0000","last_undegraded":"2024-02-19T16:22:22.354439+0000","last_fullsized":"2024-02-19T16:22:22.354439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351048+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:22:22.351048+0000","last_peered":"2024-02-19T16:22:22.351048+0000","last_clean":"2024-02-19T16:22:22.351048+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:22:22.351048+0000","last_undegraded":"2024-02-19T16:22:22.351048+0000","last_fullsized":"2024-02-19T16:22:22.351048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353266+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:22:22.353266+0000","last_peered":"2024-02-19T16:22:22.353266+0000","last_clean":"2024-02-19T16:22:22.353266+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:22:22.353266+0000","last_undegraded":"2024-02-19T16:22:22.353266+0000","last_fullsized":"2024-02-19T16:22:22.353266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354966+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:22:22.354966+0000","last_peered":"2024-02-19T16:22:22.354966+0000","last_clean":"2024-02-19T16:22:22.354966+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:22:22.354966+0000","last_undegraded":"2024-02-19T16:22:22.354966+0000","last_fullsized":"2024-02-19T16:22:22.354966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351376+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:22:22.351376+0000","last_peered":"2024-02-19T16:22:22.351376+0000","last_clean":"2024-02-19T16:22:22.351376+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:22:22.351376+0000","last_undegraded":"2024-02-19T16:22:22.351376+0000","last_fullsized":"2024-02-19T16:22:22.351376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449671+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:22:22.449671+0000","last_peered":"2024-02-19T16:22:22.449671+0000","last_clean":"2024-02-19T16:22:22.449671+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:22:22.449671+0000","last_undegraded":"2024-02-19T16:22:22.449671+0000","last_fullsized":"2024-02-19T16:22:22.449671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352733+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:22:22.352733+0000","last_peered":"2024-02-19T16:22:22.352733+0000","last_clean":"2024-02-19T16:22:22.352733+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:22:22.352733+0000","last_undegraded":"2024-02-19T16:22:22.352733+0000","last_fullsized":"2024-02-19T16:22:22.352733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352775+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:22:22.352775+0000","last_peered":"2024-02-19T16:22:22.352775+0000","last_clean":"2024-02-19T16:22:22.352775+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:22:22.352775+0000","last_undegraded":"2024-02-19T16:22:22.352775+0000","last_fullsized":"2024-02-19T16:22:22.352775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450420+0000","last_change":"2024-02-19T16:21:21.884814+0000","last_active":"2024-02-19T16:22:22.450420+0000","last_peered":"2024-02-19T16:22:22.450420+0000","last_clean":"2024-02-19T16:22:22.450420+0000","last_became_active":"2024-02-19T16:21:21.884410+0000","last_became_peered":"2024-02-19T16:21:21.884410+0000","last_unstale":"2024-02-19T16:22:22.450420+0000","last_undegraded":"2024-02-19T16:22:22.450420+0000","last_fullsized":"2024-02-19T16:22:22.450420+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:39:44.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354725+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:22:22.354725+0000","last_peered":"2024-02-19T16:22:22.354725+0000","last_clean":"2024-02-19T16:22:22.354725+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:22:22.354725+0000","last_undegraded":"2024-02-19T16:22:22.354725+0000","last_fullsized":"2024-02-19T16:22:22.354725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449925+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:22:22.449925+0000","last_peered":"2024-02-19T16:22:22.449925+0000","last_clean":"2024-02-19T16:22:22.449925+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:22:22.449925+0000","last_undegraded":"2024-02-19T16:22:22.449925+0000","last_fullsized":"2024-02-19T16:22:22.449925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354587+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:22:22.354587+0000","last_peered":"2024-02-19T16:22:22.354587+0000","last_clean":"2024-02-19T16:22:22.354587+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:22:22.354587+0000","last_undegraded":"2024-02-19T16:22:22.354587+0000","last_fullsized":"2024-02-19T16:22:22.354587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349718+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:22:22.349718+0000","last_peered":"2024-02-19T16:22:22.349718+0000","last_clean":"2024-02-19T16:22:22.349718+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:22:22.349718+0000","last_undegraded":"2024-02-19T16:22:22.349718+0000","last_fullsized":"2024-02-19T16:22:22.349718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353764+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:22:22.353764+0000","last_peered":"2024-02-19T16:22:22.353764+0000","last_clean":"2024-02-19T16:22:22.353764+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:22:22.353764+0000","last_undegraded":"2024-02-19T16:22:22.353764+0000","last_fullsized":"2024-02-19T16:22:22.353764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349237+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:22:22.349237+0000","last_peered":"2024-02-19T16:22:22.349237+0000","last_clean":"2024-02-19T16:22:22.349237+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:22:22.349237+0000","last_undegraded":"2024-02-19T16:22:22.349237+0000","last_fullsized":"2024-02-19T16:22:22.349237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449690+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:22:22.449690+0000","last_peered":"2024-02-19T16:22:22.449690+0000","last_clean":"2024-02-19T16:22:22.449690+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:22:22.449690+0000","last_undegraded":"2024-02-19T16:22:22.449690+0000","last_fullsized":"2024-02-19T16:22:22.449690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353873+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:22:22.353873+0000","last_peered":"2024-02-19T16:22:22.353873+0000","last_clean":"2024-02-19T16:22:22.353873+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:22:22.353873+0000","last_undegraded":"2024-02-19T16:22:22.353873+0000","last_fullsized":"2024-02-19T16:22:22.353873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449777+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:22:22.449777+0000","last_peered":"2024-02-19T16:22:22.449777+0000","last_clean":"2024-02-19T16:22:22.449777+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:22:22.449777+0000","last_undegraded":"2024-02-19T16:22:22.449777+0000","last_fullsized":"2024-02-19T16:22:22.449777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":732,"num_read_kb":442,"num_write":173,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":213861,"data_compressed":5154,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":532611,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.56999999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.498,"15min":0.54200000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.53500000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.47999999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40200000000000002,"15min":0.42299999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.53300000000000003,"15min":0.56699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60999999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48199999999999998,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.51800000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.61699999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44700000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.61699999999999999},"min":{"1min":0.48099999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69999999999999996,"5min":0.998,"15min":0.998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61099999999999999,"15min":0.64400000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67800000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.66500000000000004},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63700000000000001}]},{"osd":9,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65600000000000003,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71999999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66400000000000003,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.78400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68200000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.755,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.65500000000000003,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.63800000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60499999999999998}]}]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1188316,"data_compressed":67184,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.32900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63900000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.495},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.55700000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.377}]},{"osd":1,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.40200000000000002,"15min":0.432},"min":{"1min":0.23499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.34000000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.45700000000000002,"15min":0.49399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.377,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.499,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.48399999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.50700000000000001,"15min":0.54700000000000004},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.438,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35499999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56299999999999994,"15min":0.60099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61799999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.67400000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.50600000000000001,"15min":0.55000000000000004},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.46200000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.28299999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57599999999999996,"15min":0.61799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.56999999999999995,"15min":0.61899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.50800000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38700000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.65400000000000003,"5min":0.999,"15min":0.999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.65900000000000003,"15min":0.69499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71999999999999997,"5min":1.028,"15min":1.028},"last":0.50800000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.67400000000000004,"15min":0.73199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60299999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.66200000000000003,"15min":0.72399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.55800000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.48499999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.69199999999999995,"15min":0.73599999999999999},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.63300000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.68899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:21:37 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.56399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54100000000000004}]}]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":536361,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629}]},{"osd":1,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.626,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.55600000000000005,"15min":0.58099999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.627},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66000000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72699999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.29299999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.436},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.66400000000000003,"15min":0.69399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.749,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82099999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.73299999999999998,"15min":0.78000000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76000000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57199999999999995}]},{"osd":8,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67700000000000005,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":1.169,"15min":1.169},"last":0.70899999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.75,"15min":0.78400000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.89500000000000002,"5min":1.278,"15min":1.278},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71999999999999997,"15min":0.76200000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.78000000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64400000000000002}]},{"osd":11,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":0.78700000000000003,"15min":0.68200000000000005},"min":{"1min":0.53600000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":13.175000000000001,"5min":13.175000000000001,"15min":13.175000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54000000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532185,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.44400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48499999999999999,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.52500000000000002,"15min":0.56899999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.505,"5min":0.875,"15min":0.875},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.313,"5min":0.31,"15min":0.31},"max":{"1min":0.56799999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35099999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40200000000000002,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41299999999999998,"15min":0.42399999999999999},"min":{"1min":0.27300000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.53100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.317}]},{"osd":3,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.64600000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.497},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.54800000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56799999999999995,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.58299999999999996},"min":{"1min":0.29599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76100000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70899999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.68200000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77600000000000002,"5min":1.393,"15min":1.393},"last":0.46800000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.52200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.73599999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68600000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.72299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.441},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.74299999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63300000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.78500000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.77300000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1127653,"data_compressed":65466,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.379,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.38200000000000001,"5min":0.75,"15min":0.75},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.49299999999999999,"15min":0.53700000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.41899999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40699999999999997}]},{"osd":3,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.64100000000000001,"5min":0.87,"15min":0.87},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.46600000000000003,"15min":0.48099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.46500000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432}]},{"osd":5,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.55500000000000005,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.504,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.61299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45400000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64100000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.61799999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59199999999999997}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.60199999999999998,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.57199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.58499999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.625}]},{"osd":9,"last update":"Mon Feb 19 16:21:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67400000000000004,"5min":0.997,"15min":0.997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70099999999999996,"15min":0.71499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.70599999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54600000000000004}]},{"osd":10,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.67200000000000004,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.64200000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.64000000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48999999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.65200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.66800000000000004,"5min":1.024,"15min":1.024},"last":0.60499999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":527441,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80600000000000005}]},{"osd":2,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82899999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.48099999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.56999999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94399999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76000000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.56799999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":1.014,"15min":1.014},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91300000000000003,"5min":1.113,"15min":1.113},"last":0.84099999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86299999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.50700000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.872},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.60399999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0089999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81999999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.878,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0449999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":589375,"data_compressed":31426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.68700000000000006,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.2210000000000001,"5min":14.067,"15min":14.067},"last":0.52800000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.47299999999999998,"15min":0.51400000000000001},"min":{"1min":0.26700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.46400000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.502,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.41399999999999998,"15min":0.376},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59999999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44400000000000001,"15min":0.40500000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.313}]},{"osd":5,"last update":"Mon Feb 19 16:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.371},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.44600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70899999999999996,"5min":0.879,"15min":0.879},"last":0.40400000000000003}]},{"osd":6,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.625,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71399999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51500000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68100000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.503}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.71799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42199999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.82099999999999995,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":14.105,"5min":14.105,"15min":14.105},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71099999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48199999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.63400000000000001,"15min":0.69599999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.63800000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71199999999999997,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.67200000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56100000000000005}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.68999999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.70599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.62}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528829,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.54400000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67100000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71499999999999997}]},{"osd":2,"last update":"Mon Feb 19 16:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58899999999999997,"5min":1.899,"15min":1.899},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.38,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Feb 19 16:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.62,"5min":0.63300000000000001,"15min":0.60199999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80700000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.624}]},{"osd":7,"last update":"Mon Feb 19 16:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84999999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.51300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86499999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.45900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.98499999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.90500000000000003}]},{"osd":11,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":599317,"data_compressed":31426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59199999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61099999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.39000000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.47299999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70599999999999996,"5min":1.758,"15min":1.758},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59299999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.377},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61799999999999999,"5min":2.012,"15min":2.012},"last":0.439}]},{"osd":5,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.64100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.47799999999999998},"min":{"1min":0.317,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58299999999999996,"5min":1.026,"15min":1.026},"last":0.496}]},{"osd":6,"last update":"Mon Feb 19 16:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.621},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.622},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61699999999999999,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69799999999999995,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.52300000000000002}]},{"osd":8,"last update":"Mon Feb 19 16:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53900000000000003,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.63800000000000001},"min":{"1min":0.503,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.626,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53500000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69399999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.627},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.755,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.60499999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.70899999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.67600000000000005,"5min":1.282,"15min":1.282},"last":0.56200000000000006}]},{"osd":11,"last update":"Mon Feb 19 16:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.72599999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59399999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":527635,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.36499999999999999},"min":{"1min":0.214,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.379},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.378}]},{"osd":1,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60699999999999998,"5min":0.81399999999999995,"15min":0.90800000000000003},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56799999999999995,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64000000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82399999999999995,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77800000000000002,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.018},"last":0.66700000000000004}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.05,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57799999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.999,"5min":1.03,"15min":1.03},"last":0.71799999999999997}]},{"osd":11,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":532623,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.26300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.34899999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.373,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.311}]},{"osd":2,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.51500000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.433}]},{"osd":3,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":1.03,"15min":1.03},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.60499999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.47399999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.55000000000000004,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.51100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47399999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.502}]},{"osd":7,"last update":"Mon Feb 19 16:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.53400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55200000000000005}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.58299999999999996,"5min":1.073,"15min":1.073},"last":0.52800000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":0.64100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.70299999999999996,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.65300000000000002,"5min":1.044,"15min":1.044},"last":0.61199999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.67000000000000004,"15min":0.70799999999999996},"min":{"1min":0.442,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.64100000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.70699999999999996,"15min":0.75},"min":{"1min":0.45400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66500000000000004,"5min":1.056,"15min":1.056},"last":0.51500000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70599999999999996,"5min":1.133,"15min":1.133},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.69899999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.621}]}]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1123291,"data_compressed":65466,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.64300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67400000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.48299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53200000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.372,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38600000000000001,"15min":0.375},"min":{"1min":0.25,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68700000000000006,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.871,"15min":0.871},"last":0.436},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.505,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65600000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.65100000000000002,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70799999999999996,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.42199999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:22:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66600000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64200000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46800000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.318,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72099999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.57099999999999995,"15min":0.623},"min":{"1min":0.38200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53300000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66300000000000003,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.74199999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55600000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68700000000000006,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.68200000000000005,"15min":0.72199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.67800000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62332,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71966,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70597,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:22:41.685 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-19T16:22:41.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1679133695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:41.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/675690730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2642237061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:41.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 -- 172.21.15.8:0/1825250897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5601030e0 msgr2=0x7fc560109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 --2- 172.21.15.8:0/1825250897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5601030e0 0x7fc560109970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc558009ca0 tx=0x7fc55802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 -- 172.21.15.8:0/1825250897 shutdown_connections 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 --2- 172.21.15.8:0/1825250897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5601030e0 0x7fc560109970 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 --2- 172.21.15.8:0/1825250897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc560102720 0x7fc560102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 --2- 172.21.15.8:0/1825250897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc560101530 0x7fc560101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.820+0000 7fc56804e640 1 -- 172.21.15.8:0/1825250897 >> 172.21.15.8:0/1825250897 conn(0x7fc5600fcaa0 msgr2=0x7fc5600feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 -- 172.21.15.8:0/1825250897 shutdown_connections 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 -- 172.21.15.8:0/1825250897 wait complete. 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 Processor -- start 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 -- start start 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 0x7fc560197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc560102720 0x7fc560197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5601030e0 0x7fc56019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56019f050 con 0x7fc560101530 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56019f1c0 con 0x7fc560102720 2024-02-19T16:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc56804e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56019f380 con 0x7fc5601030e0 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc565dea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 0x7fc560197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc565dea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 0x7fc560197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49576/0 (socket says 172.21.15.8:49576) 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc565dea640 1 -- 172.21.15.8:0/1199166009 learned_addr learned my addr 172.21.15.8:0/1199166009 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc5665eb640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5601030e0 0x7fc56019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.821+0000 7fc5655e9640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc560102720 0x7fc560197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc565dea640 1 -- 172.21.15.8:0/1199166009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5601030e0 msgr2=0x7fc56019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc565dea640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5601030e0 0x7fc56019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc565dea640 1 -- 172.21.15.8:0/1199166009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc560102720 msgr2=0x7fc560197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc565dea640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc560102720 0x7fc560197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc565dea640 1 -- 172.21.15.8:0/1199166009 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc558009900 con 0x7fc560101530 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc5665eb640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5601030e0 0x7fc56019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc565dea640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 0x7fc560197560 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e9c0 tx=0x7fc55c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c009700 con 0x7fc560101530 2024-02-19T16:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc55c00cd80 con 0x7fc560101530 2024-02-19T16:22:41.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56019f660 con 0x7fc560101530 2024-02-19T16:22:41.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c010430 con 0x7fc560101530 2024-02-19T16:22:41.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.822+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56019fb70 con 0x7fc560101530 2024-02-19T16:22:41.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.823+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5601087f0 con 0x7fc560101530 2024-02-19T16:22:41.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.824+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fc55c003680 con 0x7fc560101530 2024-02-19T16:22:41.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.825+0000 7fc556ffd640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc54c0a2150 0x7fc54c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:41.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.825+0000 7fc5655e9640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc54c0a2150 0x7fc54c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:41.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.825+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fc55c014070 con 0x7fc560101530 2024-02-19T16:22:41.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.825+0000 7fc5655e9640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc54c0a2150 0x7fc54c0a4610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc560198a70 tx=0x7fc550005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1679133695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/675690730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:41.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2642237061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:41.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:41.829+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fc55c097060 con 0x7fc560101530 2024-02-19T16:22:42.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.010+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc56002d070 con 0x7fc54c0a2150 2024-02-19T16:22:42.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.023+0000 7fc556ffd640 1 -- 172.21.15.8:0/1199166009 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428108 (secure 0 0 0) 0x7fc56002d070 con 0x7fc54c0a2150 2024-02-19T16:22:42.028 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:42.031 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:22:42.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc54c0a2150 msgr2=0x7fc54c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc54c0a2150 0x7fc54c0a4610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc560198a70 tx=0x7fc550005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:42.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 msgr2=0x7fc560197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 0x7fc560197560 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e9c0 tx=0x7fc55c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 shutdown_connections 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc54c0a2150 0x7fc54c0a4610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5601030e0 0x7fc56019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc560102720 0x7fc560197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 --2- 172.21.15.8:0/1199166009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc560101530 0x7fc560197560 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 >> 172.21.15.8:0/1199166009 conn(0x7fc5600fcaa0 msgr2=0x7fc5601059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 shutdown_connections 2024-02-19T16:22:42.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.026+0000 7fc56804e640 1 -- 172.21.15.8:0/1199166009 wait complete. 2024-02-19T16:22:42.038 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-02-19T16:22:40.748176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":868,"num_read_kb":561,"num_write":356,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378548,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563276,"statfs":{"total":1151940427776,"available":1151552794624,"internally_reserved":0,"allocated":12992512,"data_stored":8345637,"data_compressed":468924,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003373"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450074+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:22:22.450074+0000","last_peered":"2024-02-19T16:22:22.450074+0000","last_clean":"2024-02-19T16:22:22.450074+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:22:22.450074+0000","last_undegraded":"2024-02-19T16:22:22.450074+0000","last_fullsized":"2024-02-19T16:22:22.450074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450435+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:22:22.450435+0000","last_peered":"2024-02-19T16:22:22.450435+0000","last_clean":"2024-02-19T16:22:22.450435+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:22:22.450435+0000","last_undegraded":"2024-02-19T16:22:22.450435+0000","last_fullsized":"2024-02-19T16:22:22.450435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450286+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:22:22.450286+0000","last_peered":"2024-02-19T16:22:22.450286+0000","last_clean":"2024-02-19T16:22:22.450286+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:22:22.450286+0000","last_undegraded":"2024-02-19T16:22:22.450286+0000","last_fullsized":"2024-02-19T16:22:22.450286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349393+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:22:22.349393+0000","last_peered":"2024-02-19T16:22:22.349393+0000","last_clean":"2024-02-19T16:22:22.349393+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:22:22.349393+0000","last_undegraded":"2024-02-19T16:22:22.349393+0000","last_fullsized":"2024-02-19T16:22:22.349393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351320+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:22:22.351320+0000","last_peered":"2024-02-19T16:22:22.351320+0000","last_clean":"2024-02-19T16:22:22.351320+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:22:22.351320+0000","last_undegraded":"2024-02-19T16:22:22.351320+0000","last_fullsized":"2024-02-19T16:22:22.351320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428710+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:22:22.428710+0000","last_peered":"2024-02-19T16:22:22.428710+0000","last_clean":"2024-02-19T16:22:22.428710+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:22:22.428710+0000","last_undegraded":"2024-02-19T16:22:22.428710+0000","last_fullsized":"2024-02-19T16:22:22.428710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350749+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:22:22.350749+0000","last_peered":"2024-02-19T16:22:22.350749+0000","last_clean":"2024-02-19T16:22:22.350749+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:22:22.350749+0000","last_undegraded":"2024-02-19T16:22:22.350749+0000","last_fullsized":"2024-02-19T16:22:22.350749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348892+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:22:22.348892+0000","last_peered":"2024-02-19T16:22:22.348892+0000","last_clean":"2024-02-19T16:22:22.348892+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:22:22.348892+0000","last_undegraded":"2024-02-19T16:22:22.348892+0000","last_fullsized":"2024-02-19T16:22:22.348892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449821+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:22:22.449821+0000","last_peered":"2024-02-19T16:22:22.449821+0000","last_clean":"2024-02-19T16:22:22.449821+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:22:22.449821+0000","last_undegraded":"2024-02-19T16:22:22.449821+0000","last_fullsized":"2024-02-19T16:22:22.449821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428995+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:22:22.428995+0000","last_peered":"2024-02-19T16:22:22.428995+0000","last_clean":"2024-02-19T16:22:22.428995+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:22:22.428995+0000","last_undegraded":"2024-02-19T16:22:22.428995+0000","last_fullsized":"2024-02-19T16:22:22.428995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351456+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:22:22.351456+0000","last_peered":"2024-02-19T16:22:22.351456+0000","last_clean":"2024-02-19T16:22:22.351456+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:22:22.351456+0000","last_undegraded":"2024-02-19T16:22:22.351456+0000","last_fullsized":"2024-02-19T16:22:22.351456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349352+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:22:22.349352+0000","last_peered":"2024-02-19T16:22:22.349352+0000","last_clean":"2024-02-19T16:22:22.349352+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:22:22.349352+0000","last_undegraded":"2024-02-19T16:22:22.349352+0000","last_fullsized":"2024-02-19T16:22:22.349352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354017+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:22:22.354017+0000","last_peered":"2024-02-19T16:22:22.354017+0000","last_clean":"2024-02-19T16:22:22.354017+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:22:22.354017+0000","last_undegraded":"2024-02-19T16:22:22.354017+0000","last_fullsized":"2024-02-19T16:22:22.354017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450028+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:22:22.450028+0000","last_peered":"2024-02-19T16:22:22.450028+0000","last_clean":"2024-02-19T16:22:22.450028+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:22:22.450028+0000","last_undegraded":"2024-02-19T16:22:22.450028+0000","last_fullsized":"2024-02-19T16:22:22.450028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450061+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:22:22.450061+0000","last_peered":"2024-02-19T16:22:22.450061+0000","last_clean":"2024-02-19T16:22:22.450061+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:22:22.450061+0000","last_undegraded":"2024-02-19T16:22:22.450061+0000","last_fullsized":"2024-02-19T16:22:22.450061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449363+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:22:22.449363+0000","last_peered":"2024-02-19T16:22:22.449363+0000","last_clean":"2024-02-19T16:22:22.449363+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:22:22.449363+0000","last_undegraded":"2024-02-19T16:22:22.449363+0000","last_fullsized":"2024-02-19T16:22:22.449363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449854+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:22:22.449854+0000","last_peered":"2024-02-19T16:22:22.449854+0000","last_clean":"2024-02-19T16:22:22.449854+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:22:22.449854+0000","last_undegraded":"2024-02-19T16:22:22.449854+0000","last_fullsized":"2024-02-19T16:22:22.449854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351299+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:22:22.351299+0000","last_peered":"2024-02-19T16:22:22.351299+0000","last_clean":"2024-02-19T16:22:22.351299+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:22:22.351299+0000","last_undegraded":"2024-02-19T16:22:22.351299+0000","last_fullsized":"2024-02-19T16:22:22.351299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353320+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:22:22.353320+0000","last_peered":"2024-02-19T16:22:22.353320+0000","last_clean":"2024-02-19T16:22:22.353320+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:22:22.353320+0000","last_undegraded":"2024-02-19T16:22:22.353320+0000","last_fullsized":"2024-02-19T16:22:22.353320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352554+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:22:22.352554+0000","last_peered":"2024-02-19T16:22:22.352554+0000","last_clean":"2024-02-19T16:22:22.352554+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:22:22.352554+0000","last_undegraded":"2024-02-19T16:22:22.352554+0000","last_fullsized":"2024-02-19T16:22:22.352554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351551+0000","last_change":"2024-02-19T16:21:21.892098+0000","last_active":"2024-02-19T16:22:22.351551+0000","last_peered":"2024-02-19T16:22:22.351551+0000","last_clean":"2024-02-19T16:22:22.351551+0000","last_became_active":"2024-02-19T16:21:21.884753+0000","last_became_peered":"2024-02-19T16:21:21.884753+0000","last_unstale":"2024-02-19T16:22:22.351551+0000","last_undegraded":"2024-02-19T16:22:22.351551+0000","last_fullsized":"2024-02-19T16:22:22.351551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:24.360019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355648+0000","last_change":"2024-02-19T16:22:22.355648+0000","last_active":"2024-02-19T16:22:22.355648+0000","last_peered":"2024-02-19T16:22:22.355648+0000","last_clean":"2024-02-19T16:22:22.355648+0000","last_became_active":"2024-02-19T16:22:22.355213+0000","last_became_peered":"2024-02-19T16:22:22.355213+0000","last_unstale":"2024-02-19T16:22:22.355648+0000","last_undegraded":"2024-02-19T16:22:22.355648+0000","last_fullsized":"2024-02-19T16:22:22.355648+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:49:24.360051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348894+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:22:22.348894+0000","last_peered":"2024-02-19T16:22:22.348894+0000","last_clean":"2024-02-19T16:22:22.348894+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:22:22.348894+0000","last_undegraded":"2024-02-19T16:22:22.348894+0000","last_fullsized":"2024-02-19T16:22:22.348894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350052+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:22:22.350052+0000","last_peered":"2024-02-19T16:22:22.350052+0000","last_clean":"2024-02-19T16:22:22.350052+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:22:22.350052+0000","last_undegraded":"2024-02-19T16:22:22.350052+0000","last_fullsized":"2024-02-19T16:22:22.350052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450474+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:22:22.450474+0000","last_peered":"2024-02-19T16:22:22.450474+0000","last_clean":"2024-02-19T16:22:22.450474+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:22:22.450474+0000","last_undegraded":"2024-02-19T16:22:22.450474+0000","last_fullsized":"2024-02-19T16:22:22.450474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349058+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:22:22.349058+0000","last_peered":"2024-02-19T16:22:22.349058+0000","last_clean":"2024-02-19T16:22:22.349058+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:22:22.349058+0000","last_undegraded":"2024-02-19T16:22:22.349058+0000","last_fullsized":"2024-02-19T16:22:22.349058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351817+0000","last_change":"2024-02-19T16:21:21.889760+0000","last_active":"2024-02-19T16:22:22.351817+0000","last_peered":"2024-02-19T16:22:22.351817+0000","last_clean":"2024-02-19T16:22:22.351817+0000","last_became_active":"2024-02-19T16:21:21.884230+0000","last_became_peered":"2024-02-19T16:21:21.884230+0000","last_unstale":"2024-02-19T16:22:22.351817+0000","last_undegraded":"2024-02-19T16:22:22.351817+0000","last_fullsized":"2024-02-19T16:22:22.351817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:48:45.580976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449511+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:22:22.449511+0000","last_peered":"2024-02-19T16:22:22.449511+0000","last_clean":"2024-02-19T16:22:22.449511+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:22:22.449511+0000","last_undegraded":"2024-02-19T16:22:22.449511+0000","last_fullsized":"2024-02-19T16:22:22.449511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349008+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:22:22.349008+0000","last_peered":"2024-02-19T16:22:22.349008+0000","last_clean":"2024-02-19T16:22:22.349008+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:22:22.349008+0000","last_undegraded":"2024-02-19T16:22:22.349008+0000","last_fullsized":"2024-02-19T16:22:22.349008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351467+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:22:22.351467+0000","last_peered":"2024-02-19T16:22:22.351467+0000","last_clean":"2024-02-19T16:22:22.351467+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:22:22.351467+0000","last_undegraded":"2024-02-19T16:22:22.351467+0000","last_fullsized":"2024-02-19T16:22:22.351467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354355+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:22:22.354355+0000","last_peered":"2024-02-19T16:22:22.354355+0000","last_clean":"2024-02-19T16:22:22.354355+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:22:22.354355+0000","last_undegraded":"2024-02-19T16:22:22.354355+0000","last_fullsized":"2024-02-19T16:22:22.354355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353349+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:22:22.353349+0000","last_peered":"2024-02-19T16:22:22.353349+0000","last_clean":"2024-02-19T16:22:22.353349+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:22:22.353349+0000","last_undegraded":"2024-02-19T16:22:22.353349+0000","last_fullsized":"2024-02-19T16:22:22.353349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":159,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:38.445988+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:22:38.445988+0000","last_peered":"2024-02-19T16:22:38.445988+0000","last_clean":"2024-02-19T16:22:38.445988+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:22:38.445988+0000","last_undegraded":"2024-02-19T16:22:38.445988+0000","last_fullsized":"2024-02-19T16:22:38.445988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351396+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:22:22.351396+0000","last_peered":"2024-02-19T16:22:22.351396+0000","last_clean":"2024-02-19T16:22:22.351396+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:22:22.351396+0000","last_undegraded":"2024-02-19T16:22:22.351396+0000","last_fullsized":"2024-02-19T16:22:22.351396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352924+0000","last_change":"2024-02-19T16:21:21.885536+0000","last_active":"2024-02-19T16:22:22.352924+0000","last_peered":"2024-02-19T16:22:22.352924+0000","last_clean":"2024-02-19T16:22:22.352924+0000","last_became_active":"2024-02-19T16:21:21.885125+0000","last_became_peered":"2024-02-19T16:21:21.885125+0000","last_unstale":"2024-02-19T16:22:22.352924+0000","last_undegraded":"2024-02-19T16:22:22.352924+0000","last_fullsized":"2024-02-19T16:22:22.352924+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:23:23.180070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428691+0000","last_change":"2024-02-19T16:21:22.344872+0000","last_active":"2024-02-19T16:22:22.428691+0000","last_peered":"2024-02-19T16:22:22.428691+0000","last_clean":"2024-02-19T16:22:22.428691+0000","last_became_active":"2024-02-19T16:21:22.342317+0000","last_became_peered":"2024-02-19T16:21:22.342317+0000","last_unstale":"2024-02-19T16:22:22.428691+0000","last_undegraded":"2024-02-19T16:22:22.428691+0000","last_fullsized":"2024-02-19T16:22:22.428691+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:53:45.080091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354273+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:22:22.354273+0000","last_peered":"2024-02-19T16:22:22.354273+0000","last_clean":"2024-02-19T16:22:22.354273+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:22:22.354273+0000","last_undegraded":"2024-02-19T16:22:22.354273+0000","last_fullsized":"2024-02-19T16:22:22.354273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449894+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:22:22.449894+0000","last_peered":"2024-02-19T16:22:22.449894+0000","last_clean":"2024-02-19T16:22:22.449894+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:22:22.449894+0000","last_undegraded":"2024-02-19T16:22:22.449894+0000","last_fullsized":"2024-02-19T16:22:22.449894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353391+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:22:22.353391+0000","last_peered":"2024-02-19T16:22:22.353391+0000","last_clean":"2024-02-19T16:22:22.353391+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:22:22.353391+0000","last_undegraded":"2024-02-19T16:22:22.353391+0000","last_fullsized":"2024-02-19T16:22:22.353391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349154+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:22:22.349154+0000","last_peered":"2024-02-19T16:22:22.349154+0000","last_clean":"2024-02-19T16:22:22.349154+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:22:22.349154+0000","last_undegraded":"2024-02-19T16:22:22.349154+0000","last_fullsized":"2024-02-19T16:22:22.349154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450308+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:22:22.450308+0000","last_peered":"2024-02-19T16:22:22.450308+0000","last_clean":"2024-02-19T16:22:22.450308+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:22:22.450308+0000","last_undegraded":"2024-02-19T16:22:22.450308+0000","last_fullsized":"2024-02-19T16:22:22.450308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353726+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:22:22.353726+0000","last_peered":"2024-02-19T16:22:22.353726+0000","last_clean":"2024-02-19T16:22:22.353726+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:22:22.353726+0000","last_undegraded":"2024-02-19T16:22:22.353726+0000","last_fullsized":"2024-02-19T16:22:22.353726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350686+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:22:22.350686+0000","last_peered":"2024-02-19T16:22:22.350686+0000","last_clean":"2024-02-19T16:22:22.350686+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:22:22.350686+0000","last_undegraded":"2024-02-19T16:22:22.350686+0000","last_fullsized":"2024-02-19T16:22:22.350686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349021+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:22:22.349021+0000","last_peered":"2024-02-19T16:22:22.349021+0000","last_clean":"2024-02-19T16:22:22.349021+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:22:22.349021+0000","last_undegraded":"2024-02-19T16:22:22.349021+0000","last_fullsized":"2024-02-19T16:22:22.349021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:37.444275+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:22:37.444275+0000","last_peered":"2024-02-19T16:22:37.444275+0000","last_clean":"2024-02-19T16:22:37.444275+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:22:37.444275+0000","last_undegraded":"2024-02-19T16:22:37.444275+0000","last_fullsized":"2024-02-19T16:22:37.444275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449813+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:22:22.449813+0000","last_peered":"2024-02-19T16:22:22.449813+0000","last_clean":"2024-02-19T16:22:22.449813+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:22:22.449813+0000","last_undegraded":"2024-02-19T16:22:22.449813+0000","last_fullsized":"2024-02-19T16:22:22.449813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351146+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:22:22.351146+0000","last_peered":"2024-02-19T16:22:22.351146+0000","last_clean":"2024-02-19T16:22:22.351146+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:22:22.351146+0000","last_undegraded":"2024-02-19T16:22:22.351146+0000","last_fullsized":"2024-02-19T16:22:22.351146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351218+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:22:22.351218+0000","last_peered":"2024-02-19T16:22:22.351218+0000","last_clean":"2024-02-19T16:22:22.351218+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:22:22.351218+0000","last_undegraded":"2024-02-19T16:22:22.351218+0000","last_fullsized":"2024-02-19T16:22:22.351218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450461+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:22:22.450461+0000","last_peered":"2024-02-19T16:22:22.450461+0000","last_clean":"2024-02-19T16:22:22.450461+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:22:22.450461+0000","last_undegraded":"2024-02-19T16:22:22.450461+0000","last_fullsized":"2024-02-19T16:22:22.450461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349162+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:22:22.349162+0000","last_peered":"2024-02-19T16:22:22.349162+0000","last_clean":"2024-02-19T16:22:22.349162+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:22:22.349162+0000","last_undegraded":"2024-02-19T16:22:22.349162+0000","last_fullsized":"2024-02-19T16:22:22.349162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354277+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:22:22.354277+0000","last_peered":"2024-02-19T16:22:22.354277+0000","last_clean":"2024-02-19T16:22:22.354277+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:22:22.354277+0000","last_undegraded":"2024-02-19T16:22:22.354277+0000","last_fullsized":"2024-02-19T16:22:22.354277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449706+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:22:22.449706+0000","last_peered":"2024-02-19T16:22:22.449706+0000","last_clean":"2024-02-19T16:22:22.449706+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:22:22.449706+0000","last_undegraded":"2024-02-19T16:22:22.449706+0000","last_fullsized":"2024-02-19T16:22:22.449706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428529+0000","last_change":"2024-02-19T16:21:21.885002+0000","last_active":"2024-02-19T16:22:22.428529+0000","last_peered":"2024-02-19T16:22:22.428529+0000","last_clean":"2024-02-19T16:22:22.428529+0000","last_became_active":"2024-02-19T16:21:21.884557+0000","last_became_peered":"2024-02-19T16:21:21.884557+0000","last_unstale":"2024-02-19T16:22:22.428529+0000","last_undegraded":"2024-02-19T16:22:22.428529+0000","last_fullsized":"2024-02-19T16:22:22.428529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:08:31.859577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'37","reported_seq":315,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:38.445992+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:22:38.445992+0000","last_peered":"2024-02-19T16:22:38.445992+0000","last_clean":"2024-02-19T16:22:38.445992+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:22:38.445992+0000","last_undegraded":"2024-02-19T16:22:38.445992+0000","last_fullsized":"2024-02-19T16:22:38.445992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":190,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351523+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:22:22.351523+0000","last_peered":"2024-02-19T16:22:22.351523+0000","last_clean":"2024-02-19T16:22:22.351523+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:22:22.351523+0000","last_undegraded":"2024-02-19T16:22:22.351523+0000","last_fullsized":"2024-02-19T16:22:22.351523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354566+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:22:22.354566+0000","last_peered":"2024-02-19T16:22:22.354566+0000","last_clean":"2024-02-19T16:22:22.354566+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:22:22.354566+0000","last_undegraded":"2024-02-19T16:22:22.354566+0000","last_fullsized":"2024-02-19T16:22:22.354566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352980+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:22:22.352980+0000","last_peered":"2024-02-19T16:22:22.352980+0000","last_clean":"2024-02-19T16:22:22.352980+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:22:22.352980+0000","last_undegraded":"2024-02-19T16:22:22.352980+0000","last_fullsized":"2024-02-19T16:22:22.352980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354094+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:22:22.354094+0000","last_peered":"2024-02-19T16:22:22.354094+0000","last_clean":"2024-02-19T16:22:22.354094+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:22:22.354094+0000","last_undegraded":"2024-02-19T16:22:22.354094+0000","last_fullsized":"2024-02-19T16:22:22.354094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449654+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:22:22.449654+0000","last_peered":"2024-02-19T16:22:22.449654+0000","last_clean":"2024-02-19T16:22:22.449654+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:22:22.449654+0000","last_undegraded":"2024-02-19T16:22:22.449654+0000","last_fullsized":"2024-02-19T16:22:22.449654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354252+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:22:22.354252+0000","last_peered":"2024-02-19T16:22:22.354252+0000","last_clean":"2024-02-19T16:22:22.354252+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:22:22.354252+0000","last_undegraded":"2024-02-19T16:22:22.354252+0000","last_fullsized":"2024-02-19T16:22:22.354252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355544+0000","last_change":"2024-02-19T16:22:22.355544+0000","last_active":"2024-02-19T16:22:22.355544+0000","last_peered":"2024-02-19T16:22:22.355544+0000","last_clean":"2024-02-19T16:22:22.355544+0000","last_became_active":"2024-02-19T16:22:22.355184+0000","last_became_peered":"2024-02-19T16:22:22.355184+0000","last_unstale":"2024-02-19T16:22:22.355544+0000","last_undegraded":"2024-02-19T16:22:22.355544+0000","last_fullsized":"2024-02-19T16:22:22.355544+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:48:26.236286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351943+0000","last_change":"2024-02-19T16:21:22.216357+0000","last_active":"2024-02-19T16:22:22.351943+0000","last_peered":"2024-02-19T16:22:22.351943+0000","last_clean":"2024-02-19T16:22:22.351943+0000","last_became_active":"2024-02-19T16:21:22.215883+0000","last_became_peered":"2024-02-19T16:21:22.215883+0000","last_unstale":"2024-02-19T16:22:22.351943+0000","last_undegraded":"2024-02-19T16:22:22.351943+0000","last_fullsized":"2024-02-19T16:22:22.351943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:32:43.235515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":157,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:34.460388+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:22:34.460388+0000","last_peered":"2024-02-19T16:22:34.460388+0000","last_clean":"2024-02-19T16:22:34.460388+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:22:34.460388+0000","last_undegraded":"2024-02-19T16:22:34.460388+0000","last_fullsized":"2024-02-19T16:22:34.460388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350839+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:22:22.350839+0000","last_peered":"2024-02-19T16:22:22.350839+0000","last_clean":"2024-02-19T16:22:22.350839+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:22:22.350839+0000","last_undegraded":"2024-02-19T16:22:22.350839+0000","last_fullsized":"2024-02-19T16:22:22.350839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354823+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:22:22.354823+0000","last_peered":"2024-02-19T16:22:22.354823+0000","last_clean":"2024-02-19T16:22:22.354823+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:22:22.354823+0000","last_undegraded":"2024-02-19T16:22:22.354823+0000","last_fullsized":"2024-02-19T16:22:22.354823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351234+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:22:22.351234+0000","last_peered":"2024-02-19T16:22:22.351234+0000","last_clean":"2024-02-19T16:22:22.351234+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:22:22.351234+0000","last_undegraded":"2024-02-19T16:22:22.351234+0000","last_fullsized":"2024-02-19T16:22:22.351234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353983+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:22:22.353983+0000","last_peered":"2024-02-19T16:22:22.353983+0000","last_clean":"2024-02-19T16:22:22.353983+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:22:22.353983+0000","last_undegraded":"2024-02-19T16:22:22.353983+0000","last_fullsized":"2024-02-19T16:22:22.353983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353999+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:22:22.353999+0000","last_peered":"2024-02-19T16:22:22.353999+0000","last_clean":"2024-02-19T16:22:22.353999+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:22:22.353999+0000","last_undegraded":"2024-02-19T16:22:22.353999+0000","last_fullsized":"2024-02-19T16:22:22.353999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352102+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:22:22.352102+0000","last_peered":"2024-02-19T16:22:22.352102+0000","last_clean":"2024-02-19T16:22:22.352102+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:22:22.352102+0000","last_undegraded":"2024-02-19T16:22:22.352102+0000","last_fullsized":"2024-02-19T16:22:22.352102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449849+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:22:22.449849+0000","last_peered":"2024-02-19T16:22:22.449849+0000","last_clean":"2024-02-19T16:22:22.449849+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:22:22.449849+0000","last_undegraded":"2024-02-19T16:22:22.449849+0000","last_fullsized":"2024-02-19T16:22:22.449849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351345+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:22:22.351345+0000","last_peered":"2024-02-19T16:22:22.351345+0000","last_clean":"2024-02-19T16:22:22.351345+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:22:22.351345+0000","last_undegraded":"2024-02-19T16:22:22.351345+0000","last_fullsized":"2024-02-19T16:22:22.351345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351655+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:22:22.351655+0000","last_peered":"2024-02-19T16:22:22.351655+0000","last_clean":"2024-02-19T16:22:22.351655+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:22:22.351655+0000","last_undegraded":"2024-02-19T16:22:22.351655+0000","last_fullsized":"2024-02-19T16:22:22.351655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428974+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:22:22.428974+0000","last_peered":"2024-02-19T16:22:22.428974+0000","last_clean":"2024-02-19T16:22:22.428974+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:22:22.428974+0000","last_undegraded":"2024-02-19T16:22:22.428974+0000","last_fullsized":"2024-02-19T16:22:22.428974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354592+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:22:22.354592+0000","last_peered":"2024-02-19T16:22:22.354592+0000","last_clean":"2024-02-19T16:22:22.354592+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:22:22.354592+0000","last_undegraded":"2024-02-19T16:22:22.354592+0000","last_fullsized":"2024-02-19T16:22:22.354592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350901+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:22:22.350901+0000","last_peered":"2024-02-19T16:22:22.350901+0000","last_clean":"2024-02-19T16:22:22.350901+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:22:22.350901+0000","last_undegraded":"2024-02-19T16:22:22.350901+0000","last_fullsized":"2024-02-19T16:22:22.350901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428794+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:22:22.428794+0000","last_peered":"2024-02-19T16:22:22.428794+0000","last_clean":"2024-02-19T16:22:22.428794+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:22:22.428794+0000","last_undegraded":"2024-02-19T16:22:22.428794+0000","last_fullsized":"2024-02-19T16:22:22.428794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355307+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:22:22.355307+0000","last_peered":"2024-02-19T16:22:22.355307+0000","last_clean":"2024-02-19T16:22:22.355307+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:22:22.355307+0000","last_undegraded":"2024-02-19T16:22:22.355307+0000","last_fullsized":"2024-02-19T16:22:22.355307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353828+0000","last_change":"2024-02-19T16:22:22.353828+0000","last_active":"2024-02-19T16:22:22.353828+0000","last_peered":"2024-02-19T16:22:22.353828+0000","last_clean":"2024-02-19T16:22:22.353828+0000","last_became_active":"2024-02-19T16:22:22.353369+0000","last_became_peered":"2024-02-19T16:22:22.353369+0000","last_unstale":"2024-02-19T16:22:22.353828+0000","last_undegraded":"2024-02-19T16:22:22.353828+0000","last_fullsized":"2024-02-19T16:22:22.353828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:26:32.976202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":155,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:35.461056+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:22:35.461056+0000","last_peered":"2024-02-19T16:22:35.461056+0000","last_clean":"2024-02-19T16:22:35.461056+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:22:35.461056+0000","last_undegraded":"2024-02-19T16:22:35.461056+0000","last_fullsized":"2024-02-19T16:22:35.461056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351122+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:22:22.351122+0000","last_peered":"2024-02-19T16:22:22.351122+0000","last_clean":"2024-02-19T16:22:22.351122+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:22:22.351122+0000","last_undegraded":"2024-02-19T16:22:22.351122+0000","last_fullsized":"2024-02-19T16:22:22.351122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349313+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:22:22.349313+0000","last_peered":"2024-02-19T16:22:22.349313+0000","last_clean":"2024-02-19T16:22:22.349313+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:22:22.349313+0000","last_undegraded":"2024-02-19T16:22:22.349313+0000","last_fullsized":"2024-02-19T16:22:22.349313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349176+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:22:22.349176+0000","last_peered":"2024-02-19T16:22:22.349176+0000","last_clean":"2024-02-19T16:22:22.349176+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:22:22.349176+0000","last_undegraded":"2024-02-19T16:22:22.349176+0000","last_fullsized":"2024-02-19T16:22:22.349176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350666+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:22:22.350666+0000","last_peered":"2024-02-19T16:22:22.350666+0000","last_clean":"2024-02-19T16:22:22.350666+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:22:22.350666+0000","last_undegraded":"2024-02-19T16:22:22.350666+0000","last_fullsized":"2024-02-19T16:22:22.350666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354508+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:22:22.354508+0000","last_peered":"2024-02-19T16:22:22.354508+0000","last_clean":"2024-02-19T16:22:22.354508+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:22:22.354508+0000","last_undegraded":"2024-02-19T16:22:22.354508+0000","last_fullsized":"2024-02-19T16:22:22.354508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353185+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:22:22.353185+0000","last_peered":"2024-02-19T16:22:22.353185+0000","last_clean":"2024-02-19T16:22:22.353185+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:22:22.353185+0000","last_undegraded":"2024-02-19T16:22:22.353185+0000","last_fullsized":"2024-02-19T16:22:22.353185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350730+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:22:22.350730+0000","last_peered":"2024-02-19T16:22:22.350730+0000","last_clean":"2024-02-19T16:22:22.350730+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:22:22.350730+0000","last_undegraded":"2024-02-19T16:22:22.350730+0000","last_fullsized":"2024-02-19T16:22:22.350730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449675+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:22:22.449675+0000","last_peered":"2024-02-19T16:22:22.449675+0000","last_clean":"2024-02-19T16:22:22.449675+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:22:22.449675+0000","last_undegraded":"2024-02-19T16:22:22.449675+0000","last_fullsized":"2024-02-19T16:22:22.449675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351689+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:22:22.351689+0000","last_peered":"2024-02-19T16:22:22.351689+0000","last_clean":"2024-02-19T16:22:22.351689+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:22:22.351689+0000","last_undegraded":"2024-02-19T16:22:22.351689+0000","last_fullsized":"2024-02-19T16:22:22.351689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354591+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:22:22.354591+0000","last_peered":"2024-02-19T16:22:22.354591+0000","last_clean":"2024-02-19T16:22:22.354591+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:22:22.354591+0000","last_undegraded":"2024-02-19T16:22:22.354591+0000","last_fullsized":"2024-02-19T16:22:22.354591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348967+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:22:22.348967+0000","last_peered":"2024-02-19T16:22:22.348967+0000","last_clean":"2024-02-19T16:22:22.348967+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:22:22.348967+0000","last_undegraded":"2024-02-19T16:22:22.348967+0000","last_fullsized":"2024-02-19T16:22:22.348967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:32.339521+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:22:32.339521+0000","last_peered":"2024-02-19T16:22:32.339521+0000","last_clean":"2024-02-19T16:22:32.339521+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:22:32.339521+0000","last_undegraded":"2024-02-19T16:22:32.339521+0000","last_fullsized":"2024-02-19T16:22:32.339521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349548+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:22:22.349548+0000","last_peered":"2024-02-19T16:22:22.349548+0000","last_clean":"2024-02-19T16:22:22.349548+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:22:22.349548+0000","last_undegraded":"2024-02-19T16:22:22.349548+0000","last_fullsized":"2024-02-19T16:22:22.349548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352447+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:22:22.352447+0000","last_peered":"2024-02-19T16:22:22.352447+0000","last_clean":"2024-02-19T16:22:22.352447+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:22:22.352447+0000","last_undegraded":"2024-02-19T16:22:22.352447+0000","last_fullsized":"2024-02-19T16:22:22.352447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349728+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:22:22.349728+0000","last_peered":"2024-02-19T16:22:22.349728+0000","last_clean":"2024-02-19T16:22:22.349728+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:22:22.349728+0000","last_undegraded":"2024-02-19T16:22:22.349728+0000","last_fullsized":"2024-02-19T16:22:22.349728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450488+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:22:22.450488+0000","last_peered":"2024-02-19T16:22:22.450488+0000","last_clean":"2024-02-19T16:22:22.450488+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:22:22.450488+0000","last_undegraded":"2024-02-19T16:22:22.450488+0000","last_fullsized":"2024-02-19T16:22:22.450488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349405+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:22:22.349405+0000","last_peered":"2024-02-19T16:22:22.349405+0000","last_clean":"2024-02-19T16:22:22.349405+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:22:22.349405+0000","last_undegraded":"2024-02-19T16:22:22.349405+0000","last_fullsized":"2024-02-19T16:22:22.349405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449451+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:22:22.449451+0000","last_peered":"2024-02-19T16:22:22.449451+0000","last_clean":"2024-02-19T16:22:22.449451+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:22:22.449451+0000","last_undegraded":"2024-02-19T16:22:22.449451+0000","last_fullsized":"2024-02-19T16:22:22.449451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351520+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:22:22.351520+0000","last_peered":"2024-02-19T16:22:22.351520+0000","last_clean":"2024-02-19T16:22:22.351520+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:22:22.351520+0000","last_undegraded":"2024-02-19T16:22:22.351520+0000","last_fullsized":"2024-02-19T16:22:22.351520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351365+0000","last_change":"2024-02-19T16:21:22.216182+0000","last_active":"2024-02-19T16:22:22.351365+0000","last_peered":"2024-02-19T16:22:22.351365+0000","last_clean":"2024-02-19T16:22:22.351365+0000","last_became_active":"2024-02-19T16:21:22.215705+0000","last_became_peered":"2024-02-19T16:21:22.215705+0000","last_unstale":"2024-02-19T16:22:22.351365+0000","last_undegraded":"2024-02-19T16:22:22.351365+0000","last_fullsized":"2024-02-19T16:22:22.351365+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:29:29.574422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349162+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:22:22.349162+0000","last_peered":"2024-02-19T16:22:22.349162+0000","last_clean":"2024-02-19T16:22:22.349162+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:22:22.349162+0000","last_undegraded":"2024-02-19T16:22:22.349162+0000","last_fullsized":"2024-02-19T16:22:22.349162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449565+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:22:22.449565+0000","last_peered":"2024-02-19T16:22:22.449565+0000","last_clean":"2024-02-19T16:22:22.449565+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:22:22.449565+0000","last_undegraded":"2024-02-19T16:22:22.449565+0000","last_fullsized":"2024-02-19T16:22:22.449565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349272+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:22:22.349272+0000","last_peered":"2024-02-19T16:22:22.349272+0000","last_clean":"2024-02-19T16:22:22.349272+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:22:22.349272+0000","last_undegraded":"2024-02-19T16:22:22.349272+0000","last_fullsized":"2024-02-19T16:22:22.349272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353013+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:22:22.353013+0000","last_peered":"2024-02-19T16:22:22.353013+0000","last_clean":"2024-02-19T16:22:22.353013+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:22:22.353013+0000","last_undegraded":"2024-02-19T16:22:22.353013+0000","last_fullsized":"2024-02-19T16:22:22.353013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349632+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:22:22.349632+0000","last_peered":"2024-02-19T16:22:22.349632+0000","last_clean":"2024-02-19T16:22:22.349632+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:22:22.349632+0000","last_undegraded":"2024-02-19T16:22:22.349632+0000","last_fullsized":"2024-02-19T16:22:22.349632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349434+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:22:22.349434+0000","last_peered":"2024-02-19T16:22:22.349434+0000","last_clean":"2024-02-19T16:22:22.349434+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:22:22.349434+0000","last_undegraded":"2024-02-19T16:22:22.349434+0000","last_fullsized":"2024-02-19T16:22:22.349434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449599+0000","last_change":"2024-02-19T16:21:21.894342+0000","last_active":"2024-02-19T16:22:22.449599+0000","last_peered":"2024-02-19T16:22:22.449599+0000","last_clean":"2024-02-19T16:22:22.449599+0000","last_became_active":"2024-02-19T16:21:21.885142+0000","last_became_peered":"2024-02-19T16:21:21.885142+0000","last_unstale":"2024-02-19T16:22:22.449599+0000","last_undegraded":"2024-02-19T16:22:22.449599+0000","last_fullsized":"2024-02-19T16:22:22.449599+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:19:30.846573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351023+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:22:22.351023+0000","last_peered":"2024-02-19T16:22:22.351023+0000","last_clean":"2024-02-19T16:22:22.351023+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:22:22.351023+0000","last_undegraded":"2024-02-19T16:22:22.351023+0000","last_fullsized":"2024-02-19T16:22:22.351023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449534+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:22:22.449534+0000","last_peered":"2024-02-19T16:22:22.449534+0000","last_clean":"2024-02-19T16:22:22.449534+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:22:22.449534+0000","last_undegraded":"2024-02-19T16:22:22.449534+0000","last_fullsized":"2024-02-19T16:22:22.449534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353273+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:22:22.353273+0000","last_peered":"2024-02-19T16:22:22.353273+0000","last_clean":"2024-02-19T16:22:22.353273+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:22:22.353273+0000","last_undegraded":"2024-02-19T16:22:22.353273+0000","last_fullsized":"2024-02-19T16:22:22.353273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351249+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:22:22.351249+0000","last_peered":"2024-02-19T16:22:22.351249+0000","last_clean":"2024-02-19T16:22:22.351249+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:22:22.351249+0000","last_undegraded":"2024-02-19T16:22:22.351249+0000","last_fullsized":"2024-02-19T16:22:22.351249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450177+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:22:22.450177+0000","last_peered":"2024-02-19T16:22:22.450177+0000","last_clean":"2024-02-19T16:22:22.450177+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:22:22.450177+0000","last_undegraded":"2024-02-19T16:22:22.450177+0000","last_fullsized":"2024-02-19T16:22:22.450177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354439+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:22:22.354439+0000","last_peered":"2024-02-19T16:22:22.354439+0000","last_clean":"2024-02-19T16:22:22.354439+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:22:22.354439+0000","last_undegraded":"2024-02-19T16:22:22.354439+0000","last_fullsized":"2024-02-19T16:22:22.354439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351048+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:22:22.351048+0000","last_peered":"2024-02-19T16:22:22.351048+0000","last_clean":"2024-02-19T16:22:22.351048+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:22:22.351048+0000","last_undegraded":"2024-02-19T16:22:22.351048+0000","last_fullsized":"2024-02-19T16:22:22.351048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353266+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:22:22.353266+0000","last_peered":"2024-02-19T16:22:22.353266+0000","last_clean":"2024-02-19T16:22:22.353266+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:22:22.353266+0000","last_undegraded":"2024-02-19T16:22:22.353266+0000","last_fullsized":"2024-02-19T16:22:22.353266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354966+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:22:22.354966+0000","last_peered":"2024-02-19T16:22:22.354966+0000","last_clean":"2024-02-19T16:22:22.354966+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:22:22.354966+0000","last_undegraded":"2024-02-19T16:22:22.354966+0000","last_fullsized":"2024-02-19T16:22:22.354966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351376+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:22:22.351376+0000","last_peered":"2024-02-19T16:22:22.351376+0000","last_clean":"2024-02-19T16:22:22.351376+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:22:22.351376+0000","last_undegraded":"2024-02-19T16:22:22.351376+0000","last_fullsized":"2024-02-19T16:22:22.351376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449671+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:22:22.449671+0000","last_peered":"2024-02-19T16:22:22.449671+0000","last_clean":"2024-02-19T16:22:22.449671+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:22:22.449671+0000","last_undegraded":"2024-02-19T16:22:22.449671+0000","last_fullsized":"2024-02-19T16:22:22.449671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352733+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:22:22.352733+0000","last_peered":"2024-02-19T16:22:22.352733+0000","last_clean":"2024-02-19T16:22:22.352733+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:22:22.352733+0000","last_undegraded":"2024-02-19T16:22:22.352733+0000","last_fullsized":"2024-02-19T16:22:22.352733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352775+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:22:22.352775+0000","last_peered":"2024-02-19T16:22:22.352775+0000","last_clean":"2024-02-19T16:22:22.352775+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:22:22.352775+0000","last_undegraded":"2024-02-19T16:22:22.352775+0000","last_fullsized":"2024-02-19T16:22:22.352775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450420+0000","last_change":"2024-02-19T16:21:21.884814+0000","last_active":"2024-02-19T16:22:22.450420+0000","last_peered":"2024-02-19T16:22:22.450420+0000","last_clean":"2024-02-19T16:22:22.450420+0000","last_became_active":"2024-02-19T16:21:21.884410+0000","last_became_peered":"2024-02-19T16:21:21.884410+0000","last_unstale":"2024-02-19T16:22:22.450420+0000","last_undegraded":"2024-02-19T16:22:22.450420+0000","last_fullsized":"2024-02-19T16:22:22.450420+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:39:44.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354725+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:22:22.354725+0000","last_peered":"2024-02-19T16:22:22.354725+0000","last_clean":"2024-02-19T16:22:22.354725+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:22:22.354725+0000","last_undegraded":"2024-02-19T16:22:22.354725+0000","last_fullsized":"2024-02-19T16:22:22.354725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449925+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:22:22.449925+0000","last_peered":"2024-02-19T16:22:22.449925+0000","last_clean":"2024-02-19T16:22:22.449925+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:22:22.449925+0000","last_undegraded":"2024-02-19T16:22:22.449925+0000","last_fullsized":"2024-02-19T16:22:22.449925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354587+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:22:22.354587+0000","last_peered":"2024-02-19T16:22:22.354587+0000","last_clean":"2024-02-19T16:22:22.354587+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:22:22.354587+0000","last_undegraded":"2024-02-19T16:22:22.354587+0000","last_fullsized":"2024-02-19T16:22:22.354587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349718+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:22:22.349718+0000","last_peered":"2024-02-19T16:22:22.349718+0000","last_clean":"2024-02-19T16:22:22.349718+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:22:22.349718+0000","last_undegraded":"2024-02-19T16:22:22.349718+0000","last_fullsized":"2024-02-19T16:22:22.349718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353764+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:22:22.353764+0000","last_peered":"2024-02-19T16:22:22.353764+0000","last_clean":"2024-02-19T16:22:22.353764+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:22:22.353764+0000","last_undegraded":"2024-02-19T16:22:22.353764+0000","last_fullsized":"2024-02-19T16:22:22.353764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349237+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:22:22.349237+0000","last_peered":"2024-02-19T16:22:22.349237+0000","last_clean":"2024-02-19T16:22:22.349237+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:22:22.349237+0000","last_undegraded":"2024-02-19T16:22:22.349237+0000","last_fullsized":"2024-02-19T16:22:22.349237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449690+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:22:22.449690+0000","last_peered":"2024-02-19T16:22:22.449690+0000","last_clean":"2024-02-19T16:22:22.449690+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:22:22.449690+0000","last_undegraded":"2024-02-19T16:22:22.449690+0000","last_fullsized":"2024-02-19T16:22:22.449690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353873+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:22:22.353873+0000","last_peered":"2024-02-19T16:22:22.353873+0000","last_clean":"2024-02-19T16:22:22.353873+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:22:22.353873+0000","last_undegraded":"2024-02-19T16:22:22.353873+0000","last_fullsized":"2024-02-19T16:22:22.353873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449777+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:22:22.449777+0000","last_peered":"2024-02-19T16:22:22.449777+0000","last_clean":"2024-02-19T16:22:22.449777+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:22:22.449777+0000","last_undegraded":"2024-02-19T16:22:22.449777+0000","last_fullsized":"2024-02-19T16:22:22.449777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":732,"num_read_kb":442,"num_write":173,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":213861,"data_compressed":5154,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":532611,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.56999999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.498,"15min":0.54200000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.53500000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.47999999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40200000000000002,"15min":0.42299999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.53300000000000003,"15min":0.56699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60999999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48199999999999998,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.51800000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.61699999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44700000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.61699999999999999},"min":{"1min":0.48099999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69999999999999996,"5min":0.998,"15min":0.998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61099999999999999,"15min":0.64400000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67800000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.66500000000000004},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63700000000000001}]},{"osd":9,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65600000000000003,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71999999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66400000000000003,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.78400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68200000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.755,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.65500000000000003,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.63800000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60499999999999998}]}]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1188316,"data_compressed":67184,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.32900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63900000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.495},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.55700000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.377}]},{"osd":1,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.40200000000000002,"15min":0.432},"min":{"1min":0.23499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.34000000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.45700000000000002,"15min":0.49399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.377,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.499,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.48399999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.50700000000000001,"15min":0.54700000000000004},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.438,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35499999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56299999999999994,"15min":0.60099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61799999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.67400000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.50600000000000001,"15min":0.55000000000000004},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.46200000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.28299999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57599999999999996,"15min":0.61799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.56999999999999995,"15min":0.61899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.50800000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38700000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.65400000000000003,"5min":0.999,"15min":0.999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.65900000000000003,"15min":0.69499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71999999999999997,"5min":1.028,"15min":1.028},"last":0.50800000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.67400000000000004,"15min":0.73199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60299999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.66200000000000003,"15min":0.72399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.55800000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.48499999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.69199999999999995,"15min":0.73599999999999999},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.63300000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.68899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:21:37 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.56399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54100000000000004}]}]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":536361,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629}]},{"osd":1,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.626,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.55600000000000005,"15min":0.58099999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.627},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66000000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72699999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.29299999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.436},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.66400000000000003,"15min":0.69399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.749,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82099999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.73299999999999998,"15min":0.78000000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76000000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57199999999999995}]},{"osd":8,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67700000000000005,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":1.169,"15min":1.169},"last":0.70899999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.75,"15min":0.78400000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.89500000000000002,"5min":1.278,"15min":1.278},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71999999999999997,"15min":0.76200000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.78000000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64400000000000002}]},{"osd":11,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":0.78700000000000003,"15min":0.68200000000000005},"min":{"1min":0.53600000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":13.175000000000001,"5min":13.175000000000001,"15min":13.175000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54000000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532185,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.44400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48499999999999999,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.52500000000000002,"15min":0.56899999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.505,"5min":0.875,"15min":0.875},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.313,"5min":0.31,"15min":0.31},"max":{"1min":0.56799999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35099999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40200000000000002,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41299999999999998,"15min":0.42399999999999999},"min":{"1min":0.27300000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.53100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.317}]},{"osd":3,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.64600000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.497},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.54800000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56799999999999995,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.58299999999999996},"min":{"1min":0.29599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76100000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70899999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.68200000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77600000000000002,"5min":1.393,"15min":1.393},"last":0.46800000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.52200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.73599999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68600000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.72299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.441},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.74299999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63300000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.78500000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.77300000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1127653,"data_compressed":65466,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.379,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.38200000000000001,"5min":0.75,"15min":0.75},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.49299999999999999,"15min":0.53700000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.41899999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40699999999999997}]},{"osd":3,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.64100000000000001,"5min":0.87,"15min":0.87},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.46600000000000003,"15min":0.48099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.46500000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432}]},{"osd":5,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.55500000000000005,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.504,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.61299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45400000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64100000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.61799999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59199999999999997}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.60199999999999998,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.57199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.58499999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.625}]},{"osd":9,"last update":"Mon Feb 19 16:21:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67400000000000004,"5min":0.997,"15min":0.997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70099999999999996,"15min":0.71499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.70599999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54600000000000004}]},{"osd":10,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.67200000000000004,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.64200000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.64000000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48999999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.65200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.66800000000000004,"5min":1.024,"15min":1.024},"last":0.60499999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":527441,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80600000000000005}]},{"osd":2,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82899999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.48099999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.56999999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94399999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76000000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.56799999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":1.014,"15min":1.014},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91300000000000003,"5min":1.113,"15min":1.113},"last":0.84099999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86299999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.50700000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.872},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.60399999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0089999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81999999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.878,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0449999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":589375,"data_compressed":31426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.68700000000000006,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.2210000000000001,"5min":14.067,"15min":14.067},"last":0.52800000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.47299999999999998,"15min":0.51400000000000001},"min":{"1min":0.26700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.46400000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.502,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.41399999999999998,"15min":0.376},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59999999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44400000000000001,"15min":0.40500000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.313}]},{"osd":5,"last update":"Mon Feb 19 16:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.371},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.44600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70899999999999996,"5min":0.879,"15min":0.879},"last":0.40400000000000003}]},{"osd":6,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.625,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71399999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51500000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68100000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.503}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.71799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42199999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.82099999999999995,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":14.105,"5min":14.105,"15min":14.105},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71099999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48199999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.63400000000000001,"15min":0.69599999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.63800000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71199999999999997,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.67200000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56100000000000005}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.68999999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.70599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.62}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528829,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.54400000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67100000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71499999999999997}]},{"osd":2,"last update":"Mon Feb 19 16:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58899999999999997,"5min":1.899,"15min":1.899},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.38,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Feb 19 16:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.62,"5min":0.63300000000000001,"15min":0.60199999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80700000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.624}]},{"osd":7,"last update":"Mon Feb 19 16:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84999999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.51300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86499999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.45900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.98499999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.90500000000000003}]},{"osd":11,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":599317,"data_compressed":31426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59199999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61099999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.39000000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.47299999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70599999999999996,"5min":1.758,"15min":1.758},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59299999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.377},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61799999999999999,"5min":2.012,"15min":2.012},"last":0.439}]},{"osd":5,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.64100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.47799999999999998},"min":{"1min":0.317,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58299999999999996,"5min":1.026,"15min":1.026},"last":0.496}]},{"osd":6,"last update":"Mon Feb 19 16:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.621},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.622},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61699999999999999,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69799999999999995,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.52300000000000002}]},{"osd":8,"last update":"Mon Feb 19 16:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53900000000000003,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.63800000000000001},"min":{"1min":0.503,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.626,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53500000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69399999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.627},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.755,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.60499999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.70899999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.67600000000000005,"5min":1.282,"15min":1.282},"last":0.56200000000000006}]},{"osd":11,"last update":"Mon Feb 19 16:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.72599999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59399999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":527635,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.36499999999999999},"min":{"1min":0.214,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.379},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.378}]},{"osd":1,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60699999999999998,"5min":0.81399999999999995,"15min":0.90800000000000003},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56799999999999995,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64000000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82399999999999995,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77800000000000002,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.018},"last":0.66700000000000004}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.05,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57799999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.999,"5min":1.03,"15min":1.03},"last":0.71799999999999997}]},{"osd":11,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":532623,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.26300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.34899999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.373,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.311}]},{"osd":2,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.51500000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.433}]},{"osd":3,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":1.03,"15min":1.03},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.60499999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.47399999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.55000000000000004,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.51100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47399999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.502}]},{"osd":7,"last update":"Mon Feb 19 16:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.53400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55200000000000005}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.58299999999999996,"5min":1.073,"15min":1.073},"last":0.52800000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":0.64100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.70299999999999996,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.65300000000000002,"5min":1.044,"15min":1.044},"last":0.61199999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.67000000000000004,"15min":0.70799999999999996},"min":{"1min":0.442,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.64100000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.70699999999999996,"15min":0.75},"min":{"1min":0.45400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66500000000000004,"5min":1.056,"15min":1.056},"last":0.51500000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70599999999999996,"5min":1.133,"15min":1.133},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.69899999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.621}]}]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1123291,"data_compressed":65466,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.64300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67400000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.48299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53200000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.372,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38600000000000001,"15min":0.375},"min":{"1min":0.25,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68700000000000006,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.871,"15min":0.871},"last":0.436},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.505,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65600000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.65100000000000002,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70799999999999996,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.42199999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:22:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66600000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64200000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46800000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.318,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72099999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.57099999999999995,"15min":0.623},"min":{"1min":0.38200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53300000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66300000000000003,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.74199999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55600000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68700000000000006,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.68200000000000005,"15min":0.72199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.67800000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62332,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71966,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70597,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:22:42.082 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-19T16:22:42.083 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 -- 172.21.15.8:0/3507674723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad41030d0 msgr2=0x7f8ad4109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 --2- 172.21.15.8:0/3507674723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad41030d0 0x7f8ad4109960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009ca0 tx=0x7f8acc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 -- 172.21.15.8:0/3507674723 shutdown_connections 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 --2- 172.21.15.8:0/3507674723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad41030d0 0x7f8ad4109960 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 --2- 172.21.15.8:0/3507674723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad4102710 0x7f8ad4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 --2- 172.21.15.8:0/3507674723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 0x7f8ad4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 -- 172.21.15.8:0/3507674723 >> 172.21.15.8:0/3507674723 conn(0x7f8ad40fca90 msgr2=0x7f8ad40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 -- 172.21.15.8:0/3507674723 shutdown_connections 2024-02-19T16:22:42.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 -- 172.21.15.8:0/3507674723 wait complete. 2024-02-19T16:22:42.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.185+0000 7f8ada435640 1 Processor -- start 2024-02-19T16:22:42.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 -- start start 2024-02-19T16:22:42.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 0x7f8ad40752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 0x7f8ad40738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad41030d0 0x7f8ad4073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad4074580 con 0x7f8ad4102710 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40746f0 con 0x7f8ad41030d0 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ada435640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad41a15b0 con 0x7f8ad4101520 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 0x7f8ad40738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 0x7f8ad40752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 0x7f8ad40738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49588/0 (socket says 172.21.15.8:49588) 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 -- 172.21.15.8:0/3686323807 learned_addr learned my addr 172.21.15.8:0/3686323807 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad89d2640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad41030d0 0x7f8ad4073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 -- 172.21.15.8:0/3686323807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 msgr2=0x7f8ad40752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 0x7f8ad40752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 -- 172.21.15.8:0/3686323807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad41030d0 msgr2=0x7f8ad4073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad41030d0 0x7f8ad4073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 -- 172.21.15.8:0/3686323807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8009490 con 0x7f8ad4102710 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad3fff640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 0x7f8ad40752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.186+0000 7f8ad37fe640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 0x7f8ad40738e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800ea90 tx=0x7f8ac800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.187+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac800ce40 con 0x7f8ad4102710 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.187+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ac80035f0 con 0x7f8ad4102710 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.187+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc009900 con 0x7f8ad4102710 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.187+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8010690 con 0x7f8ad4102710 2024-02-19T16:22:42.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.187+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad41a1be0 con 0x7f8ad4102710 2024-02-19T16:22:42.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.188+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad4108840 con 0x7f8ad4102710 2024-02-19T16:22:42.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.189+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f8ac8003ad0 con 0x7f8ad4102710 2024-02-19T16:22:42.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.190+0000 7f8ad17fa640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8abc0a2150 0x7f8abc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.190+0000 7f8ad3fff640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8abc0a2150 0x7f8abc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.190+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f8ac8014070 con 0x7f8ad4102710 2024-02-19T16:22:42.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.190+0000 7f8ad3fff640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8abc0a2150 0x7f8abc0a4610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4102570 tx=0x7f8ac4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:42.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.194+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8ac8097130 con 0x7f8ad4102710 2024-02-19T16:22:42.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.371+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ad402d070 con 0x7f8abc0a2150 2024-02-19T16:22:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.384+0000 7f8ad17fa640 1 -- 172.21.15.8:0/3686323807 <== mgr.14152 v2:172.21.15.8:6800/354789085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428108 (secure 0 0 0) 0x7f8ad402d070 con 0x7f8abc0a2150 2024-02-19T16:22:42.390 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:42.392 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8abc0a2150 msgr2=0x7f8abc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8abc0a2150 0x7f8abc0a4610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4102570 tx=0x7f8ac4005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 msgr2=0x7f8ad40738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 0x7f8ad40738e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800ea90 tx=0x7f8ac800ef50 comp rx=0 tx=0).stop 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 shutdown_connections 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f8abc0a2150 0x7f8abc0a4610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad41030d0 0x7f8ad4073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad4102710 0x7f8ad40738e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 --2- 172.21.15.8:0/3686323807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad4101520 0x7f8ad40752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 >> 172.21.15.8:0/3686323807 conn(0x7f8ad40fca90 msgr2=0x7f8ad4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:42.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 shutdown_connections 2024-02-19T16:22:42.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.388+0000 7f8ada435640 1 -- 172.21.15.8:0/3686323807 wait complete. 2024-02-19T16:22:42.400 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-02-19T16:22:40.748176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":868,"num_read_kb":561,"num_write":356,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378548,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563276,"statfs":{"total":1151940427776,"available":1151552794624,"internally_reserved":0,"allocated":12992512,"data_stored":8345637,"data_compressed":468924,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003373"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450074+0000","last_change":"2024-02-19T16:20:21.666731+0000","last_active":"2024-02-19T16:22:22.450074+0000","last_peered":"2024-02-19T16:22:22.450074+0000","last_clean":"2024-02-19T16:22:22.450074+0000","last_became_active":"2024-02-19T16:20:21.666079+0000","last_became_peered":"2024-02-19T16:20:21.666079+0000","last_unstale":"2024-02-19T16:22:22.450074+0000","last_undegraded":"2024-02-19T16:22:22.450074+0000","last_fullsized":"2024-02-19T16:22:22.450074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:17:27.262408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450435+0000","last_change":"2024-02-19T16:20:21.661612+0000","last_active":"2024-02-19T16:22:22.450435+0000","last_peered":"2024-02-19T16:22:22.450435+0000","last_clean":"2024-02-19T16:22:22.450435+0000","last_became_active":"2024-02-19T16:20:21.661273+0000","last_became_peered":"2024-02-19T16:20:21.661273+0000","last_unstale":"2024-02-19T16:22:22.450435+0000","last_undegraded":"2024-02-19T16:22:22.450435+0000","last_fullsized":"2024-02-19T16:22:22.450435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:46:48.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450286+0000","last_change":"2024-02-19T16:20:21.655966+0000","last_active":"2024-02-19T16:22:22.450286+0000","last_peered":"2024-02-19T16:22:22.450286+0000","last_clean":"2024-02-19T16:22:22.450286+0000","last_became_active":"2024-02-19T16:20:21.655493+0000","last_became_peered":"2024-02-19T16:20:21.655493+0000","last_unstale":"2024-02-19T16:22:22.450286+0000","last_undegraded":"2024-02-19T16:22:22.450286+0000","last_fullsized":"2024-02-19T16:22:22.450286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:05:43.120965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349393+0000","last_change":"2024-02-19T16:20:21.664061+0000","last_active":"2024-02-19T16:22:22.349393+0000","last_peered":"2024-02-19T16:22:22.349393+0000","last_clean":"2024-02-19T16:22:22.349393+0000","last_became_active":"2024-02-19T16:20:21.663700+0000","last_became_peered":"2024-02-19T16:20:21.663700+0000","last_unstale":"2024-02-19T16:22:22.349393+0000","last_undegraded":"2024-02-19T16:22:22.349393+0000","last_fullsized":"2024-02-19T16:22:22.349393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:27:05.250285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351320+0000","last_change":"2024-02-19T16:20:21.663589+0000","last_active":"2024-02-19T16:22:22.351320+0000","last_peered":"2024-02-19T16:22:22.351320+0000","last_clean":"2024-02-19T16:22:22.351320+0000","last_became_active":"2024-02-19T16:20:21.663284+0000","last_became_peered":"2024-02-19T16:20:21.663284+0000","last_unstale":"2024-02-19T16:22:22.351320+0000","last_undegraded":"2024-02-19T16:22:22.351320+0000","last_fullsized":"2024-02-19T16:22:22.351320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:39:04.139980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428710+0000","last_change":"2024-02-19T16:20:21.660864+0000","last_active":"2024-02-19T16:22:22.428710+0000","last_peered":"2024-02-19T16:22:22.428710+0000","last_clean":"2024-02-19T16:22:22.428710+0000","last_became_active":"2024-02-19T16:20:21.660206+0000","last_became_peered":"2024-02-19T16:20:21.660206+0000","last_unstale":"2024-02-19T16:22:22.428710+0000","last_undegraded":"2024-02-19T16:22:22.428710+0000","last_fullsized":"2024-02-19T16:22:22.428710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:49:55.982147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350749+0000","last_change":"2024-02-19T16:20:21.661703+0000","last_active":"2024-02-19T16:22:22.350749+0000","last_peered":"2024-02-19T16:22:22.350749+0000","last_clean":"2024-02-19T16:22:22.350749+0000","last_became_active":"2024-02-19T16:20:21.661379+0000","last_became_peered":"2024-02-19T16:20:21.661379+0000","last_unstale":"2024-02-19T16:22:22.350749+0000","last_undegraded":"2024-02-19T16:22:22.350749+0000","last_fullsized":"2024-02-19T16:22:22.350749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:36:11.542378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348892+0000","last_change":"2024-02-19T16:20:21.670783+0000","last_active":"2024-02-19T16:22:22.348892+0000","last_peered":"2024-02-19T16:22:22.348892+0000","last_clean":"2024-02-19T16:22:22.348892+0000","last_became_active":"2024-02-19T16:20:21.670452+0000","last_became_peered":"2024-02-19T16:20:21.670452+0000","last_unstale":"2024-02-19T16:22:22.348892+0000","last_undegraded":"2024-02-19T16:22:22.348892+0000","last_fullsized":"2024-02-19T16:22:22.348892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:18.054886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449821+0000","last_change":"2024-02-19T16:20:21.662006+0000","last_active":"2024-02-19T16:22:22.449821+0000","last_peered":"2024-02-19T16:22:22.449821+0000","last_clean":"2024-02-19T16:22:22.449821+0000","last_became_active":"2024-02-19T16:20:21.661698+0000","last_became_peered":"2024-02-19T16:20:21.661698+0000","last_unstale":"2024-02-19T16:22:22.449821+0000","last_undegraded":"2024-02-19T16:22:22.449821+0000","last_fullsized":"2024-02-19T16:22:22.449821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:40.244841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428995+0000","last_change":"2024-02-19T16:20:21.661651+0000","last_active":"2024-02-19T16:22:22.428995+0000","last_peered":"2024-02-19T16:22:22.428995+0000","last_clean":"2024-02-19T16:22:22.428995+0000","last_became_active":"2024-02-19T16:20:21.661357+0000","last_became_peered":"2024-02-19T16:20:21.661357+0000","last_unstale":"2024-02-19T16:22:22.428995+0000","last_undegraded":"2024-02-19T16:22:22.428995+0000","last_fullsized":"2024-02-19T16:22:22.428995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:24:32.409229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351456+0000","last_change":"2024-02-19T16:20:21.665275+0000","last_active":"2024-02-19T16:22:22.351456+0000","last_peered":"2024-02-19T16:22:22.351456+0000","last_clean":"2024-02-19T16:22:22.351456+0000","last_became_active":"2024-02-19T16:20:21.664562+0000","last_became_peered":"2024-02-19T16:20:21.664562+0000","last_unstale":"2024-02-19T16:22:22.351456+0000","last_undegraded":"2024-02-19T16:22:22.351456+0000","last_fullsized":"2024-02-19T16:22:22.351456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:07.025405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349352+0000","last_change":"2024-02-19T16:20:21.668258+0000","last_active":"2024-02-19T16:22:22.349352+0000","last_peered":"2024-02-19T16:22:22.349352+0000","last_clean":"2024-02-19T16:22:22.349352+0000","last_became_active":"2024-02-19T16:20:21.667902+0000","last_became_peered":"2024-02-19T16:20:21.667902+0000","last_unstale":"2024-02-19T16:22:22.349352+0000","last_undegraded":"2024-02-19T16:22:22.349352+0000","last_fullsized":"2024-02-19T16:22:22.349352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:43:51.887411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354017+0000","last_change":"2024-02-19T16:20:21.661767+0000","last_active":"2024-02-19T16:22:22.354017+0000","last_peered":"2024-02-19T16:22:22.354017+0000","last_clean":"2024-02-19T16:22:22.354017+0000","last_became_active":"2024-02-19T16:20:21.661432+0000","last_became_peered":"2024-02-19T16:20:21.661432+0000","last_unstale":"2024-02-19T16:22:22.354017+0000","last_undegraded":"2024-02-19T16:22:22.354017+0000","last_fullsized":"2024-02-19T16:22:22.354017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:21:21.680826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450028+0000","last_change":"2024-02-19T16:20:21.659704+0000","last_active":"2024-02-19T16:22:22.450028+0000","last_peered":"2024-02-19T16:22:22.450028+0000","last_clean":"2024-02-19T16:22:22.450028+0000","last_became_active":"2024-02-19T16:20:21.659345+0000","last_became_peered":"2024-02-19T16:20:21.659345+0000","last_unstale":"2024-02-19T16:22:22.450028+0000","last_undegraded":"2024-02-19T16:22:22.450028+0000","last_fullsized":"2024-02-19T16:22:22.450028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:11:57.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450061+0000","last_change":"2024-02-19T16:20:21.657749+0000","last_active":"2024-02-19T16:22:22.450061+0000","last_peered":"2024-02-19T16:22:22.450061+0000","last_clean":"2024-02-19T16:22:22.450061+0000","last_became_active":"2024-02-19T16:20:21.656233+0000","last_became_peered":"2024-02-19T16:20:21.656233+0000","last_unstale":"2024-02-19T16:22:22.450061+0000","last_undegraded":"2024-02-19T16:22:22.450061+0000","last_fullsized":"2024-02-19T16:22:22.450061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:47:22.590720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449363+0000","last_change":"2024-02-19T16:20:21.657047+0000","last_active":"2024-02-19T16:22:22.449363+0000","last_peered":"2024-02-19T16:22:22.449363+0000","last_clean":"2024-02-19T16:22:22.449363+0000","last_became_active":"2024-02-19T16:20:21.656282+0000","last_became_peered":"2024-02-19T16:20:21.656282+0000","last_unstale":"2024-02-19T16:22:22.449363+0000","last_undegraded":"2024-02-19T16:22:22.449363+0000","last_fullsized":"2024-02-19T16:22:22.449363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:43:41.611341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449854+0000","last_change":"2024-02-19T16:20:21.666909+0000","last_active":"2024-02-19T16:22:22.449854+0000","last_peered":"2024-02-19T16:22:22.449854+0000","last_clean":"2024-02-19T16:22:22.449854+0000","last_became_active":"2024-02-19T16:20:21.666464+0000","last_became_peered":"2024-02-19T16:20:21.666464+0000","last_unstale":"2024-02-19T16:22:22.449854+0000","last_undegraded":"2024-02-19T16:22:22.449854+0000","last_fullsized":"2024-02-19T16:22:22.449854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:06:38.285998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351299+0000","last_change":"2024-02-19T16:20:21.659745+0000","last_active":"2024-02-19T16:22:22.351299+0000","last_peered":"2024-02-19T16:22:22.351299+0000","last_clean":"2024-02-19T16:22:22.351299+0000","last_became_active":"2024-02-19T16:20:21.659426+0000","last_became_peered":"2024-02-19T16:20:21.659426+0000","last_unstale":"2024-02-19T16:22:22.351299+0000","last_undegraded":"2024-02-19T16:22:22.351299+0000","last_fullsized":"2024-02-19T16:22:22.351299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:54.537026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353320+0000","last_change":"2024-02-19T16:20:21.660241+0000","last_active":"2024-02-19T16:22:22.353320+0000","last_peered":"2024-02-19T16:22:22.353320+0000","last_clean":"2024-02-19T16:22:22.353320+0000","last_became_active":"2024-02-19T16:20:21.659903+0000","last_became_peered":"2024-02-19T16:20:21.659903+0000","last_unstale":"2024-02-19T16:22:22.353320+0000","last_undegraded":"2024-02-19T16:22:22.353320+0000","last_fullsized":"2024-02-19T16:22:22.353320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.828563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352554+0000","last_change":"2024-02-19T16:20:25.765539+0000","last_active":"2024-02-19T16:22:22.352554+0000","last_peered":"2024-02-19T16:22:22.352554+0000","last_clean":"2024-02-19T16:22:22.352554+0000","last_became_active":"2024-02-19T16:20:25.764854+0000","last_became_peered":"2024-02-19T16:20:25.764854+0000","last_unstale":"2024-02-19T16:22:22.352554+0000","last_undegraded":"2024-02-19T16:22:22.352554+0000","last_fullsized":"2024-02-19T16:22:22.352554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:50.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351551+0000","last_change":"2024-02-19T16:21:21.892098+0000","last_active":"2024-02-19T16:22:22.351551+0000","last_peered":"2024-02-19T16:22:22.351551+0000","last_clean":"2024-02-19T16:22:22.351551+0000","last_became_active":"2024-02-19T16:21:21.884753+0000","last_became_peered":"2024-02-19T16:21:21.884753+0000","last_unstale":"2024-02-19T16:22:22.351551+0000","last_undegraded":"2024-02-19T16:22:22.351551+0000","last_fullsized":"2024-02-19T16:22:22.351551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:24.360019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355648+0000","last_change":"2024-02-19T16:22:22.355648+0000","last_active":"2024-02-19T16:22:22.355648+0000","last_peered":"2024-02-19T16:22:22.355648+0000","last_clean":"2024-02-19T16:22:22.355648+0000","last_became_active":"2024-02-19T16:22:22.355213+0000","last_became_peered":"2024-02-19T16:22:22.355213+0000","last_unstale":"2024-02-19T16:22:22.355648+0000","last_undegraded":"2024-02-19T16:22:22.355648+0000","last_fullsized":"2024-02-19T16:22:22.355648+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:49:24.360051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348894+0000","last_change":"2024-02-19T16:20:25.761475+0000","last_active":"2024-02-19T16:22:22.348894+0000","last_peered":"2024-02-19T16:22:22.348894+0000","last_clean":"2024-02-19T16:22:22.348894+0000","last_became_active":"2024-02-19T16:20:25.761077+0000","last_became_peered":"2024-02-19T16:20:25.761077+0000","last_unstale":"2024-02-19T16:22:22.348894+0000","last_undegraded":"2024-02-19T16:22:22.348894+0000","last_fullsized":"2024-02-19T16:22:22.348894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:27:19.262849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350052+0000","last_change":"2024-02-19T16:20:20.637040+0000","last_active":"2024-02-19T16:22:22.350052+0000","last_peered":"2024-02-19T16:22:22.350052+0000","last_clean":"2024-02-19T16:22:22.350052+0000","last_became_active":"2024-02-19T16:20:20.636723+0000","last_became_peered":"2024-02-19T16:20:20.636723+0000","last_unstale":"2024-02-19T16:22:22.350052+0000","last_undegraded":"2024-02-19T16:22:22.350052+0000","last_fullsized":"2024-02-19T16:22:22.350052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:59:10.218659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450474+0000","last_change":"2024-02-19T16:20:21.654912+0000","last_active":"2024-02-19T16:22:22.450474+0000","last_peered":"2024-02-19T16:22:22.450474+0000","last_clean":"2024-02-19T16:22:22.450474+0000","last_became_active":"2024-02-19T16:20:21.654553+0000","last_became_peered":"2024-02-19T16:20:21.654553+0000","last_unstale":"2024-02-19T16:22:22.450474+0000","last_undegraded":"2024-02-19T16:22:22.450474+0000","last_fullsized":"2024-02-19T16:22:22.450474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:16:21.692057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349058+0000","last_change":"2024-02-19T16:20:25.757128+0000","last_active":"2024-02-19T16:22:22.349058+0000","last_peered":"2024-02-19T16:22:22.349058+0000","last_clean":"2024-02-19T16:22:22.349058+0000","last_became_active":"2024-02-19T16:20:25.756696+0000","last_became_peered":"2024-02-19T16:20:25.756696+0000","last_unstale":"2024-02-19T16:22:22.349058+0000","last_undegraded":"2024-02-19T16:22:22.349058+0000","last_fullsized":"2024-02-19T16:22:22.349058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:22:10.232212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351817+0000","last_change":"2024-02-19T16:21:21.889760+0000","last_active":"2024-02-19T16:22:22.351817+0000","last_peered":"2024-02-19T16:22:22.351817+0000","last_clean":"2024-02-19T16:22:22.351817+0000","last_became_active":"2024-02-19T16:21:21.884230+0000","last_became_peered":"2024-02-19T16:21:21.884230+0000","last_unstale":"2024-02-19T16:22:22.351817+0000","last_undegraded":"2024-02-19T16:22:22.351817+0000","last_fullsized":"2024-02-19T16:22:22.351817+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:48:45.580976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449511+0000","last_change":"2024-02-19T16:20:21.659094+0000","last_active":"2024-02-19T16:22:22.449511+0000","last_peered":"2024-02-19T16:22:22.449511+0000","last_clean":"2024-02-19T16:22:22.449511+0000","last_became_active":"2024-02-19T16:20:21.658726+0000","last_became_peered":"2024-02-19T16:20:21.658726+0000","last_unstale":"2024-02-19T16:22:22.449511+0000","last_undegraded":"2024-02-19T16:22:22.449511+0000","last_fullsized":"2024-02-19T16:22:22.449511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:44:21.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349008+0000","last_change":"2024-02-19T16:20:25.757907+0000","last_active":"2024-02-19T16:22:22.349008+0000","last_peered":"2024-02-19T16:22:22.349008+0000","last_clean":"2024-02-19T16:22:22.349008+0000","last_became_active":"2024-02-19T16:20:25.757434+0000","last_became_peered":"2024-02-19T16:20:25.757434+0000","last_unstale":"2024-02-19T16:22:22.349008+0000","last_undegraded":"2024-02-19T16:22:22.349008+0000","last_fullsized":"2024-02-19T16:22:22.349008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:48:29.799204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351467+0000","last_change":"2024-02-19T16:20:20.639267+0000","last_active":"2024-02-19T16:22:22.351467+0000","last_peered":"2024-02-19T16:22:22.351467+0000","last_clean":"2024-02-19T16:22:22.351467+0000","last_became_active":"2024-02-19T16:20:20.638773+0000","last_became_peered":"2024-02-19T16:20:20.638773+0000","last_unstale":"2024-02-19T16:22:22.351467+0000","last_undegraded":"2024-02-19T16:22:22.351467+0000","last_fullsized":"2024-02-19T16:22:22.351467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:58:20.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354355+0000","last_change":"2024-02-19T16:20:21.671626+0000","last_active":"2024-02-19T16:22:22.354355+0000","last_peered":"2024-02-19T16:22:22.354355+0000","last_clean":"2024-02-19T16:22:22.354355+0000","last_became_active":"2024-02-19T16:20:21.670970+0000","last_became_peered":"2024-02-19T16:20:21.670970+0000","last_unstale":"2024-02-19T16:22:22.354355+0000","last_undegraded":"2024-02-19T16:22:22.354355+0000","last_fullsized":"2024-02-19T16:22:22.354355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:39:45.786174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353349+0000","last_change":"2024-02-19T16:20:25.753879+0000","last_active":"2024-02-19T16:22:22.353349+0000","last_peered":"2024-02-19T16:22:22.353349+0000","last_clean":"2024-02-19T16:22:22.353349+0000","last_became_active":"2024-02-19T16:20:25.753511+0000","last_became_peered":"2024-02-19T16:20:25.753511+0000","last_unstale":"2024-02-19T16:22:22.353349+0000","last_undegraded":"2024-02-19T16:22:22.353349+0000","last_fullsized":"2024-02-19T16:22:22.353349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:45:06.922763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":159,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:38.445988+0000","last_change":"2024-02-19T16:20:20.634336+0000","last_active":"2024-02-19T16:22:38.445988+0000","last_peered":"2024-02-19T16:22:38.445988+0000","last_clean":"2024-02-19T16:22:38.445988+0000","last_became_active":"2024-02-19T16:20:20.634019+0000","last_became_peered":"2024-02-19T16:20:20.634019+0000","last_unstale":"2024-02-19T16:22:38.445988+0000","last_undegraded":"2024-02-19T16:22:38.445988+0000","last_fullsized":"2024-02-19T16:22:38.445988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:36:19.277389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351396+0000","last_change":"2024-02-19T16:20:21.666905+0000","last_active":"2024-02-19T16:22:22.351396+0000","last_peered":"2024-02-19T16:22:22.351396+0000","last_clean":"2024-02-19T16:22:22.351396+0000","last_became_active":"2024-02-19T16:20:21.666596+0000","last_became_peered":"2024-02-19T16:20:21.666596+0000","last_unstale":"2024-02-19T16:22:22.351396+0000","last_undegraded":"2024-02-19T16:22:22.351396+0000","last_fullsized":"2024-02-19T16:22:22.351396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:25:06.571979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352924+0000","last_change":"2024-02-19T16:21:21.885536+0000","last_active":"2024-02-19T16:22:22.352924+0000","last_peered":"2024-02-19T16:22:22.352924+0000","last_clean":"2024-02-19T16:22:22.352924+0000","last_became_active":"2024-02-19T16:21:21.885125+0000","last_became_peered":"2024-02-19T16:21:21.885125+0000","last_unstale":"2024-02-19T16:22:22.352924+0000","last_undegraded":"2024-02-19T16:22:22.352924+0000","last_fullsized":"2024-02-19T16:22:22.352924+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:23:23.180070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428691+0000","last_change":"2024-02-19T16:21:22.344872+0000","last_active":"2024-02-19T16:22:22.428691+0000","last_peered":"2024-02-19T16:22:22.428691+0000","last_clean":"2024-02-19T16:22:22.428691+0000","last_became_active":"2024-02-19T16:21:22.342317+0000","last_became_peered":"2024-02-19T16:21:22.342317+0000","last_unstale":"2024-02-19T16:22:22.428691+0000","last_undegraded":"2024-02-19T16:22:22.428691+0000","last_fullsized":"2024-02-19T16:22:22.428691+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:53:45.080091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354273+0000","last_change":"2024-02-19T16:20:21.668648+0000","last_active":"2024-02-19T16:22:22.354273+0000","last_peered":"2024-02-19T16:22:22.354273+0000","last_clean":"2024-02-19T16:22:22.354273+0000","last_became_active":"2024-02-19T16:20:21.668327+0000","last_became_peered":"2024-02-19T16:20:21.668327+0000","last_unstale":"2024-02-19T16:22:22.354273+0000","last_undegraded":"2024-02-19T16:22:22.354273+0000","last_fullsized":"2024-02-19T16:22:22.354273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:33:51.907661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449894+0000","last_change":"2024-02-19T16:20:25.757753+0000","last_active":"2024-02-19T16:22:22.449894+0000","last_peered":"2024-02-19T16:22:22.449894+0000","last_clean":"2024-02-19T16:22:22.449894+0000","last_became_active":"2024-02-19T16:20:25.757393+0000","last_became_peered":"2024-02-19T16:20:25.757393+0000","last_unstale":"2024-02-19T16:22:22.449894+0000","last_undegraded":"2024-02-19T16:22:22.449894+0000","last_fullsized":"2024-02-19T16:22:22.449894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:28.464306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353391+0000","last_change":"2024-02-19T16:20:20.634669+0000","last_active":"2024-02-19T16:22:22.353391+0000","last_peered":"2024-02-19T16:22:22.353391+0000","last_clean":"2024-02-19T16:22:22.353391+0000","last_became_active":"2024-02-19T16:20:20.634329+0000","last_became_peered":"2024-02-19T16:20:20.634329+0000","last_unstale":"2024-02-19T16:22:22.353391+0000","last_undegraded":"2024-02-19T16:22:22.353391+0000","last_fullsized":"2024-02-19T16:22:22.353391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:02:08.665640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349154+0000","last_change":"2024-02-19T16:20:21.663072+0000","last_active":"2024-02-19T16:22:22.349154+0000","last_peered":"2024-02-19T16:22:22.349154+0000","last_clean":"2024-02-19T16:22:22.349154+0000","last_became_active":"2024-02-19T16:20:21.662747+0000","last_became_peered":"2024-02-19T16:20:21.662747+0000","last_unstale":"2024-02-19T16:22:22.349154+0000","last_undegraded":"2024-02-19T16:22:22.349154+0000","last_fullsized":"2024-02-19T16:22:22.349154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:05:57.754935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450308+0000","last_change":"2024-02-19T16:20:25.756926+0000","last_active":"2024-02-19T16:22:22.450308+0000","last_peered":"2024-02-19T16:22:22.450308+0000","last_clean":"2024-02-19T16:22:22.450308+0000","last_became_active":"2024-02-19T16:20:25.756459+0000","last_became_peered":"2024-02-19T16:20:25.756459+0000","last_unstale":"2024-02-19T16:22:22.450308+0000","last_undegraded":"2024-02-19T16:22:22.450308+0000","last_fullsized":"2024-02-19T16:22:22.450308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:55.975509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353726+0000","last_change":"2024-02-19T16:20:20.635487+0000","last_active":"2024-02-19T16:22:22.353726+0000","last_peered":"2024-02-19T16:22:22.353726+0000","last_clean":"2024-02-19T16:22:22.353726+0000","last_became_active":"2024-02-19T16:20:20.635202+0000","last_became_peered":"2024-02-19T16:20:20.635202+0000","last_unstale":"2024-02-19T16:22:22.353726+0000","last_undegraded":"2024-02-19T16:22:22.353726+0000","last_fullsized":"2024-02-19T16:22:22.353726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:28:09.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350686+0000","last_change":"2024-02-19T16:20:21.660714+0000","last_active":"2024-02-19T16:22:22.350686+0000","last_peered":"2024-02-19T16:22:22.350686+0000","last_clean":"2024-02-19T16:22:22.350686+0000","last_became_active":"2024-02-19T16:20:21.660388+0000","last_became_peered":"2024-02-19T16:20:21.660388+0000","last_unstale":"2024-02-19T16:22:22.350686+0000","last_undegraded":"2024-02-19T16:22:22.350686+0000","last_fullsized":"2024-02-19T16:22:22.350686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:31:41.958717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349021+0000","last_change":"2024-02-19T16:20:25.765352+0000","last_active":"2024-02-19T16:22:22.349021+0000","last_peered":"2024-02-19T16:22:22.349021+0000","last_clean":"2024-02-19T16:22:22.349021+0000","last_became_active":"2024-02-19T16:20:25.764869+0000","last_became_peered":"2024-02-19T16:20:25.764869+0000","last_unstale":"2024-02-19T16:22:22.349021+0000","last_undegraded":"2024-02-19T16:22:22.349021+0000","last_fullsized":"2024-02-19T16:22:22.349021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:12:36.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:37.444275+0000","last_change":"2024-02-19T16:20:20.637537+0000","last_active":"2024-02-19T16:22:37.444275+0000","last_peered":"2024-02-19T16:22:37.444275+0000","last_clean":"2024-02-19T16:22:37.444275+0000","last_became_active":"2024-02-19T16:20:20.637233+0000","last_became_peered":"2024-02-19T16:20:20.637233+0000","last_unstale":"2024-02-19T16:22:37.444275+0000","last_undegraded":"2024-02-19T16:22:37.444275+0000","last_fullsized":"2024-02-19T16:22:37.444275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:27:26.606671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449813+0000","last_change":"2024-02-19T16:20:21.662351+0000","last_active":"2024-02-19T16:22:22.449813+0000","last_peered":"2024-02-19T16:22:22.449813+0000","last_clean":"2024-02-19T16:22:22.449813+0000","last_became_active":"2024-02-19T16:20:21.661835+0000","last_became_peered":"2024-02-19T16:20:21.661835+0000","last_unstale":"2024-02-19T16:22:22.449813+0000","last_undegraded":"2024-02-19T16:22:22.449813+0000","last_fullsized":"2024-02-19T16:22:22.449813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:20:40.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351146+0000","last_change":"2024-02-19T16:20:25.753434+0000","last_active":"2024-02-19T16:22:22.351146+0000","last_peered":"2024-02-19T16:22:22.351146+0000","last_clean":"2024-02-19T16:22:22.351146+0000","last_became_active":"2024-02-19T16:20:25.753000+0000","last_became_peered":"2024-02-19T16:20:25.753000+0000","last_unstale":"2024-02-19T16:22:22.351146+0000","last_undegraded":"2024-02-19T16:22:22.351146+0000","last_fullsized":"2024-02-19T16:22:22.351146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:01:35.519691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351218+0000","last_change":"2024-02-19T16:20:20.635545+0000","last_active":"2024-02-19T16:22:22.351218+0000","last_peered":"2024-02-19T16:22:22.351218+0000","last_clean":"2024-02-19T16:22:22.351218+0000","last_became_active":"2024-02-19T16:20:20.635142+0000","last_became_peered":"2024-02-19T16:20:20.635142+0000","last_unstale":"2024-02-19T16:22:22.351218+0000","last_undegraded":"2024-02-19T16:22:22.351218+0000","last_fullsized":"2024-02-19T16:22:22.351218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:40:58.904909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450461+0000","last_change":"2024-02-19T16:20:21.667535+0000","last_active":"2024-02-19T16:22:22.450461+0000","last_peered":"2024-02-19T16:22:22.450461+0000","last_clean":"2024-02-19T16:22:22.450461+0000","last_became_active":"2024-02-19T16:20:21.665968+0000","last_became_peered":"2024-02-19T16:20:21.665968+0000","last_unstale":"2024-02-19T16:22:22.450461+0000","last_undegraded":"2024-02-19T16:22:22.450461+0000","last_fullsized":"2024-02-19T16:22:22.450461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:01:07.929376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349162+0000","last_change":"2024-02-19T16:20:25.753725+0000","last_active":"2024-02-19T16:22:22.349162+0000","last_peered":"2024-02-19T16:22:22.349162+0000","last_clean":"2024-02-19T16:22:22.349162+0000","last_became_active":"2024-02-19T16:20:25.753260+0000","last_became_peered":"2024-02-19T16:20:25.753260+0000","last_unstale":"2024-02-19T16:22:22.349162+0000","last_undegraded":"2024-02-19T16:22:22.349162+0000","last_fullsized":"2024-02-19T16:22:22.349162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:11:14.623007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354277+0000","last_change":"2024-02-19T16:20:20.642097+0000","last_active":"2024-02-19T16:22:22.354277+0000","last_peered":"2024-02-19T16:22:22.354277+0000","last_clean":"2024-02-19T16:22:22.354277+0000","last_became_active":"2024-02-19T16:20:20.641718+0000","last_became_peered":"2024-02-19T16:20:20.641718+0000","last_unstale":"2024-02-19T16:22:22.354277+0000","last_undegraded":"2024-02-19T16:22:22.354277+0000","last_fullsized":"2024-02-19T16:22:22.354277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:11:08.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449706+0000","last_change":"2024-02-19T16:20:21.659390+0000","last_active":"2024-02-19T16:22:22.449706+0000","last_peered":"2024-02-19T16:22:22.449706+0000","last_clean":"2024-02-19T16:22:22.449706+0000","last_became_active":"2024-02-19T16:20:21.658151+0000","last_became_peered":"2024-02-19T16:20:21.658151+0000","last_unstale":"2024-02-19T16:22:22.449706+0000","last_undegraded":"2024-02-19T16:22:22.449706+0000","last_fullsized":"2024-02-19T16:22:22.449706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:32:20.550367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428529+0000","last_change":"2024-02-19T16:21:21.885002+0000","last_active":"2024-02-19T16:22:22.428529+0000","last_peered":"2024-02-19T16:22:22.428529+0000","last_clean":"2024-02-19T16:22:22.428529+0000","last_became_active":"2024-02-19T16:21:21.884557+0000","last_became_peered":"2024-02-19T16:21:21.884557+0000","last_unstale":"2024-02-19T16:22:22.428529+0000","last_undegraded":"2024-02-19T16:22:22.428529+0000","last_fullsized":"2024-02-19T16:22:22.428529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:08:31.859577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'37","reported_seq":315,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:38.445992+0000","last_change":"2024-02-19T16:20:20.633025+0000","last_active":"2024-02-19T16:22:38.445992+0000","last_peered":"2024-02-19T16:22:38.445992+0000","last_clean":"2024-02-19T16:22:38.445992+0000","last_became_active":"2024-02-19T16:20:20.632693+0000","last_became_peered":"2024-02-19T16:20:20.632693+0000","last_unstale":"2024-02-19T16:22:38.445992+0000","last_undegraded":"2024-02-19T16:22:38.445992+0000","last_fullsized":"2024-02-19T16:22:38.445992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:08:33.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68096,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":190,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351523+0000","last_change":"2024-02-19T16:20:21.669226+0000","last_active":"2024-02-19T16:22:22.351523+0000","last_peered":"2024-02-19T16:22:22.351523+0000","last_clean":"2024-02-19T16:22:22.351523+0000","last_became_active":"2024-02-19T16:20:21.668866+0000","last_became_peered":"2024-02-19T16:20:21.668866+0000","last_unstale":"2024-02-19T16:22:22.351523+0000","last_undegraded":"2024-02-19T16:22:22.351523+0000","last_fullsized":"2024-02-19T16:22:22.351523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:51:15.531251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354566+0000","last_change":"2024-02-19T16:20:25.765009+0000","last_active":"2024-02-19T16:22:22.354566+0000","last_peered":"2024-02-19T16:22:22.354566+0000","last_clean":"2024-02-19T16:22:22.354566+0000","last_became_active":"2024-02-19T16:20:25.764600+0000","last_became_peered":"2024-02-19T16:20:25.764600+0000","last_unstale":"2024-02-19T16:22:22.354566+0000","last_undegraded":"2024-02-19T16:22:22.354566+0000","last_fullsized":"2024-02-19T16:22:22.354566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:51:40.529365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352980+0000","last_change":"2024-02-19T16:20:20.636880+0000","last_active":"2024-02-19T16:22:22.352980+0000","last_peered":"2024-02-19T16:22:22.352980+0000","last_clean":"2024-02-19T16:22:22.352980+0000","last_became_active":"2024-02-19T16:20:20.636564+0000","last_became_peered":"2024-02-19T16:20:20.636564+0000","last_unstale":"2024-02-19T16:22:22.352980+0000","last_undegraded":"2024-02-19T16:22:22.352980+0000","last_fullsized":"2024-02-19T16:22:22.352980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:09:12.765732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354094+0000","last_change":"2024-02-19T16:20:21.657295+0000","last_active":"2024-02-19T16:22:22.354094+0000","last_peered":"2024-02-19T16:22:22.354094+0000","last_clean":"2024-02-19T16:22:22.354094+0000","last_became_active":"2024-02-19T16:20:21.656977+0000","last_became_peered":"2024-02-19T16:20:21.656977+0000","last_unstale":"2024-02-19T16:22:22.354094+0000","last_undegraded":"2024-02-19T16:22:22.354094+0000","last_fullsized":"2024-02-19T16:22:22.354094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:03.520947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449654+0000","last_change":"2024-02-19T16:20:25.764095+0000","last_active":"2024-02-19T16:22:22.449654+0000","last_peered":"2024-02-19T16:22:22.449654+0000","last_clean":"2024-02-19T16:22:22.449654+0000","last_became_active":"2024-02-19T16:20:25.763650+0000","last_became_peered":"2024-02-19T16:20:25.763650+0000","last_unstale":"2024-02-19T16:22:22.449654+0000","last_undegraded":"2024-02-19T16:22:22.449654+0000","last_fullsized":"2024-02-19T16:22:22.449654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:52:14.733674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354252+0000","last_change":"2024-02-19T16:20:20.640615+0000","last_active":"2024-02-19T16:22:22.354252+0000","last_peered":"2024-02-19T16:22:22.354252+0000","last_clean":"2024-02-19T16:22:22.354252+0000","last_became_active":"2024-02-19T16:20:20.640190+0000","last_became_peered":"2024-02-19T16:20:20.640190+0000","last_unstale":"2024-02-19T16:22:22.354252+0000","last_undegraded":"2024-02-19T16:22:22.354252+0000","last_fullsized":"2024-02-19T16:22:22.354252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:38:10.798207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355544+0000","last_change":"2024-02-19T16:22:22.355544+0000","last_active":"2024-02-19T16:22:22.355544+0000","last_peered":"2024-02-19T16:22:22.355544+0000","last_clean":"2024-02-19T16:22:22.355544+0000","last_became_active":"2024-02-19T16:22:22.355184+0000","last_became_peered":"2024-02-19T16:22:22.355184+0000","last_unstale":"2024-02-19T16:22:22.355544+0000","last_undegraded":"2024-02-19T16:22:22.355544+0000","last_fullsized":"2024-02-19T16:22:22.355544+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:48:26.236286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351943+0000","last_change":"2024-02-19T16:21:22.216357+0000","last_active":"2024-02-19T16:22:22.351943+0000","last_peered":"2024-02-19T16:22:22.351943+0000","last_clean":"2024-02-19T16:22:22.351943+0000","last_became_active":"2024-02-19T16:21:22.215883+0000","last_became_peered":"2024-02-19T16:21:22.215883+0000","last_unstale":"2024-02-19T16:22:22.351943+0000","last_undegraded":"2024-02-19T16:22:22.351943+0000","last_fullsized":"2024-02-19T16:22:22.351943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:32:43.235515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":157,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:34.460388+0000","last_change":"2024-02-19T16:20:20.632865+0000","last_active":"2024-02-19T16:22:34.460388+0000","last_peered":"2024-02-19T16:22:34.460388+0000","last_clean":"2024-02-19T16:22:34.460388+0000","last_became_active":"2024-02-19T16:20:20.632521+0000","last_became_peered":"2024-02-19T16:20:20.632521+0000","last_unstale":"2024-02-19T16:22:34.460388+0000","last_undegraded":"2024-02-19T16:22:34.460388+0000","last_fullsized":"2024-02-19T16:22:34.460388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:36:27.252091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350839+0000","last_change":"2024-02-19T16:20:21.668598+0000","last_active":"2024-02-19T16:22:22.350839+0000","last_peered":"2024-02-19T16:22:22.350839+0000","last_clean":"2024-02-19T16:22:22.350839+0000","last_became_active":"2024-02-19T16:20:21.668270+0000","last_became_peered":"2024-02-19T16:20:21.668270+0000","last_unstale":"2024-02-19T16:22:22.350839+0000","last_undegraded":"2024-02-19T16:22:22.350839+0000","last_fullsized":"2024-02-19T16:22:22.350839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:47:27.247910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354823+0000","last_change":"2024-02-19T16:20:25.757915+0000","last_active":"2024-02-19T16:22:22.354823+0000","last_peered":"2024-02-19T16:22:22.354823+0000","last_clean":"2024-02-19T16:22:22.354823+0000","last_became_active":"2024-02-19T16:20:25.757486+0000","last_became_peered":"2024-02-19T16:20:25.757486+0000","last_unstale":"2024-02-19T16:22:22.354823+0000","last_undegraded":"2024-02-19T16:22:22.354823+0000","last_fullsized":"2024-02-19T16:22:22.354823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:46:18.823152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351234+0000","last_change":"2024-02-19T16:20:20.639027+0000","last_active":"2024-02-19T16:22:22.351234+0000","last_peered":"2024-02-19T16:22:22.351234+0000","last_clean":"2024-02-19T16:22:22.351234+0000","last_became_active":"2024-02-19T16:20:20.638358+0000","last_became_peered":"2024-02-19T16:20:20.638358+0000","last_unstale":"2024-02-19T16:22:22.351234+0000","last_undegraded":"2024-02-19T16:22:22.351234+0000","last_fullsized":"2024-02-19T16:22:22.351234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:53:34.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353983+0000","last_change":"2024-02-19T16:20:21.669068+0000","last_active":"2024-02-19T16:22:22.353983+0000","last_peered":"2024-02-19T16:22:22.353983+0000","last_clean":"2024-02-19T16:22:22.353983+0000","last_became_active":"2024-02-19T16:20:21.668520+0000","last_became_peered":"2024-02-19T16:20:21.668520+0000","last_unstale":"2024-02-19T16:22:22.353983+0000","last_undegraded":"2024-02-19T16:22:22.353983+0000","last_fullsized":"2024-02-19T16:22:22.353983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:11:09.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353999+0000","last_change":"2024-02-19T16:20:25.765170+0000","last_active":"2024-02-19T16:22:22.353999+0000","last_peered":"2024-02-19T16:22:22.353999+0000","last_clean":"2024-02-19T16:22:22.353999+0000","last_became_active":"2024-02-19T16:20:25.764720+0000","last_became_peered":"2024-02-19T16:20:25.764720+0000","last_unstale":"2024-02-19T16:22:22.353999+0000","last_undegraded":"2024-02-19T16:22:22.353999+0000","last_fullsized":"2024-02-19T16:22:22.353999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:18:53.770465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352102+0000","last_change":"2024-02-19T16:20:20.637709+0000","last_active":"2024-02-19T16:22:22.352102+0000","last_peered":"2024-02-19T16:22:22.352102+0000","last_clean":"2024-02-19T16:22:22.352102+0000","last_became_active":"2024-02-19T16:20:20.637412+0000","last_became_peered":"2024-02-19T16:20:20.637412+0000","last_unstale":"2024-02-19T16:22:22.352102+0000","last_undegraded":"2024-02-19T16:22:22.352102+0000","last_fullsized":"2024-02-19T16:22:22.352102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:07.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449849+0000","last_change":"2024-02-19T16:20:21.663613+0000","last_active":"2024-02-19T16:22:22.449849+0000","last_peered":"2024-02-19T16:22:22.449849+0000","last_clean":"2024-02-19T16:22:22.449849+0000","last_became_active":"2024-02-19T16:20:21.663143+0000","last_became_peered":"2024-02-19T16:20:21.663143+0000","last_unstale":"2024-02-19T16:22:22.449849+0000","last_undegraded":"2024-02-19T16:22:22.449849+0000","last_fullsized":"2024-02-19T16:22:22.449849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:56:27.719038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351345+0000","last_change":"2024-02-19T16:20:26.383207+0000","last_active":"2024-02-19T16:22:22.351345+0000","last_peered":"2024-02-19T16:22:22.351345+0000","last_clean":"2024-02-19T16:22:22.351345+0000","last_became_active":"2024-02-19T16:20:26.382625+0000","last_became_peered":"2024-02-19T16:20:26.382625+0000","last_unstale":"2024-02-19T16:22:22.351345+0000","last_undegraded":"2024-02-19T16:22:22.351345+0000","last_fullsized":"2024-02-19T16:22:22.351345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:19:33.247031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351655+0000","last_change":"2024-02-19T16:20:20.633340+0000","last_active":"2024-02-19T16:22:22.351655+0000","last_peered":"2024-02-19T16:22:22.351655+0000","last_clean":"2024-02-19T16:22:22.351655+0000","last_became_active":"2024-02-19T16:20:20.633015+0000","last_became_peered":"2024-02-19T16:20:20.633015+0000","last_unstale":"2024-02-19T16:22:22.351655+0000","last_undegraded":"2024-02-19T16:22:22.351655+0000","last_fullsized":"2024-02-19T16:22:22.351655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:25:00.453222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428974+0000","last_change":"2024-02-19T16:20:21.670377+0000","last_active":"2024-02-19T16:22:22.428974+0000","last_peered":"2024-02-19T16:22:22.428974+0000","last_clean":"2024-02-19T16:22:22.428974+0000","last_became_active":"2024-02-19T16:20:21.669883+0000","last_became_peered":"2024-02-19T16:20:21.669883+0000","last_unstale":"2024-02-19T16:22:22.428974+0000","last_undegraded":"2024-02-19T16:22:22.428974+0000","last_fullsized":"2024-02-19T16:22:22.428974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:45:55.775817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354592+0000","last_change":"2024-02-19T16:20:25.762093+0000","last_active":"2024-02-19T16:22:22.354592+0000","last_peered":"2024-02-19T16:22:22.354592+0000","last_clean":"2024-02-19T16:22:22.354592+0000","last_became_active":"2024-02-19T16:20:25.761701+0000","last_became_peered":"2024-02-19T16:20:25.761701+0000","last_unstale":"2024-02-19T16:22:22.354592+0000","last_undegraded":"2024-02-19T16:22:22.354592+0000","last_fullsized":"2024-02-19T16:22:22.354592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:44:14.806798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350901+0000","last_change":"2024-02-19T16:20:20.639731+0000","last_active":"2024-02-19T16:22:22.350901+0000","last_peered":"2024-02-19T16:22:22.350901+0000","last_clean":"2024-02-19T16:22:22.350901+0000","last_became_active":"2024-02-19T16:20:20.639414+0000","last_became_peered":"2024-02-19T16:20:20.639414+0000","last_unstale":"2024-02-19T16:22:22.350901+0000","last_undegraded":"2024-02-19T16:22:22.350901+0000","last_fullsized":"2024-02-19T16:22:22.350901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:13:58.490503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.428794+0000","last_change":"2024-02-19T16:20:21.659981+0000","last_active":"2024-02-19T16:22:22.428794+0000","last_peered":"2024-02-19T16:22:22.428794+0000","last_clean":"2024-02-19T16:22:22.428794+0000","last_became_active":"2024-02-19T16:20:21.659553+0000","last_became_peered":"2024-02-19T16:20:21.659553+0000","last_unstale":"2024-02-19T16:22:22.428794+0000","last_undegraded":"2024-02-19T16:22:22.428794+0000","last_fullsized":"2024-02-19T16:22:22.428794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:38:40.325981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.355307+0000","last_change":"2024-02-19T16:20:26.381608+0000","last_active":"2024-02-19T16:22:22.355307+0000","last_peered":"2024-02-19T16:22:22.355307+0000","last_clean":"2024-02-19T16:22:22.355307+0000","last_became_active":"2024-02-19T16:20:26.381166+0000","last_became_peered":"2024-02-19T16:20:26.381166+0000","last_unstale":"2024-02-19T16:22:22.355307+0000","last_undegraded":"2024-02-19T16:22:22.355307+0000","last_fullsized":"2024-02-19T16:22:22.355307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:34:55.503486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353828+0000","last_change":"2024-02-19T16:22:22.353828+0000","last_active":"2024-02-19T16:22:22.353828+0000","last_peered":"2024-02-19T16:22:22.353828+0000","last_clean":"2024-02-19T16:22:22.353828+0000","last_became_active":"2024-02-19T16:22:22.353369+0000","last_became_peered":"2024-02-19T16:22:22.353369+0000","last_unstale":"2024-02-19T16:22:22.353828+0000","last_undegraded":"2024-02-19T16:22:22.353828+0000","last_fullsized":"2024-02-19T16:22:22.353828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:26:32.976202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":155,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:35.461056+0000","last_change":"2024-02-19T16:20:20.629396+0000","last_active":"2024-02-19T16:22:35.461056+0000","last_peered":"2024-02-19T16:22:35.461056+0000","last_clean":"2024-02-19T16:22:35.461056+0000","last_became_active":"2024-02-19T16:20:20.629035+0000","last_became_peered":"2024-02-19T16:20:20.629035+0000","last_unstale":"2024-02-19T16:22:35.461056+0000","last_undegraded":"2024-02-19T16:22:35.461056+0000","last_fullsized":"2024-02-19T16:22:35.461056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:46:40.845023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351122+0000","last_change":"2024-02-19T16:20:25.754831+0000","last_active":"2024-02-19T16:22:22.351122+0000","last_peered":"2024-02-19T16:22:22.351122+0000","last_clean":"2024-02-19T16:22:22.351122+0000","last_became_active":"2024-02-19T16:20:25.754400+0000","last_became_peered":"2024-02-19T16:20:25.754400+0000","last_unstale":"2024-02-19T16:22:22.351122+0000","last_undegraded":"2024-02-19T16:22:22.351122+0000","last_fullsized":"2024-02-19T16:22:22.351122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:37:43.168065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349313+0000","last_change":"2024-02-19T16:20:21.670067+0000","last_active":"2024-02-19T16:22:22.349313+0000","last_peered":"2024-02-19T16:22:22.349313+0000","last_clean":"2024-02-19T16:22:22.349313+0000","last_became_active":"2024-02-19T16:20:21.669748+0000","last_became_peered":"2024-02-19T16:20:21.669748+0000","last_unstale":"2024-02-19T16:22:22.349313+0000","last_undegraded":"2024-02-19T16:22:22.349313+0000","last_fullsized":"2024-02-19T16:22:22.349313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:04:11.532221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349176+0000","last_change":"2024-02-19T16:20:20.635994+0000","last_active":"2024-02-19T16:22:22.349176+0000","last_peered":"2024-02-19T16:22:22.349176+0000","last_clean":"2024-02-19T16:22:22.349176+0000","last_became_active":"2024-02-19T16:20:20.635667+0000","last_became_peered":"2024-02-19T16:20:20.635667+0000","last_unstale":"2024-02-19T16:22:22.349176+0000","last_undegraded":"2024-02-19T16:22:22.349176+0000","last_fullsized":"2024-02-19T16:22:22.349176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:55.565759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350666+0000","last_change":"2024-02-19T16:19:11.980608+0000","last_active":"2024-02-19T16:22:22.350666+0000","last_peered":"2024-02-19T16:22:22.350666+0000","last_clean":"2024-02-19T16:22:22.350666+0000","last_became_active":"2024-02-19T16:19:11.980108+0000","last_became_peered":"2024-02-19T16:19:11.980108+0000","last_unstale":"2024-02-19T16:22:22.350666+0000","last_undegraded":"2024-02-19T16:22:22.350666+0000","last_fullsized":"2024-02-19T16:22:22.350666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:15:47.570086+0000","last_clean_scrub_stamp":"2024-02-19T16:15:47.570086+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:01.739400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354508+0000","last_change":"2024-02-19T16:20:26.382796+0000","last_active":"2024-02-19T16:22:22.354508+0000","last_peered":"2024-02-19T16:22:22.354508+0000","last_clean":"2024-02-19T16:22:22.354508+0000","last_became_active":"2024-02-19T16:20:26.381629+0000","last_became_peered":"2024-02-19T16:20:26.381629+0000","last_unstale":"2024-02-19T16:22:22.354508+0000","last_undegraded":"2024-02-19T16:22:22.354508+0000","last_fullsized":"2024-02-19T16:22:22.354508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:43:06.520886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353185+0000","last_change":"2024-02-19T16:20:21.656574+0000","last_active":"2024-02-19T16:22:22.353185+0000","last_peered":"2024-02-19T16:22:22.353185+0000","last_clean":"2024-02-19T16:22:22.353185+0000","last_became_active":"2024-02-19T16:20:21.656140+0000","last_became_peered":"2024-02-19T16:20:21.656140+0000","last_unstale":"2024-02-19T16:22:22.353185+0000","last_undegraded":"2024-02-19T16:22:22.353185+0000","last_fullsized":"2024-02-19T16:22:22.353185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:08:16.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.350730+0000","last_change":"2024-02-19T16:20:20.638592+0000","last_active":"2024-02-19T16:22:22.350730+0000","last_peered":"2024-02-19T16:22:22.350730+0000","last_clean":"2024-02-19T16:22:22.350730+0000","last_became_active":"2024-02-19T16:20:20.638283+0000","last_became_peered":"2024-02-19T16:20:20.638283+0000","last_unstale":"2024-02-19T16:22:22.350730+0000","last_undegraded":"2024-02-19T16:22:22.350730+0000","last_fullsized":"2024-02-19T16:22:22.350730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.858737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449675+0000","last_change":"2024-02-19T16:20:25.757702+0000","last_active":"2024-02-19T16:22:22.449675+0000","last_peered":"2024-02-19T16:22:22.449675+0000","last_clean":"2024-02-19T16:22:22.449675+0000","last_became_active":"2024-02-19T16:20:25.757317+0000","last_became_peered":"2024-02-19T16:20:25.757317+0000","last_unstale":"2024-02-19T16:22:22.449675+0000","last_undegraded":"2024-02-19T16:22:22.449675+0000","last_fullsized":"2024-02-19T16:22:22.449675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:15:30.486142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351689+0000","last_change":"2024-02-19T16:20:20.638258+0000","last_active":"2024-02-19T16:22:22.351689+0000","last_peered":"2024-02-19T16:22:22.351689+0000","last_clean":"2024-02-19T16:22:22.351689+0000","last_became_active":"2024-02-19T16:20:20.637921+0000","last_became_peered":"2024-02-19T16:20:20.637921+0000","last_unstale":"2024-02-19T16:22:22.351689+0000","last_undegraded":"2024-02-19T16:22:22.351689+0000","last_fullsized":"2024-02-19T16:22:22.351689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:05:54.556660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354591+0000","last_change":"2024-02-19T16:20:21.650864+0000","last_active":"2024-02-19T16:22:22.354591+0000","last_peered":"2024-02-19T16:22:22.354591+0000","last_clean":"2024-02-19T16:22:22.354591+0000","last_became_active":"2024-02-19T16:20:21.650584+0000","last_became_peered":"2024-02-19T16:20:21.650584+0000","last_unstale":"2024-02-19T16:22:22.354591+0000","last_undegraded":"2024-02-19T16:22:22.354591+0000","last_fullsized":"2024-02-19T16:22:22.354591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:08:38.774879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.348967+0000","last_change":"2024-02-19T16:20:26.381672+0000","last_active":"2024-02-19T16:22:22.348967+0000","last_peered":"2024-02-19T16:22:22.348967+0000","last_clean":"2024-02-19T16:22:22.348967+0000","last_became_active":"2024-02-19T16:20:26.381120+0000","last_became_peered":"2024-02-19T16:20:26.381120+0000","last_unstale":"2024-02-19T16:22:22.348967+0000","last_undegraded":"2024-02-19T16:22:22.348967+0000","last_fullsized":"2024-02-19T16:22:22.348967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:21:18.277980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:32.339521+0000","last_change":"2024-02-19T16:20:20.641154+0000","last_active":"2024-02-19T16:22:32.339521+0000","last_peered":"2024-02-19T16:22:32.339521+0000","last_clean":"2024-02-19T16:22:32.339521+0000","last_became_active":"2024-02-19T16:20:20.640820+0000","last_became_peered":"2024-02-19T16:20:20.640820+0000","last_unstale":"2024-02-19T16:22:32.339521+0000","last_undegraded":"2024-02-19T16:22:32.339521+0000","last_fullsized":"2024-02-19T16:22:32.339521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:41:40.506367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349548+0000","last_change":"2024-02-19T16:20:21.663201+0000","last_active":"2024-02-19T16:22:22.349548+0000","last_peered":"2024-02-19T16:22:22.349548+0000","last_clean":"2024-02-19T16:22:22.349548+0000","last_became_active":"2024-02-19T16:20:21.662838+0000","last_became_peered":"2024-02-19T16:20:21.662838+0000","last_unstale":"2024-02-19T16:22:22.349548+0000","last_undegraded":"2024-02-19T16:22:22.349548+0000","last_fullsized":"2024-02-19T16:22:22.349548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:43:56.866291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352447+0000","last_change":"2024-02-19T16:20:25.746888+0000","last_active":"2024-02-19T16:22:22.352447+0000","last_peered":"2024-02-19T16:22:22.352447+0000","last_clean":"2024-02-19T16:22:22.352447+0000","last_became_active":"2024-02-19T16:20:25.746406+0000","last_became_peered":"2024-02-19T16:20:25.746406+0000","last_unstale":"2024-02-19T16:22:22.352447+0000","last_undegraded":"2024-02-19T16:22:22.352447+0000","last_fullsized":"2024-02-19T16:22:22.352447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:21:50.039869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349728+0000","last_change":"2024-02-19T16:20:20.647927+0000","last_active":"2024-02-19T16:22:22.349728+0000","last_peered":"2024-02-19T16:22:22.349728+0000","last_clean":"2024-02-19T16:22:22.349728+0000","last_became_active":"2024-02-19T16:20:20.647624+0000","last_became_peered":"2024-02-19T16:20:20.647624+0000","last_unstale":"2024-02-19T16:22:22.349728+0000","last_undegraded":"2024-02-19T16:22:22.349728+0000","last_fullsized":"2024-02-19T16:22:22.349728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:21:53.679167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450488+0000","last_change":"2024-02-19T16:20:21.665537+0000","last_active":"2024-02-19T16:22:22.450488+0000","last_peered":"2024-02-19T16:22:22.450488+0000","last_clean":"2024-02-19T16:22:22.450488+0000","last_became_active":"2024-02-19T16:20:21.665221+0000","last_became_peered":"2024-02-19T16:20:21.665221+0000","last_unstale":"2024-02-19T16:22:22.450488+0000","last_undegraded":"2024-02-19T16:22:22.450488+0000","last_fullsized":"2024-02-19T16:22:22.450488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:44:54.761781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349405+0000","last_change":"2024-02-19T16:20:26.382419+0000","last_active":"2024-02-19T16:22:22.349405+0000","last_peered":"2024-02-19T16:22:22.349405+0000","last_clean":"2024-02-19T16:22:22.349405+0000","last_became_active":"2024-02-19T16:20:26.381979+0000","last_became_peered":"2024-02-19T16:20:26.381979+0000","last_unstale":"2024-02-19T16:22:22.349405+0000","last_undegraded":"2024-02-19T16:22:22.349405+0000","last_fullsized":"2024-02-19T16:22:22.349405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:28:52.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449451+0000","last_change":"2024-02-19T16:20:20.634309+0000","last_active":"2024-02-19T16:22:22.449451+0000","last_peered":"2024-02-19T16:22:22.449451+0000","last_clean":"2024-02-19T16:22:22.449451+0000","last_became_active":"2024-02-19T16:20:20.633994+0000","last_became_peered":"2024-02-19T16:20:20.633994+0000","last_unstale":"2024-02-19T16:22:22.449451+0000","last_undegraded":"2024-02-19T16:22:22.449451+0000","last_fullsized":"2024-02-19T16:22:22.449451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:49:05.044331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351520+0000","last_change":"2024-02-19T16:20:21.666223+0000","last_active":"2024-02-19T16:22:22.351520+0000","last_peered":"2024-02-19T16:22:22.351520+0000","last_clean":"2024-02-19T16:22:22.351520+0000","last_became_active":"2024-02-19T16:20:21.665897+0000","last_became_peered":"2024-02-19T16:20:21.665897+0000","last_unstale":"2024-02-19T16:22:22.351520+0000","last_undegraded":"2024-02-19T16:22:22.351520+0000","last_fullsized":"2024-02-19T16:22:22.351520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:09:20.017559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351365+0000","last_change":"2024-02-19T16:21:22.216182+0000","last_active":"2024-02-19T16:22:22.351365+0000","last_peered":"2024-02-19T16:22:22.351365+0000","last_clean":"2024-02-19T16:22:22.351365+0000","last_became_active":"2024-02-19T16:21:22.215705+0000","last_became_peered":"2024-02-19T16:21:22.215705+0000","last_unstale":"2024-02-19T16:22:22.351365+0000","last_undegraded":"2024-02-19T16:22:22.351365+0000","last_fullsized":"2024-02-19T16:22:22.351365+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:29:29.574422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349162+0000","last_change":"2024-02-19T16:20:20.633013+0000","last_active":"2024-02-19T16:22:22.349162+0000","last_peered":"2024-02-19T16:22:22.349162+0000","last_clean":"2024-02-19T16:22:22.349162+0000","last_became_active":"2024-02-19T16:20:20.632660+0000","last_became_peered":"2024-02-19T16:20:20.632660+0000","last_unstale":"2024-02-19T16:22:22.349162+0000","last_undegraded":"2024-02-19T16:22:22.349162+0000","last_fullsized":"2024-02-19T16:22:22.349162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:30:39.395324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449565+0000","last_change":"2024-02-19T16:20:21.666168+0000","last_active":"2024-02-19T16:22:22.449565+0000","last_peered":"2024-02-19T16:22:22.449565+0000","last_clean":"2024-02-19T16:22:22.449565+0000","last_became_active":"2024-02-19T16:20:21.665862+0000","last_became_peered":"2024-02-19T16:20:21.665862+0000","last_unstale":"2024-02-19T16:22:22.449565+0000","last_undegraded":"2024-02-19T16:22:22.449565+0000","last_fullsized":"2024-02-19T16:22:22.449565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:54:08.121856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349272+0000","last_change":"2024-02-19T16:20:25.755967+0000","last_active":"2024-02-19T16:22:22.349272+0000","last_peered":"2024-02-19T16:22:22.349272+0000","last_clean":"2024-02-19T16:22:22.349272+0000","last_became_active":"2024-02-19T16:20:25.755565+0000","last_became_peered":"2024-02-19T16:20:25.755565+0000","last_unstale":"2024-02-19T16:22:22.349272+0000","last_undegraded":"2024-02-19T16:22:22.349272+0000","last_fullsized":"2024-02-19T16:22:22.349272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:51:42.164534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353013+0000","last_change":"2024-02-19T16:20:20.641153+0000","last_active":"2024-02-19T16:22:22.353013+0000","last_peered":"2024-02-19T16:22:22.353013+0000","last_clean":"2024-02-19T16:22:22.353013+0000","last_became_active":"2024-02-19T16:20:20.640824+0000","last_became_peered":"2024-02-19T16:20:20.640824+0000","last_unstale":"2024-02-19T16:22:22.353013+0000","last_undegraded":"2024-02-19T16:22:22.353013+0000","last_fullsized":"2024-02-19T16:22:22.353013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T00:23:38.626197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349632+0000","last_change":"2024-02-19T16:20:21.659769+0000","last_active":"2024-02-19T16:22:22.349632+0000","last_peered":"2024-02-19T16:22:22.349632+0000","last_clean":"2024-02-19T16:22:22.349632+0000","last_became_active":"2024-02-19T16:20:21.659310+0000","last_became_peered":"2024-02-19T16:20:21.659310+0000","last_unstale":"2024-02-19T16:22:22.349632+0000","last_undegraded":"2024-02-19T16:22:22.349632+0000","last_fullsized":"2024-02-19T16:22:22.349632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:51:41.212406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349434+0000","last_change":"2024-02-19T16:20:26.381655+0000","last_active":"2024-02-19T16:22:22.349434+0000","last_peered":"2024-02-19T16:22:22.349434+0000","last_clean":"2024-02-19T16:22:22.349434+0000","last_became_active":"2024-02-19T16:20:26.381067+0000","last_became_peered":"2024-02-19T16:20:26.381067+0000","last_unstale":"2024-02-19T16:22:22.349434+0000","last_undegraded":"2024-02-19T16:22:22.349434+0000","last_fullsized":"2024-02-19T16:22:22.349434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:13:35.084503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449599+0000","last_change":"2024-02-19T16:21:21.894342+0000","last_active":"2024-02-19T16:22:22.449599+0000","last_peered":"2024-02-19T16:22:22.449599+0000","last_clean":"2024-02-19T16:22:22.449599+0000","last_became_active":"2024-02-19T16:21:21.885142+0000","last_became_peered":"2024-02-19T16:21:21.885142+0000","last_unstale":"2024-02-19T16:22:22.449599+0000","last_undegraded":"2024-02-19T16:22:22.449599+0000","last_fullsized":"2024-02-19T16:22:22.449599+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:19:30.846573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351023+0000","last_change":"2024-02-19T16:20:21.660310+0000","last_active":"2024-02-19T16:22:22.351023+0000","last_peered":"2024-02-19T16:22:22.351023+0000","last_clean":"2024-02-19T16:22:22.351023+0000","last_became_active":"2024-02-19T16:20:21.659798+0000","last_became_peered":"2024-02-19T16:20:21.659798+0000","last_unstale":"2024-02-19T16:22:22.351023+0000","last_undegraded":"2024-02-19T16:22:22.351023+0000","last_fullsized":"2024-02-19T16:22:22.351023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:09:44.418023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449534+0000","last_change":"2024-02-19T16:20:25.757739+0000","last_active":"2024-02-19T16:22:22.449534+0000","last_peered":"2024-02-19T16:22:22.449534+0000","last_clean":"2024-02-19T16:22:22.449534+0000","last_became_active":"2024-02-19T16:20:25.757312+0000","last_became_peered":"2024-02-19T16:20:25.757312+0000","last_unstale":"2024-02-19T16:22:22.449534+0000","last_undegraded":"2024-02-19T16:22:22.449534+0000","last_fullsized":"2024-02-19T16:22:22.449534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:09:05.306566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353273+0000","last_change":"2024-02-19T16:20:20.648139+0000","last_active":"2024-02-19T16:22:22.353273+0000","last_peered":"2024-02-19T16:22:22.353273+0000","last_clean":"2024-02-19T16:22:22.353273+0000","last_became_active":"2024-02-19T16:20:20.647783+0000","last_became_peered":"2024-02-19T16:20:20.647783+0000","last_unstale":"2024-02-19T16:22:22.353273+0000","last_undegraded":"2024-02-19T16:22:22.353273+0000","last_fullsized":"2024-02-19T16:22:22.353273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T18:17:43.632478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351249+0000","last_change":"2024-02-19T16:20:21.665515+0000","last_active":"2024-02-19T16:22:22.351249+0000","last_peered":"2024-02-19T16:22:22.351249+0000","last_clean":"2024-02-19T16:22:22.351249+0000","last_became_active":"2024-02-19T16:20:21.664982+0000","last_became_peered":"2024-02-19T16:20:21.664982+0000","last_unstale":"2024-02-19T16:22:22.351249+0000","last_undegraded":"2024-02-19T16:22:22.351249+0000","last_fullsized":"2024-02-19T16:22:22.351249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T23:47:04.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450177+0000","last_change":"2024-02-19T16:20:25.760379+0000","last_active":"2024-02-19T16:22:22.450177+0000","last_peered":"2024-02-19T16:22:22.450177+0000","last_clean":"2024-02-19T16:22:22.450177+0000","last_became_active":"2024-02-19T16:20:25.759197+0000","last_became_peered":"2024-02-19T16:20:25.759197+0000","last_unstale":"2024-02-19T16:22:22.450177+0000","last_undegraded":"2024-02-19T16:22:22.450177+0000","last_fullsized":"2024-02-19T16:22:22.450177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:58:42.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354439+0000","last_change":"2024-02-19T16:20:20.641357+0000","last_active":"2024-02-19T16:22:22.354439+0000","last_peered":"2024-02-19T16:22:22.354439+0000","last_clean":"2024-02-19T16:22:22.354439+0000","last_became_active":"2024-02-19T16:20:20.641036+0000","last_became_peered":"2024-02-19T16:20:20.641036+0000","last_unstale":"2024-02-19T16:22:22.354439+0000","last_undegraded":"2024-02-19T16:22:22.354439+0000","last_fullsized":"2024-02-19T16:22:22.354439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T01:01:49.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351048+0000","last_change":"2024-02-19T16:20:21.666265+0000","last_active":"2024-02-19T16:22:22.351048+0000","last_peered":"2024-02-19T16:22:22.351048+0000","last_clean":"2024-02-19T16:22:22.351048+0000","last_became_active":"2024-02-19T16:20:21.665919+0000","last_became_peered":"2024-02-19T16:20:21.665919+0000","last_unstale":"2024-02-19T16:22:22.351048+0000","last_undegraded":"2024-02-19T16:22:22.351048+0000","last_fullsized":"2024-02-19T16:22:22.351048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:55:27.581755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353266+0000","last_change":"2024-02-19T16:20:25.755570+0000","last_active":"2024-02-19T16:22:22.353266+0000","last_peered":"2024-02-19T16:22:22.353266+0000","last_clean":"2024-02-19T16:22:22.353266+0000","last_became_active":"2024-02-19T16:20:25.755072+0000","last_became_peered":"2024-02-19T16:20:25.755072+0000","last_unstale":"2024-02-19T16:22:22.353266+0000","last_undegraded":"2024-02-19T16:22:22.353266+0000","last_fullsized":"2024-02-19T16:22:22.353266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:24.733272+0000","last_clean_scrub_stamp":"2024-02-19T16:20:24.733272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:33:48.505710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354966+0000","last_change":"2024-02-19T16:20:20.630258+0000","last_active":"2024-02-19T16:22:22.354966+0000","last_peered":"2024-02-19T16:22:22.354966+0000","last_clean":"2024-02-19T16:22:22.354966+0000","last_became_active":"2024-02-19T16:20:20.629941+0000","last_became_peered":"2024-02-19T16:20:20.629941+0000","last_unstale":"2024-02-19T16:22:22.354966+0000","last_undegraded":"2024-02-19T16:22:22.354966+0000","last_fullsized":"2024-02-19T16:22:22.354966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:19.613409+0000","last_clean_scrub_stamp":"2024-02-19T16:20:19.613409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:44:32.435839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.351376+0000","last_change":"2024-02-19T16:20:21.663139+0000","last_active":"2024-02-19T16:22:22.351376+0000","last_peered":"2024-02-19T16:22:22.351376+0000","last_clean":"2024-02-19T16:22:22.351376+0000","last_became_active":"2024-02-19T16:20:21.662780+0000","last_became_peered":"2024-02-19T16:20:21.662780+0000","last_unstale":"2024-02-19T16:22:22.351376+0000","last_undegraded":"2024-02-19T16:22:22.351376+0000","last_fullsized":"2024-02-19T16:22:22.351376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:03:18.760236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449671+0000","last_change":"2024-02-19T16:20:21.663323+0000","last_active":"2024-02-19T16:22:22.449671+0000","last_peered":"2024-02-19T16:22:22.449671+0000","last_clean":"2024-02-19T16:22:22.449671+0000","last_became_active":"2024-02-19T16:20:21.663017+0000","last_became_peered":"2024-02-19T16:20:21.663017+0000","last_unstale":"2024-02-19T16:22:22.449671+0000","last_undegraded":"2024-02-19T16:22:22.449671+0000","last_fullsized":"2024-02-19T16:22:22.449671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:21:34.272798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352733+0000","last_change":"2024-02-19T16:20:21.670385+0000","last_active":"2024-02-19T16:22:22.352733+0000","last_peered":"2024-02-19T16:22:22.352733+0000","last_clean":"2024-02-19T16:22:22.352733+0000","last_became_active":"2024-02-19T16:20:21.669924+0000","last_became_peered":"2024-02-19T16:20:21.669924+0000","last_unstale":"2024-02-19T16:22:22.352733+0000","last_undegraded":"2024-02-19T16:22:22.352733+0000","last_fullsized":"2024-02-19T16:22:22.352733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:08:57.236359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.352775+0000","last_change":"2024-02-19T16:20:21.659394+0000","last_active":"2024-02-19T16:22:22.352775+0000","last_peered":"2024-02-19T16:22:22.352775+0000","last_clean":"2024-02-19T16:22:22.352775+0000","last_became_active":"2024-02-19T16:20:21.658929+0000","last_became_peered":"2024-02-19T16:20:21.658929+0000","last_unstale":"2024-02-19T16:22:22.352775+0000","last_undegraded":"2024-02-19T16:22:22.352775+0000","last_fullsized":"2024-02-19T16:22:22.352775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T22:35:34.848797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.450420+0000","last_change":"2024-02-19T16:21:21.884814+0000","last_active":"2024-02-19T16:22:22.450420+0000","last_peered":"2024-02-19T16:22:22.450420+0000","last_clean":"2024-02-19T16:22:22.450420+0000","last_became_active":"2024-02-19T16:21:21.884410+0000","last_became_peered":"2024-02-19T16:21:21.884410+0000","last_unstale":"2024-02-19T16:22:22.450420+0000","last_undegraded":"2024-02-19T16:22:22.450420+0000","last_fullsized":"2024-02-19T16:22:22.450420+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T03:39:44.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354725+0000","last_change":"2024-02-19T16:20:21.658039+0000","last_active":"2024-02-19T16:22:22.354725+0000","last_peered":"2024-02-19T16:22:22.354725+0000","last_clean":"2024-02-19T16:22:22.354725+0000","last_became_active":"2024-02-19T16:20:21.657695+0000","last_became_peered":"2024-02-19T16:20:21.657695+0000","last_unstale":"2024-02-19T16:22:22.354725+0000","last_undegraded":"2024-02-19T16:22:22.354725+0000","last_fullsized":"2024-02-19T16:22:22.354725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:50:53.105638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449925+0000","last_change":"2024-02-19T16:20:21.666650+0000","last_active":"2024-02-19T16:22:22.449925+0000","last_peered":"2024-02-19T16:22:22.449925+0000","last_clean":"2024-02-19T16:22:22.449925+0000","last_became_active":"2024-02-19T16:20:21.666367+0000","last_became_peered":"2024-02-19T16:20:21.666367+0000","last_unstale":"2024-02-19T16:22:22.449925+0000","last_undegraded":"2024-02-19T16:22:22.449925+0000","last_fullsized":"2024-02-19T16:22:22.449925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T04:01:02.977083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.354587+0000","last_change":"2024-02-19T16:20:21.657869+0000","last_active":"2024-02-19T16:22:22.354587+0000","last_peered":"2024-02-19T16:22:22.354587+0000","last_clean":"2024-02-19T16:22:22.354587+0000","last_became_active":"2024-02-19T16:20:21.657323+0000","last_became_peered":"2024-02-19T16:20:21.657323+0000","last_unstale":"2024-02-19T16:22:22.354587+0000","last_undegraded":"2024-02-19T16:22:22.354587+0000","last_fullsized":"2024-02-19T16:22:22.354587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T17:48:31.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349718+0000","last_change":"2024-02-19T16:20:21.661067+0000","last_active":"2024-02-19T16:22:22.349718+0000","last_peered":"2024-02-19T16:22:22.349718+0000","last_clean":"2024-02-19T16:22:22.349718+0000","last_became_active":"2024-02-19T16:20:21.660750+0000","last_became_peered":"2024-02-19T16:20:21.660750+0000","last_unstale":"2024-02-19T16:22:22.349718+0000","last_undegraded":"2024-02-19T16:22:22.349718+0000","last_fullsized":"2024-02-19T16:22:22.349718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T16:36:17.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353764+0000","last_change":"2024-02-19T16:20:21.664514+0000","last_active":"2024-02-19T16:22:22.353764+0000","last_peered":"2024-02-19T16:22:22.353764+0000","last_clean":"2024-02-19T16:22:22.353764+0000","last_became_active":"2024-02-19T16:20:21.664169+0000","last_became_peered":"2024-02-19T16:20:21.664169+0000","last_unstale":"2024-02-19T16:22:22.353764+0000","last_undegraded":"2024-02-19T16:22:22.353764+0000","last_fullsized":"2024-02-19T16:22:22.353764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-21T02:55:18.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.349237+0000","last_change":"2024-02-19T16:20:21.663383+0000","last_active":"2024-02-19T16:22:22.349237+0000","last_peered":"2024-02-19T16:22:22.349237+0000","last_clean":"2024-02-19T16:22:22.349237+0000","last_became_active":"2024-02-19T16:20:21.662777+0000","last_became_peered":"2024-02-19T16:20:21.662777+0000","last_unstale":"2024-02-19T16:22:22.349237+0000","last_undegraded":"2024-02-19T16:22:22.349237+0000","last_fullsized":"2024-02-19T16:22:22.349237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:04:48.668268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449690+0000","last_change":"2024-02-19T16:20:21.657632+0000","last_active":"2024-02-19T16:22:22.449690+0000","last_peered":"2024-02-19T16:22:22.449690+0000","last_clean":"2024-02-19T16:22:22.449690+0000","last_became_active":"2024-02-19T16:20:21.657304+0000","last_became_peered":"2024-02-19T16:20:21.657304+0000","last_unstale":"2024-02-19T16:22:22.449690+0000","last_undegraded":"2024-02-19T16:22:22.449690+0000","last_fullsized":"2024-02-19T16:22:22.449690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:56:09.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.353873+0000","last_change":"2024-02-19T16:20:21.655774+0000","last_active":"2024-02-19T16:22:22.353873+0000","last_peered":"2024-02-19T16:22:22.353873+0000","last_clean":"2024-02-19T16:22:22.353873+0000","last_became_active":"2024-02-19T16:20:21.655307+0000","last_became_peered":"2024-02-19T16:20:21.655307+0000","last_unstale":"2024-02-19T16:22:22.353873+0000","last_undegraded":"2024-02-19T16:22:22.353873+0000","last_fullsized":"2024-02-19T16:22:22.353873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T21:32:27.356061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-19T16:22:22.449777+0000","last_change":"2024-02-19T16:20:21.660126+0000","last_active":"2024-02-19T16:22:22.449777+0000","last_peered":"2024-02-19T16:22:22.449777+0000","last_clean":"2024-02-19T16:22:22.449777+0000","last_became_active":"2024-02-19T16:20:21.659804+0000","last_became_peered":"2024-02-19T16:20:21.659804+0000","last_unstale":"2024-02-19T16:22:22.449777+0000","last_undegraded":"2024-02-19T16:22:22.449777+0000","last_fullsized":"2024-02-19T16:22:22.449777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T16:20:20.617414+0000","last_clean_scrub_stamp":"2024-02-19T16:20:20.617414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T20:36:02.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281144,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":732,"num_read_kb":442,"num_write":173,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":213861,"data_compressed":5154,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":532611,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.56999999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.51100000000000001,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.498,"15min":0.54200000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.53500000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.47999999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.40200000000000002,"15min":0.42299999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.53300000000000003,"15min":0.56699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60999999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48199999999999998,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.51800000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.61699999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44700000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.61699999999999999},"min":{"1min":0.48099999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69999999999999996,"5min":0.998,"15min":0.998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61099999999999999,"15min":0.64400000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67800000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.66500000000000004},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.63700000000000001}]},{"osd":9,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65600000000000003,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71999999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66400000000000003,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.78400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68200000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.755,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.65500000000000003,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.63800000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60499999999999998}]}]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1188316,"data_compressed":67184,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.32900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63900000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.495},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.55700000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.377}]},{"osd":1,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.40200000000000002,"15min":0.432},"min":{"1min":0.23499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.34000000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.45700000000000002,"15min":0.49399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.377,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.499,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.48399999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.50700000000000001,"15min":0.54700000000000004},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.438,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35499999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56299999999999994,"15min":0.60099999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61799999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.67400000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.50600000000000001,"15min":0.55000000000000004},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47999999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.46200000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.28299999999999997}]},{"osd":5,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57599999999999996,"15min":0.61799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.56999999999999995,"15min":0.61899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.50800000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38700000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.65400000000000003,"5min":0.999,"15min":0.999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.65900000000000003,"15min":0.69499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71999999999999997,"5min":1.028,"15min":1.028},"last":0.50800000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.67400000000000004,"15min":0.73199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60299999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.66200000000000003,"15min":0.72399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.55800000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.48499999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.69199999999999995,"15min":0.73599999999999999},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.63300000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.68899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:21:37 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.56399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54100000000000004}]}]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":536361,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629}]},{"osd":1,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.626,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.55600000000000005,"15min":0.58099999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.627},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66000000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72699999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.29299999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.436},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.66400000000000003,"15min":0.69399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.749,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Feb 19 16:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82099999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.73299999999999998,"15min":0.78000000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76000000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57199999999999995}]},{"osd":8,"last update":"Mon Feb 19 16:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67700000000000005,"15min":0.71399999999999997},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":1.169,"15min":1.169},"last":0.70899999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.75,"15min":0.78400000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.89500000000000002,"5min":1.278,"15min":1.278},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71999999999999997,"15min":0.76200000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.78000000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64400000000000002}]},{"osd":11,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":0.78700000000000003,"15min":0.68200000000000005},"min":{"1min":0.53600000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":13.175000000000001,"5min":13.175000000000001,"15min":13.175000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54000000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532185,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.44400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48499999999999999,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.52500000000000002,"15min":0.56899999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.505,"5min":0.875,"15min":0.875},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.313,"5min":0.31,"15min":0.31},"max":{"1min":0.56799999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35099999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40200000000000002,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41299999999999998,"15min":0.42399999999999999},"min":{"1min":0.27300000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.53100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.317}]},{"osd":3,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.64600000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.497},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.47699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.54800000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56799999999999995,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.58299999999999996},"min":{"1min":0.29599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68500000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76100000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70899999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64800000000000002,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.68200000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77600000000000002,"5min":1.393,"15min":1.393},"last":0.46800000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.52200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.73599999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68600000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.72299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.441},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.74299999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63300000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.54800000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.78500000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.77300000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1127653,"data_compressed":65466,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.753,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72899999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.379,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.38200000000000001,"5min":0.75,"15min":0.75},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.49299999999999999,"15min":0.53700000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.41899999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40699999999999997}]},{"osd":3,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.64100000000000001,"5min":0.87,"15min":0.87},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.46600000000000003,"15min":0.48099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.46500000000000002,"15min":0.45200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432}]},{"osd":5,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.55500000000000005,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.504,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.61299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45400000000000001}]},{"osd":6,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64100000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.61799999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59199999999999997}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.60199999999999998,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.57199999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.58499999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.625}]},{"osd":9,"last update":"Mon Feb 19 16:21:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67400000000000004,"5min":0.997,"15min":0.997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70099999999999996,"15min":0.71499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.70599999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54600000000000004}]},{"osd":10,"last update":"Mon Feb 19 16:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.67200000000000004,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.64200000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.64000000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48999999999999999}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.65200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.66800000000000004,"5min":1.024,"15min":1.024},"last":0.60499999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":527441,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80600000000000005}]},{"osd":2,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82899999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.48099999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.56999999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94399999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76000000000000001}]},{"osd":8,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.56799999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":1.014,"15min":1.014},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91300000000000003,"5min":1.113,"15min":1.113},"last":0.84099999999999997}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.57299999999999995,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86299999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.50700000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.872},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.60399999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0089999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81999999999999995}]},{"osd":11,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.878,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0449999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":589375,"data_compressed":31426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.50800000000000001},"min":{"1min":0.311,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.68700000000000006,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.2210000000000001,"5min":14.067,"15min":14.067},"last":0.52800000000000002}]},{"osd":2,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.47299999999999998,"15min":0.51400000000000001},"min":{"1min":0.26700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.46400000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.49199999999999999,"15min":0.51500000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.502,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66200000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.41399999999999998,"15min":0.376},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59999999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.44400000000000001,"15min":0.40500000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.313}]},{"osd":5,"last update":"Mon Feb 19 16:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.371},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.44600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70899999999999996,"5min":0.879,"15min":0.879},"last":0.40400000000000003}]},{"osd":6,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.625,"15min":0.624},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71399999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51500000000000001}]},{"osd":7,"last update":"Mon Feb 19 16:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68100000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.38100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.503}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.71799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42199999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.82099999999999995,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":14.105,"5min":14.105,"15min":14.105},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67700000000000005,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71099999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48199999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.63400000000000001,"15min":0.69599999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.63800000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71199999999999997,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.67200000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56100000000000005}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.68999999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.70599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.62}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528829,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 16:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.54400000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67100000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71499999999999997}]},{"osd":2,"last update":"Mon Feb 19 16:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58899999999999997,"5min":1.899,"15min":1.899},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.58399999999999996,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Mon Feb 19 16:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.38,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Feb 19 16:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.62,"5min":0.63300000000000001,"15min":0.60199999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80700000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.624}]},{"osd":7,"last update":"Mon Feb 19 16:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84999999999999998}]},{"osd":8,"last update":"Mon Feb 19 16:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.51300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86499999999999999}]},{"osd":9,"last update":"Mon Feb 19 16:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.45900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.98499999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.90500000000000003}]},{"osd":11,"last update":"Mon Feb 19 16:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":599317,"data_compressed":31426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59199999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61099999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.39000000000000001}]},{"osd":1,"last update":"Mon Feb 19 16:21:58 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.47299999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70599999999999996,"5min":1.758,"15min":1.758},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59299999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.377},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61799999999999999,"5min":2.012,"15min":2.012},"last":0.439}]},{"osd":5,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.64100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.47799999999999998},"min":{"1min":0.317,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58299999999999996,"5min":1.026,"15min":1.026},"last":0.496}]},{"osd":6,"last update":"Mon Feb 19 16:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.621},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72099999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Feb 19 16:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.622},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61699999999999999,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69799999999999995,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.52300000000000002}]},{"osd":8,"last update":"Mon Feb 19 16:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53900000000000003,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.63800000000000001},"min":{"1min":0.503,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.626,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53500000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69399999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.627},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.755,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.60499999999999998}]},{"osd":10,"last update":"Mon Feb 19 16:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.70899999999999996,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.67600000000000005,"5min":1.282,"15min":1.282},"last":0.56200000000000006}]},{"osd":11,"last update":"Mon Feb 19 16:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.72599999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59399999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":527635,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.36499999999999999},"min":{"1min":0.214,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.379},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.378}]},{"osd":1,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60699999999999998,"5min":0.81399999999999995,"15min":0.90800000000000003},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56799999999999995,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64000000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82399999999999995,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77800000000000002,"5min":17.469999999999999,"15min":17.469999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Feb 19 16:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Feb 19 16:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Feb 19 16:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.018},"last":0.66700000000000004}]},{"osd":8,"last update":"Mon Feb 19 16:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.621}]},{"osd":9,"last update":"Mon Feb 19 16:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.05,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57799999999999996}]},{"osd":10,"last update":"Mon Feb 19 16:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.999,"5min":1.03,"15min":1.03},"last":0.71799999999999997}]},{"osd":11,"last update":"Mon Feb 19 16:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":532623,"data_compressed":29708,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.436}]},{"osd":1,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.38700000000000001,"15min":0.375},"min":{"1min":0.26300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.34899999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38900000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.373,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.311}]},{"osd":2,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.51500000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.433}]},{"osd":3,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59899999999999998,"5min":1.03,"15min":1.03},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.60499999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Feb 19 16:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.47399999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.55000000000000004,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.51100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47399999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.502}]},{"osd":7,"last update":"Mon Feb 19 16:22:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.53400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55200000000000005}]},{"osd":8,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.58299999999999996,"5min":1.073,"15min":1.073},"last":0.52800000000000002}]},{"osd":9,"last update":"Mon Feb 19 16:21:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":0.64100000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.70299999999999996,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.65300000000000002,"5min":1.044,"15min":1.044},"last":0.61199999999999999}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.67000000000000004,"15min":0.70799999999999996},"min":{"1min":0.442,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.64100000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.70699999999999996,"15min":0.75},"min":{"1min":0.45400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66500000000000004,"5min":1.056,"15min":1.056},"last":0.51500000000000001}]},{"osd":11,"last update":"Mon Feb 19 16:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.70599999999999996,"5min":1.133,"15min":1.133},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.69899999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.621}]}]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1123291,"data_compressed":65466,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.64300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67400000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.48299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53200000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.372,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38600000000000001,"15min":0.375},"min":{"1min":0.25,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68700000000000006,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Feb 19 16:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.871,"15min":0.871},"last":0.436},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.505,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Mon Feb 19 16:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.65600000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.65100000000000002,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70799999999999996,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":0.42199999999999999}]},{"osd":7,"last update":"Mon Feb 19 16:22:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66600000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64200000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46800000000000003}]},{"osd":8,"last update":"Mon Feb 19 16:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.318,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72099999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.57099999999999995,"15min":0.623},"min":{"1min":0.38200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53300000000000003}]},{"osd":9,"last update":"Mon Feb 19 16:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66300000000000003,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.73299999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.74199999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55600000000000005}]},{"osd":10,"last update":"Mon Feb 19 16:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68700000000000006,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.68200000000000005,"15min":0.72199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.67800000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34118,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35758,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62332,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71966,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70597,"data_compressed":1718,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T16:22:42.409 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:42.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 -- 172.21.15.8:0/1792889386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c1030d0 msgr2=0x7efd2c109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 --2- 172.21.15.8:0/1792889386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c1030d0 0x7efd2c109960 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd24009ca0 tx=0x7efd2402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 -- 172.21.15.8:0/1792889386 shutdown_connections 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 --2- 172.21.15.8:0/1792889386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c1030d0 0x7efd2c109960 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 --2- 172.21.15.8:0/1792889386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c102710 0x7efd2c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 --2- 172.21.15.8:0/1792889386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c101520 0x7efd2c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 -- 172.21.15.8:0/1792889386 >> 172.21.15.8:0/1792889386 conn(0x7efd2c0fcad0 msgr2=0x7efd2c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 -- 172.21.15.8:0/1792889386 shutdown_connections 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.542+0000 7efd3473e640 1 -- 172.21.15.8:0/1792889386 wait complete. 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 Processor -- start 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 -- start start 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c101520 0x7efd2c0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 0x7efd2c0738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c1030d0 0x7efd2c073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c074580 con 0x7efd2c102710 2024-02-19T16:22:42.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0746f0 con 0x7efd2c1030d0 2024-02-19T16:22:42.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd3473e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c1a1590 con 0x7efd2c101520 2024-02-19T16:22:42.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd31cd9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 0x7efd2c0738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd31cd9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 0x7efd2c0738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49606/0 (socket says 172.21.15.8:49606) 2024-02-19T16:22:42.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd31cd9640 1 -- 172.21.15.8:0/4084018540 learned_addr learned my addr 172.21.15.8:0/4084018540 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:42.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd32cdb640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c1030d0 0x7efd2c073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.543+0000 7efd324da640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c101520 0x7efd2c0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd31cd9640 1 -- 172.21.15.8:0/4084018540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c101520 msgr2=0x7efd2c0752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd31cd9640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c101520 0x7efd2c0752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd31cd9640 1 -- 172.21.15.8:0/4084018540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c1030d0 msgr2=0x7efd2c073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd31cd9640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c1030d0 0x7efd2c073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd31cd9640 1 -- 172.21.15.8:0/4084018540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24009900 con 0x7efd2c102710 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd32cdb640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c1030d0 0x7efd2c073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd31cd9640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 0x7efd2c0738e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00ea20 tx=0x7efd1c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c009720 con 0x7efd2c102710 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd1c00cd80 con 0x7efd2c102710 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c1a1840 con 0x7efd2c102710 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c010490 con 0x7efd2c102710 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.544+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c1a1d00 con 0x7efd2c102710 2024-02-19T16:22:42.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.545+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c1087e0 con 0x7efd2c102710 2024-02-19T16:22:42.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.546+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7efd1c005000 con 0x7efd2c102710 2024-02-19T16:22:42.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.546+0000 7efd237fe640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2150 0x7efd180a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.547+0000 7efd324da640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2150 0x7efd180a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.547+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7efd1c014070 con 0x7efd2c102710 2024-02-19T16:22:42.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.547+0000 7efd324da640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2150 0x7efd180a4610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efd2c102570 tx=0x7efd28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:42.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.550+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7efd1c0972a0 con 0x7efd2c102710 2024-02-19T16:22:42.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.731+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd2c10cf30 con 0x7efd2c102710 2024-02-19T16:22:42.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.733+0000 7efd237fe640 1 -- 172.21.15.8:0/4084018540 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7efd1c09a960 con 0x7efd2c102710 2024-02-19T16:22:42.734 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:42.734 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:42.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.734+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2150 msgr2=0x7efd180a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.734+0000 7efd3473e640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2150 0x7efd180a4610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efd2c102570 tx=0x7efd28005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 msgr2=0x7efd2c0738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 0x7efd2c0738e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00ea20 tx=0x7efd1c00ee20 comp rx=0 tx=0).stop 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 shutdown_connections 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7efd180a2150 0x7efd180a4610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd2c1030d0 0x7efd2c073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd2c102710 0x7efd2c0738e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 --2- 172.21.15.8:0/4084018540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c101520 0x7efd2c0752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 >> 172.21.15.8:0/4084018540 conn(0x7efd2c0fcad0 msgr2=0x7efd2c1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 shutdown_connections 2024-02-19T16:22:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.735+0000 7efd3473e640 1 -- 172.21.15.8:0/4084018540 wait complete. 2024-02-19T16:22:42.745 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:42 smithi008 ceph-mon[32573]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-02-19T16:22:42.745 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:42 smithi008 ceph-mon[32573]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:42.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:22:42.747 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:22:42.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:42 smithi191 ceph-mon[37203]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-02-19T16:22:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:42 smithi191 ceph-mon[37203]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:42 smithi156 ceph-mon[36502]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-02-19T16:22:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:42 smithi156 ceph-mon[36502]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:42.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.851+0000 7faa479bc640 1 -- 172.21.15.8:0/3760396639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa40101fa0 msgr2=0x7faa40108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.851+0000 7faa479bc640 1 --2- 172.21.15.8:0/3760396639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa40101fa0 0x7faa40108830 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faa38009ca0 tx=0x7faa3802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- 172.21.15.8:0/3760396639 shutdown_connections 2024-02-19T16:22:42.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 --2- 172.21.15.8:0/3760396639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa40101fa0 0x7faa40108830 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 --2- 172.21.15.8:0/3760396639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa401015e0 0x7faa40101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 --2- 172.21.15.8:0/3760396639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa401003f0 0x7faa40100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- 172.21.15.8:0/3760396639 >> 172.21.15.8:0/3760396639 conn(0x7faa400fb980 msgr2=0x7faa400fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- 172.21.15.8:0/3760396639 shutdown_connections 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- 172.21.15.8:0/3760396639 wait complete. 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 Processor -- start 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- start start 2024-02-19T16:22:42.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa401003f0 0x7faa401975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 0x7faa40197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa40101fa0 0x7faa4019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4019f0b0 con 0x7faa401015e0 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4019f220 con 0x7faa40101fa0 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa479bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4019f3e0 con 0x7faa401003f0 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa44f57640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 0x7faa40197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa44f57640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 0x7faa40197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49622/0 (socket says 172.21.15.8:49622) 2024-02-19T16:22:42.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.852+0000 7faa44f57640 1 -- 172.21.15.8:0/4086725595 learned_addr learned my addr 172.21.15.8:0/4086725595 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa45758640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa401003f0 0x7faa401975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa45f59640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa40101fa0 0x7faa4019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa44f57640 1 -- 172.21.15.8:0/4086725595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa401003f0 msgr2=0x7faa401975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa44f57640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa401003f0 0x7faa401975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa44f57640 1 -- 172.21.15.8:0/4086725595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa40101fa0 msgr2=0x7faa4019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa44f57640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa40101fa0 0x7faa4019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:42.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa44f57640 1 -- 172.21.15.8:0/4086725595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009900 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa44f57640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 0x7faa40197b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c009700 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa3c00cd80 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4019f6c0 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c018920 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.853+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4019fbd0 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.854+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4010bdd0 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.856+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7faa3c018a80 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.856+0000 7faa327fc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa2c0a2150 0x7faa2c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.857+0000 7faa45758640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa2c0a2150 0x7faa2c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.857+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7faa3c014070 con 0x7faa401015e0 2024-02-19T16:22:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.857+0000 7faa45758640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa2c0a2150 0x7faa2c0a4610 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faa40101440 tx=0x7faa34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:42.860+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7faa3c097560 con 0x7faa401015e0 2024-02-19T16:22:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.043+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa401a0090 con 0x7faa401015e0 2024-02-19T16:22:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.044+0000 7faa327fc640 1 -- 172.21.15.8:0/4086725595 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa3c0260a0 con 0x7faa401015e0 2024-02-19T16:22:43.044 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:22:43.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.045+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa2c0a2150 msgr2=0x7faa2c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa2c0a2150 0x7faa2c0a4610 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faa40101440 tx=0x7faa34005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 msgr2=0x7faa40197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 0x7faa40197b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 shutdown_connections 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7faa2c0a2150 0x7faa2c0a4610 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa40101fa0 0x7faa4019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa401015e0 0x7faa40197b00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 --2- 172.21.15.8:0/4086725595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa401003f0 0x7faa401975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 >> 172.21.15.8:0/4086725595 conn(0x7faa400fb980 msgr2=0x7faa400fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 shutdown_connections 2024-02-19T16:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.046+0000 7faa479bc640 1 -- 172.21.15.8:0/4086725595 wait complete. 2024-02-19T16:22:43.058 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:22:43.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.163+0000 7f9e92ced640 1 -- 172.21.15.8:0/1015979388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 msgr2=0x7f9e8c100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.163+0000 7f9e92ced640 1 --2- 172.21.15.8:0/1015979388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c100850 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009900 tx=0x7f9e8002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:43.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.163+0000 7f9e92ced640 1 -- 172.21.15.8:0/1015979388 shutdown_connections 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.163+0000 7f9e92ced640 1 --2- 172.21.15.8:0/1015979388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 0x7f9e8c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.163+0000 7f9e92ced640 1 --2- 172.21.15.8:0/1015979388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e8c101620 0x7f9e8c101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 --2- 172.21.15.8:0/1015979388 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c100850 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- 172.21.15.8:0/1015979388 >> 172.21.15.8:0/1015979388 conn(0x7f9e8c0fb9a0 msgr2=0x7f9e8c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- 172.21.15.8:0/1015979388 shutdown_connections 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- 172.21.15.8:0/1015979388 wait complete. 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 Processor -- start 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- start start 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e8c101620 0x7f9e8c19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 0x7f9e8c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19f160 con 0x7f9e8c100430 2024-02-19T16:22:43.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19f2d0 con 0x7f9e8c101fe0 2024-02-19T16:22:43.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e92ced640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19f490 con 0x7f9e8c101620 2024-02-19T16:22:43.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e90a89640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e90a89640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c06a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49626/0 (socket says 172.21.15.8:49626) 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e90a89640 1 -- 172.21.15.8:0/504182281 learned_addr learned my addr 172.21.15.8:0/504182281 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e8bfff640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e8c101620 0x7f9e8c19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.164+0000 7f9e9128a640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 0x7f9e8c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e90a89640 1 -- 172.21.15.8:0/504182281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e8c101620 msgr2=0x7f9e8c19c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e90a89640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e8c101620 0x7f9e8c19c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e90a89640 1 -- 172.21.15.8:0/504182281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 msgr2=0x7f9e8c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e90a89640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 0x7f9e8c19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e90a89640 1 -- 172.21.15.8:0/504182281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e84009840 con 0x7f9e8c100430 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e9128a640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 0x7f9e8c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e90a89640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c06a3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005ab0 tx=0x7f9e80005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:43.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80004350 con 0x7f9e8c100430 2024-02-19T16:22:43.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e80009560 con 0x7f9e8c100430 2024-02-19T16:22:43.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e800044b0 con 0x7f9e8c100430 2024-02-19T16:22:43.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80003d00 con 0x7f9e8c100430 2024-02-19T16:22:43.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.165+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c19fae0 con 0x7f9e8c100430 2024-02-19T16:22:43.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.166+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c067a50 con 0x7f9e8c100430 2024-02-19T16:22:43.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.168+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f9e80004df0 con 0x7f9e8c100430 2024-02-19T16:22:43.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.168+0000 7f9e89ffb640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9e7c0a2150 0x7f9e7c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.168+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f9e800f0700 con 0x7f9e8c100430 2024-02-19T16:22:43.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.168+0000 7f9e8bfff640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9e7c0a2150 0x7f9e7c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.169+0000 7f9e8bfff640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9e7c0a2150 0x7f9e7c0a4610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c06a110 tx=0x7f9e78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:43.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.176+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9e800b92e0 con 0x7f9e8c100430 2024-02-19T16:22:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.359+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e8c02cf60 con 0x7f9e8c100430 2024-02-19T16:22:43.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.360+0000 7f9e89ffb640 1 -- 172.21.15.8:0/504182281 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e80003e60 con 0x7f9e8c100430 2024-02-19T16:22:43.361 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9e7c0a2150 msgr2=0x7f9e7c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9e7c0a2150 0x7f9e7c0a4610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c06a110 tx=0x7f9e78005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 msgr2=0x7f9e8c06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c06a3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005ab0 tx=0x7f9e80005de0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 shutdown_connections 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9e7c0a2150 0x7f9e7c0a4610 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101fe0 0x7f9e8c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e8c101620 0x7f9e8c19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 --2- 172.21.15.8:0/504182281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e8c100430 0x7f9e8c06a3a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 >> 172.21.15.8:0/504182281 conn(0x7f9e8c0fb9a0 msgr2=0x7f9e8c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:43.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 shutdown_connections 2024-02-19T16:22:43.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.362+0000 7f9e92ced640 1 -- 172.21.15.8:0/504182281 wait complete. 2024-02-19T16:22:43.374 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:22:43.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 -- 172.21.15.8:0/1257666580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8101620 msgr2=0x7f79d8101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/1257666580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8101620 0x7f79d8101aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009900 tx=0x7f79c002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:43.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 -- 172.21.15.8:0/1257666580 shutdown_connections 2024-02-19T16:22:43.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/1257666580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8101fe0 0x7f79d8108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/1257666580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8101620 0x7f79d8101aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/1257666580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8100430 0x7f79d8100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.493+0000 7f79dcc1b640 1 -- 172.21.15.8:0/1257666580 >> 172.21.15.8:0/1257666580 conn(0x7f79d80fb9a0 msgr2=0x7f79d80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 -- 172.21.15.8:0/1257666580 shutdown_connections 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 -- 172.21.15.8:0/1257666580 wait complete. 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 Processor -- start 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 -- start start 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8100430 0x7f79d8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8101620 0x7f79d8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 0x7f79d819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d819f120 con 0x7f79d8101fe0 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d819f290 con 0x7f79d8101620 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.494+0000 7f79dcc1b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d819f450 con 0x7f79d8100430 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 0x7f79d819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 0x7f79d819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49644/0 (socket says 172.21.15.8:49644) 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 -- 172.21.15.8:0/3332403508 learned_addr learned my addr 172.21.15.8:0/3332403508 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d659c640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8100430 0x7f79d8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d5d9b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8101620 0x7f79d8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 -- 172.21.15.8:0/3332403508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8100430 msgr2=0x7f79d8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8100430 0x7f79d8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 -- 172.21.15.8:0/3332403508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8101620 msgr2=0x7f79d8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8101620 0x7f79d8197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 -- 172.21.15.8:0/3332403508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009490 con 0x7f79d8101fe0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79d6d9d640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 0x7f79d819ebe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79c800eb10 tx=0x7f79c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8004b20 con 0x7f79d8101fe0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c0009560 con 0x7f79d8101fe0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79c8004c80 con 0x7f79d8101fe0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8010670 con 0x7f79d8101fe0 2024-02-19T16:22:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.495+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d819fa80 con 0x7f79d8101fe0 2024-02-19T16:22:43.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.496+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d810be40 con 0x7f79d8101fe0 2024-02-19T16:22:43.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.497+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f79c8005020 con 0x7f79d8101fe0 2024-02-19T16:22:43.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.498+0000 7f79cf7fe640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f79c40a2150 0x7f79c40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.498+0000 7f79d659c640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f79c40a2150 0x7f79c40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.498+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f79c8014070 con 0x7f79d8101fe0 2024-02-19T16:22:43.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.499+0000 7f79d659c640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f79c40a2150 0x7f79c40a4610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f79d8101480 tx=0x7f79d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:43.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.503+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f79c80971f0 con 0x7f79d8101fe0 2024-02-19T16:22:43.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.686+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79d819fd00 con 0x7f79d8101fe0 2024-02-19T16:22:43.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.687+0000 7f79cf7fe640 1 -- 172.21.15.8:0/3332403508 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79d819fd00 con 0x7f79d8101fe0 2024-02-19T16:22:43.688 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f79c40a2150 msgr2=0x7f79c40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f79c40a2150 0x7f79c40a4610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f79d8101480 tx=0x7f79d0005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 msgr2=0x7f79d819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 0x7f79d819ebe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79c800eb10 tx=0x7f79c800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 shutdown_connections 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f79c40a2150 0x7f79c40a4610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79d8101fe0 0x7f79d819ebe0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8101620 0x7f79d8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 --2- 172.21.15.8:0/3332403508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79d8100430 0x7f79d8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 >> 172.21.15.8:0/3332403508 conn(0x7f79d80fb9a0 msgr2=0x7f79d8069810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:43.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 shutdown_connections 2024-02-19T16:22:43.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.689+0000 7f79dcc1b640 1 -- 172.21.15.8:0/3332403508 wait complete. 2024-02-19T16:22:43.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:43 smithi008 ceph-mon[32573]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:43.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:43 smithi008 ceph-mon[32573]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:43.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4084018540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:43.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4086725595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:43.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/504182281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:43.701 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:22:43.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:43 smithi191 ceph-mon[37203]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:43 smithi191 ceph-mon[37203]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4084018540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4086725595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:43.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/504182281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 -- 172.21.15.8:0/865165641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d14101fa0 msgr2=0x7f9d14108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 --2- 172.21.15.8:0/865165641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d14101fa0 0x7f9d14108830 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009ca0 tx=0x7f9d0c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 -- 172.21.15.8:0/865165641 shutdown_connections 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 --2- 172.21.15.8:0/865165641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d14101fa0 0x7f9d14108830 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 --2- 172.21.15.8:0/865165641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d141015e0 0x7f9d14101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 --2- 172.21.15.8:0/865165641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 0x7f9d14100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 -- 172.21.15.8:0/865165641 >> 172.21.15.8:0/865165641 conn(0x7f9d140fb980 msgr2=0x7f9d140fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 -- 172.21.15.8:0/865165641 shutdown_connections 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 -- 172.21.15.8:0/865165641 wait complete. 2024-02-19T16:22:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.808+0000 7f9d1961c640 1 Processor -- start 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 -- start start 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 0x7f9d141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 0x7f9d14199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d14101fa0 0x7f9d141a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d141a12b0 con 0x7f9d141015e0 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d141a1420 con 0x7f9d14101fa0 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d1961c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d141a15e0 con 0x7f9d141003f0 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 0x7f9d14199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 0x7f9d141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 0x7f9d14199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49654/0 (socket says 172.21.15.8:49654) 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d0bfff640 1 -- 172.21.15.8:0/3549922516 learned_addr learned my addr 172.21.15.8:0/3549922516 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d137fe640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d14101fa0 0x7f9d141a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 -- 172.21.15.8:0/3549922516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d14101fa0 msgr2=0x7f9d141a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d14101fa0 0x7f9d141a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 -- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 msgr2=0x7f9d14199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 0x7f9d14199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 -- 172.21.15.8:0/3549922516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d04009490 con 0x7f9d141003f0 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d0bfff640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 0x7f9d14199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d12ffd640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 0x7f9d141997c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9c0 tx=0x7f9d0400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.809+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400cd80 con 0x7f9d141003f0 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.810+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d04010910 con 0x7f9d141003f0 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.810+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04018900 con 0x7f9d141003f0 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.810+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c009900 con 0x7f9d141003f0 2024-02-19T16:22:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.810+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d141a1c40 con 0x7f9d141003f0 2024-02-19T16:22:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.811+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1410bdd0 con 0x7f9d141003f0 2024-02-19T16:22:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.812+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f9d04010a80 con 0x7f9d141003f0 2024-02-19T16:22:43.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.812+0000 7f9d10ff9640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d000a2150 0x7f9d000a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:43.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.812+0000 7f9d0bfff640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d000a2150 0x7f9d000a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:43.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.812+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f9d04014070 con 0x7f9d141003f0 2024-02-19T16:22:43.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.813+0000 7f9d0bfff640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d000a2150 0x7f9d000a4610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9d1419acd0 tx=0x7f9cfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:43.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:43.821+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9d04097100 con 0x7f9d141003f0 2024-02-19T16:22:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:43 smithi156 ceph-mon[36502]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:43 smithi156 ceph-mon[36502]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T16:22:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4084018540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4086725595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:22:43.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/504182281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.003+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9d141a20e0 con 0x7f9d141003f0 2024-02-19T16:22:44.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.003+0000 7f9d10ff9640 1 -- 172.21.15.8:0/3549922516 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d0409a7c0 con 0x7f9d141003f0 2024-02-19T16:22:44.004 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d000a2150 msgr2=0x7f9d000a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d000a2150 0x7f9d000a4610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9d1419acd0 tx=0x7f9cfc005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 msgr2=0x7f9d141997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 0x7f9d141997c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9c0 tx=0x7f9d0400ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 shutdown_connections 2024-02-19T16:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f9d000a2150 0x7f9d000a4610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d14101fa0 0x7f9d141a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d141015e0 0x7f9d14199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 --2- 172.21.15.8:0/3549922516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d141003f0 0x7f9d141997c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 >> 172.21.15.8:0/3549922516 conn(0x7f9d140fb980 msgr2=0x7f9d140fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 shutdown_connections 2024-02-19T16:22:44.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.005+0000 7f9d1961c640 1 -- 172.21.15.8:0/3549922516 wait complete. 2024-02-19T16:22:44.017 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 -- 172.21.15.8:0/886927691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101048c0 msgr2=0x7fd510106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 --2- 172.21.15.8:0/886927691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101048c0 0x7fd510106cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd500009900 tx=0x7fd50002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 -- 172.21.15.8:0/886927691 shutdown_connections 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 --2- 172.21.15.8:0/886927691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5101071f0 0x7fd510109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 --2- 172.21.15.8:0/886927691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101048c0 0x7fd510106cb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 --2- 172.21.15.8:0/886927691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 0x7fd510104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 -- 172.21.15.8:0/886927691 >> 172.21.15.8:0/886927691 conn(0x7fd5100fb980 msgr2=0x7fd5100fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 -- 172.21.15.8:0/886927691 shutdown_connections 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 -- 172.21.15.8:0/886927691 wait complete. 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 Processor -- start 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.120+0000 7fd51684e640 1 -- start start 2024-02-19T16:22:44.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 0x7fd510101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5101048c0 0x7fd5100fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 0x7fd510100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510100be0 con 0x7fd5101071f0 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510100d50 con 0x7fd510101f90 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5101a1640 con 0x7fd5101048c0 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 0x7fd510100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 0x7fd510101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 0x7fd510100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49674/0 (socket says 172.21.15.8:49674) 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd50f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5101048c0 0x7fd5100fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 -- 172.21.15.8:0/307685803 learned_addr learned my addr 172.21.15.8:0/307685803 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 -- 172.21.15.8:0/307685803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5101048c0 msgr2=0x7fd5100fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5101048c0 0x7fd5100fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 -- 172.21.15.8:0/307685803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 msgr2=0x7fd510101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 0x7fd510101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 -- 172.21.15.8:0/307685803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504009490 con 0x7fd5101071f0 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd50ffff640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 0x7fd510101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd514deb640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 0x7fd510100480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd50800eb40 tx=0x7fd50800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508004b20 con 0x7fd5101071f0 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd500009560 con 0x7fd5101071f0 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.121+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd508004c80 con 0x7fd5101071f0 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.122+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5101a1c90 con 0x7fd5101071f0 2024-02-19T16:22:44.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.122+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508018630 con 0x7fd5101071f0 2024-02-19T16:22:44.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.123+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5100fd470 con 0x7fd5101071f0 2024-02-19T16:22:44.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.124+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fd508005020 con 0x7fd5101071f0 2024-02-19T16:22:44.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.124+0000 7fd50d7fa640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd4f80a2150 0x7fd4f80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.124+0000 7fd50ffff640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd4f80a2150 0x7fd4f80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.124+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fd508014070 con 0x7fd5101071f0 2024-02-19T16:22:44.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.125+0000 7fd50ffff640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd4f80a2150 0x7fd4f80a4610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd504011630 tx=0x7fd504005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.129+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd5080972f0 con 0x7fd5101071f0 2024-02-19T16:22:44.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.364+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd51010cf20 con 0x7fd5101071f0 2024-02-19T16:22:44.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.365+0000 7fd50d7fa640 1 -- 172.21.15.8:0/307685803 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fd51010cf20 con 0x7fd5101071f0 2024-02-19T16:22:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:44.366 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:22:36.842920+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:22:44.367 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd4f80a2150 msgr2=0x7fd4f80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd4f80a2150 0x7fd4f80a4610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd504011630 tx=0x7fd504005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 msgr2=0x7fd510100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 0x7fd510100480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd50800eb40 tx=0x7fd50800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 shutdown_connections 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fd4f80a2150 0x7fd4f80a4610 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5101071f0 0x7fd510100480 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5101048c0 0x7fd5100fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 --2- 172.21.15.8:0/307685803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd510101f90 0x7fd510101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 >> 172.21.15.8:0/307685803 conn(0x7fd5100fb980 msgr2=0x7fd510102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 shutdown_connections 2024-02-19T16:22:44.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.367+0000 7fd51684e640 1 -- 172.21.15.8:0/307685803 wait complete. 2024-02-19T16:22:44.379 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:22:44.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- 172.21.15.8:0/689918880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 msgr2=0x7fe148108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/689918880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe148108850 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe138009ca0 tx=0x7fe13802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- 172.21.15.8:0/689918880 shutdown_connections 2024-02-19T16:22:44.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/689918880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe148108850 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/689918880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe148101600 0x7fe148101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/689918880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe148100410 0x7fe148100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- 172.21.15.8:0/689918880 >> 172.21.15.8:0/689918880 conn(0x7fe1480fb9a0 msgr2=0x7fe1480fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- 172.21.15.8:0/689918880 shutdown_connections 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- 172.21.15.8:0/689918880 wait complete. 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 Processor -- start 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- start start 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe148100410 0x7fe14806a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe148101600 0x7fe14819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe14819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14819f090 con 0x7fe148101fc0 2024-02-19T16:22:44.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14819f200 con 0x7fe148101600 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.482+0000 7fe14d2c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14819f3c0 con 0x7fe148100410 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe14819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe14819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49692/0 (socket says 172.21.15.8:49692) 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe148100410 0x7fe14806a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 -- 172.21.15.8:0/309727549 learned_addr learned my addr 172.21.15.8:0/309727549 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1467fc640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe148101600 0x7fe14819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 -- 172.21.15.8:0/309727549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe148100410 msgr2=0x7fe14806a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe148100410 0x7fe14806a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 -- 172.21.15.8:0/309727549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe148101600 msgr2=0x7fe14819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe148101600 0x7fe14819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 -- 172.21.15.8:0/309727549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009490 con 0x7fe148101fc0 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe1477fe640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe14819e8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe13802c630 tx=0x7fe138032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138034e20 con 0x7fe148101fc0 2024-02-19T16:22:44.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe138003e50 con 0x7fe148101fc0 2024-02-19T16:22:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe138009900 con 0x7fe148101fc0 2024-02-19T16:22:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138004880 con 0x7fe148101fc0 2024-02-19T16:22:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.483+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14819f950 con 0x7fe148101fc0 2024-02-19T16:22:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.484+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe148107730 con 0x7fe148101fc0 2024-02-19T16:22:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.485+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fe13800a040 con 0x7fe148101fc0 2024-02-19T16:22:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.486+0000 7fe13ffff640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe1340a21d0 0x7fe1340a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.486+0000 7fe146ffd640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe1340a21d0 0x7fe1340a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.486+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fe1380f0e10 con 0x7fe148101fc0 2024-02-19T16:22:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.486+0000 7fe146ffd640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe1340a21d0 0x7fe1340a4690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe148101460 tx=0x7fe140005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.491+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe1380f4030 con 0x7fe148101fc0 2024-02-19T16:22:44.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.675+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe14810c020 con 0x7fe148101fc0 2024-02-19T16:22:44.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.677+0000 7fe13ffff640 1 -- 172.21.15.8:0/309727549 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24685 (secure 0 0 0) 0x7fe1380dbe60 con 0x7fe148101fc0 2024-02-19T16:22:44.678 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:22:44.678 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":95,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:22:22.345953+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe1340a21d0 msgr2=0x7fe1340a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe1340a21d0 0x7fe1340a4690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe148101460 tx=0x7fe140005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 msgr2=0x7fe14819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe14819e8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe13802c630 tx=0x7fe138032f80 comp rx=0 tx=0).stop 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 shutdown_connections 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe1340a21d0 0x7fe1340a4690 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe148101fc0 0x7fe14819e8b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe148101600 0x7fe14819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 --2- 172.21.15.8:0/309727549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe148100410 0x7fe14806a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 >> 172.21.15.8:0/309727549 conn(0x7fe1480fb9a0 msgr2=0x7fe148104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 shutdown_connections 2024-02-19T16:22:44.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.679+0000 7fe14d2c7640 1 -- 172.21.15.8:0/309727549 wait complete. 2024-02-19T16:22:44.690 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:44 smithi008 ceph-mon[32573]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-19T16:22:44.690 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3332403508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.690 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3549922516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.690 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/307685803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:44.691 DEBUG:tasks.fwd_scrub:Yielding 2024-02-19T16:22:44.692 INFO:teuthology.run_tasks:Running task exec... 2024-02-19T16:22:44.701 INFO:teuthology.task.exec:Executing custom commands... 2024-02-19T16:22:44.702 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:22:44.702 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-19T16:22:44.702 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-19T16:22:44.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:22:44.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:22:44.703 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:22:44.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:44 smithi191 ceph-mon[37203]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-19T16:22:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3332403508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3549922516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/307685803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 -- 172.21.15.8:0/1361227076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c102f10 msgr2=0x7f7a9c1097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/1361227076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c102f10 0x7f7a9c1097a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009ca0 tx=0x7f7a9402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 -- 172.21.15.8:0/1361227076 shutdown_connections 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/1361227076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c102f10 0x7f7a9c1097a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/1361227076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a9c102550 0x7f7a9c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/1361227076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c101360 0x7f7a9c101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 -- 172.21.15.8:0/1361227076 >> 172.21.15.8:0/1361227076 conn(0x7f7a9c0fc8d0 msgr2=0x7f7a9c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 -- 172.21.15.8:0/1361227076 shutdown_connections 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.805+0000 7f7a9bfff640 1 -- 172.21.15.8:0/1361227076 wait complete. 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 Processor -- start 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 -- start start 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 0x7f7a9c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102550 0x7f7a9c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a9c102f10 0x7f7a9c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c19ee90 con 0x7f7a9c101360 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c19f000 con 0x7f7a9c102f10 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9bfff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c19f1c0 con 0x7f7a9c102550 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 0x7f7a9c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 0x7f7a9c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49714/0 (socket says 172.21.15.8:49714) 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 -- 172.21.15.8:0/3882545326 learned_addr learned my addr 172.21.15.8:0/3882545326 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a93fff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102550 0x7f7a9c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9b7fe640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a9c102f10 0x7f7a9c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 -- 172.21.15.8:0/3882545326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102550 msgr2=0x7f7a9c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102550 0x7f7a9c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 -- 172.21.15.8:0/3882545326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a9c102f10 msgr2=0x7f7a9c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a9c102f10 0x7f7a9c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.806+0000 7f7a9affd640 1 -- 172.21.15.8:0/3882545326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009900 con 0x7f7a9c101360 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.807+0000 7f7a9affd640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 0x7f7a9c1973a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00e9c0 tx=0x7f7a8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.807+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c009700 con 0x7f7a9c101360 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.807+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c19f4a0 con 0x7f7a9c101360 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.807+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a8c00cd80 con 0x7f7a9c101360 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.807+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c010430 con 0x7f7a9c101360 2024-02-19T16:22:44.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.807+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c19fa00 con 0x7f7a9c101360 2024-02-19T16:22:44.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.808+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c108720 con 0x7f7a9c101360 2024-02-19T16:22:44.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 -- 172.21.15.8:0/1180047486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c1030d0 msgr2=0x7fab9c109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 --2- 172.21.15.8:0/1180047486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c1030d0 0x7fab9c109960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab94009be0 tx=0x7fab9402bef0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 -- 172.21.15.8:0/1180047486 shutdown_connections 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 --2- 172.21.15.8:0/1180047486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c1030d0 0x7fab9c109960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 --2- 172.21.15.8:0/1180047486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c102710 0x7fab9c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 --2- 172.21.15.8:0/1180047486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c101520 0x7fab9c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 -- 172.21.15.8:0/1180047486 >> 172.21.15.8:0/1180047486 conn(0x7fab9c0fca90 msgr2=0x7fab9c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 -- 172.21.15.8:0/1180047486 shutdown_connections 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7faba407b640 1 -- 172.21.15.8:0/1180047486 wait complete. 2024-02-19T16:22:44.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.809+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7f7a8c003680 con 0x7f7a9c101360 2024-02-19T16:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 Processor -- start 2024-02-19T16:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 -- start start 2024-02-19T16:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7f7a98ff9640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a880a2140 0x7f7a880a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 0x7fab9c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c102710 0x7fab9c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7f7a93fff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a880a2140 0x7f7a880a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f7a8c014070 con 0x7f7a9c101360 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1030d0 0x7fab9c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c19f090 con 0x7fab9c1030d0 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c19f200 con 0x7fab9c102710 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba407b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c19f3c0 con 0x7fab9c101520 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba1e17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 0x7fab9c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba1e17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 0x7fab9c1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:57756/0 (socket says 172.21.15.8:57756) 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba1e17640 1 -- 172.21.15.8:0/2761652061 learned_addr learned my addr 172.21.15.8:0/2761652061 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba1616640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c102710 0x7fab9c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba2618640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1030d0 0x7fab9c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba1e17640 1 -- 172.21.15.8:0/2761652061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c102710 msgr2=0x7fab9c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.810+0000 7faba1e17640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c102710 0x7fab9c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7faba1e17640 1 -- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1030d0 msgr2=0x7fab9c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7faba1e17640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1030d0 0x7fab9c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7faba1e17640 1 -- 172.21.15.8:0/2761652061 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009560 con 0x7fab9c101520 2024-02-19T16:22:44.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7f7a93fff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a880a2140 0x7f7a880a4600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c1988b0 tx=0x7f7a84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7faba1e17640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 0x7fab9c1975a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e9a0 tx=0x7fab9800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7fab92ffd640 1 -- 172.21.15.8:0/2761652061 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800cd80 con 0x7fab9c101520 2024-02-19T16:22:44.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab94009840 con 0x7fab9c101520 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7fab92ffd640 1 -- 172.21.15.8:0/2761652061 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab98010910 con 0x7fab9c101520 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7fab92ffd640 1 -- 172.21.15.8:0/2761652061 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98018960 con 0x7fab9c101520 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.811+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c19fa70 con 0x7fab9c101520 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.813+0000 7fab92ffd640 1 -- 172.21.15.8:0/2761652061 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fab98018ac0 con 0x7fab9c101520 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.813+0000 7faba6c4b740 1 Processor -- start 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 -- start start 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c911358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91136b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c9113d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c910a4f90 con 0x560c91137050 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c910a24c0 con 0x560c91136690 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba6c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9109dca0 con 0x560c911354c0 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c9113d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7fab92ffd640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab880a2150 0x7fab880a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba2618640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c911358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba2618640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c911358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:57758/0 (socket says 172.21.15.8:57758) 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1e17640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba2618640 1 -- 172.21.15.8:0/1749777932 learned_addr learned my addr 172.21.15.8:0/1749777932 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba2618640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab880a2150 0x7fab880a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 -- 172.21.15.8:0/1749777932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 msgr2=0x560c911358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c911358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7fab92ffd640 1 -- 172.21.15.8:0/2761652061 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fab98014070 con 0x7fab9c101520 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 -- 172.21.15.8:0/1749777932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 msgr2=0x560c91136b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91136b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 -- 172.21.15.8:0/1749777932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c910600b0 con 0x560c91137050 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.814+0000 7faba1616640 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c9113d8e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009bf0 tx=0x7fab8c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ff728e9b7861c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.815+0000 7faba2618640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab880a2150 0x7fab880a4610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x560c91136590 tx=0x7fab9402bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.815+0000 7fab87fff640 1 -- 172.21.15.8:0/1749777932 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c004050 con 0x560c91137050 2024-02-19T16:22:44.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.815+0000 7fab87fff640 1 -- 172.21.15.8:0/1749777932 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab8c005500 con 0x560c91137050 2024-02-19T16:22:44.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.815+0000 7fab87fff640 1 -- 172.21.15.8:0/1749777932 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c004d90 con 0x560c91137050 2024-02-19T16:22:44.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.815+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7a8c097060 con 0x7f7a9c101360 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 -- 172.21.15.8:0/1749777932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 msgr2=0x560c9113d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c9113d8e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009bf0 tx=0x7fab8c011810 comp rx=0 tx=0).stop 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 -- 172.21.15.8:0/1749777932 shutdown_connections 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c9113d8e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91136b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 --2- 172.21.15.8:0/1749777932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c911358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 -- 172.21.15.8:0/1749777932 >> 172.21.15.8:0/1749777932 conn(0x560c911306a0 msgr2=0x560c91130aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 -- 172.21.15.8:0/1749777932 shutdown_connections 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.816+0000 7faba6c4b740 1 -- 172.21.15.8:0/1749777932 wait complete. 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 -- start start 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 Processor -- start 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c9113f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c9113fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91146c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c91042f40 con 0x560c91137050 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c90d714b0 con 0x560c91136690 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c91023670 con 0x560c911354c0 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba6c4b740 10 client.0 _ll_register_callbacks cb 0x560c91090f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba2618640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba1e17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c9113f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba2618640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91146c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49734/0 (socket says 172.21.15.8:49734) 2024-02-19T16:22:44.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba2618640 1 -- 172.21.15.8:0/3317834349 learned_addr learned my addr 172.21.15.8:0/3317834349 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.817+0000 7faba1616640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c9113fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba1e17640 1 -- 172.21.15.8:0/3317834349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 msgr2=0x560c9113fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba1e17640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c9113fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba1e17640 1 -- 172.21.15.8:0/3317834349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 msgr2=0x560c91146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba1e17640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba1e17640 1 -- 172.21.15.8:0/3317834349 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c009850 con 0x560c911354c0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba2618640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba1e17640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c9113f6b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fab980cbca0 tx=0x7fab98013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3baffe0bcfe053fe server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7fab857fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7fab857fa640 1 -- 172.21.15.8:0/3317834349 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9809aec0 con 0x560c911354c0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7fab857fa640 1 -- 172.21.15.8:0/3317834349 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fab9809f010 con 0x560c911354c0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c91077f40 con 0x560c911354c0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7fab857fa640 1 -- 172.21.15.8:0/3317834349 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab980a3660 con 0x560c911354c0 2024-02-19T16:22:44.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 10 client.25477 fetch_fsmap learned FSMap version 24 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560c9109c320 con 0x560c911354c0 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7fab857fa640 1 -- 172.21.15.8:0/3317834349 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560c91077f40 con 0x560c911354c0 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7fab857fa640 1 -- 172.21.15.8:0/3317834349 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fab980cd070 con 0x560c911354c0 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 10 client.25477 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 10 client.25477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 20 client.25477 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c9114b950 0x560c9114dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 4 client.25477 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.818+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c91094310 con 0x560c9114b950 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.819+0000 7faba1616640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c9114b950 0x560c9114dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.819+0000 7faba1616640 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c9114b950 0x560c9114dd60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc51af12dc6d141 server_cookie=63eea57881aa2d15 in_seq=0 out_seq=0 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.819+0000 7fab857fa640 10 client.25477 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7fab857fa640 1 -- 172.21.15.8:0/3317834349 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560c91094310 con 0x560c9114b950 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7fab857fa640 10 client.25477 handle_command_reply: tid=0 2024-02-19T16:22:44.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 client.25477 shutdown 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 msgr2=0x560c9113f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c9113f6b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fab980cbca0 tx=0x7fab98013000 comp rx=0 tx=0).stop 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 shutdown_connections 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c9113fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c9114b950 0x560c9114dd60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 --2- 172.21.15.8:0/3317834349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c9113f6b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7fab857fa640 0 client.25477 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 >> 172.21.15.8:0/3317834349 conn(0x560c911306a0 msgr2=0x560c9112f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 shutdown_connections 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 1 -- 172.21.15.8:0/3317834349 wait complete. 2024-02-19T16:22:44.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.820+0000 7faba6c4b740 20 client.25477 trim_cache size 0 max 16384 2024-02-19T16:22:44.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:44 smithi156 ceph-mon[36502]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-19T16:22:44.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3332403508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3549922516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:22:44.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/307685803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.836+0000 7faba6c4b740 1 Processor -- start 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 -- start start 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c91130a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91152600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c91130060 con 0x560c91137050 2024-02-19T16:22:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c91094310 con 0x560c91136690 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba6c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c91077f40 con 0x560c911354c0 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49746/0 (socket says 172.21.15.8:49746) 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 -- 172.21.15.8:0/2001054228 learned_addr learned my addr 172.21.15.8:0/2001054228 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba1616640 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba1e17640 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c91130a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 -- 172.21.15.8:0/2001054228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 msgr2=0x560c91130a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c91130a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 -- 172.21.15.8:0/2001054228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 msgr2=0x560c91146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 -- 172.21.15.8:0/2001054228 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c91038c00 con 0x560c91137050 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7faba2618640 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91152600 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fab9402e010 tx=0x7fab9402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5a58e757a96d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.837+0000 7fab877fe640 1 -- 172.21.15.8:0/2001054228 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94033e80 con 0x560c91137050 2024-02-19T16:22:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7fab877fe640 1 -- 172.21.15.8:0/2001054228 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab94034430 con 0x560c91137050 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7fab877fe640 1 -- 172.21.15.8:0/2001054228 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9404e6c0 con 0x560c91137050 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 -- 172.21.15.8:0/2001054228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 msgr2=0x560c91152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91152600 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fab9402e010 tx=0x7fab9402e040 comp rx=0 tx=0).stop 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 -- 172.21.15.8:0/2001054228 shutdown_connections 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91137050 0x560c91152600 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c91136690 0x560c91146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2001054228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c911354c0 0x560c91130a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 -- 172.21.15.8:0/2001054228 >> 172.21.15.8:0/2001054228 conn(0x560c911306a0 msgr2=0x560c91134190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 -- 172.21.15.8:0/2001054228 shutdown_connections 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.838+0000 7faba6c4b740 1 -- 172.21.15.8:0/2001054228 wait complete. 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 -- start start 2024-02-19T16:22:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 Processor -- start 2024-02-19T16:22:44.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c911354c0 0x560c91146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91136690 0x560c91146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c91137050 0x560c91144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c91145380 con 0x560c91136690 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c911454f0 con 0x560c911354c0 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c911456b0 con 0x560c91137050 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba6c4b740 10 client.0 _ll_register_callbacks cb 0x7fab980046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1e17640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c911354c0 0x560c91146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1616640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91136690 0x560c91146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1e17640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c911354c0 0x560c91146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:33128/0 (socket says 172.21.15.8:33128) 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1e17640 1 -- 172.21.15.8:0/2456810528 learned_addr learned my addr 172.21.15.8:0/2456810528 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba2618640 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c91137050 0x560c91144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1616640 1 -- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c91137050 msgr2=0x560c91144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1616640 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c91137050 0x560c91144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1616640 1 -- 172.21.15.8:0/2456810528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c911354c0 msgr2=0x560c91146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1616640 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c911354c0 0x560c91146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.839+0000 7faba1616640 1 -- 172.21.15.8:0/2456810528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab94048040 con 0x560c91136690 2024-02-19T16:22:44.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba1616640 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91136690 0x560c91146b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab8c01b040 tx=0x7fab8c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b163cbba581c168f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7fab857fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7fab857fa640 1 -- 172.21.15.8:0/2456810528 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c02a040 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7fab857fa640 1 -- 172.21.15.8:0/2456810528 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fab8c004540 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c910c3900 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7fab857fa640 1 -- 172.21.15.8:0/2456810528 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab8c0043a0 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 10 client.15603 fetch_fsmap learned FSMap version 24 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560c9105e910 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7fab857fa640 1 -- 172.21.15.8:0/2456810528 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560c910c3900 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7fab857fa640 1 -- 172.21.15.8:0/2456810528 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fab8c034020 con 0x560c91136690 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 10 client.15603 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 20 client.15603 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:44.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c911a86a0 0x560c911aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 4 client.15603 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.840+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560c90d9d860 con 0x560c911a86a0 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.841+0000 7faba1e17640 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c911a86a0 0x560c911aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.841+0000 7fab857fa640 10 client.15603 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.841+0000 7faba1e17640 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c911a86a0 0x560c911aaac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9685377fb15a46 server_cookie=a5438dfad730f09e in_seq=0 out_seq=0 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.842+0000 7fab857fa640 1 -- 172.21.15.8:0/2456810528 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560c90d9d860 con 0x560c911a86a0 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.842+0000 7fab857fa640 10 client.15603 handle_command_reply: tid=0 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.842+0000 7faba6c4b740 1 client.15603 shutdown 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.842+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91136690 msgr2=0x560c91146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.842+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91136690 0x560c91146b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab8c01b040 tx=0x7fab8c011d10 comp rx=0 tx=0).stop 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 shutdown_connections 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560c911a86a0 0x560c911aaac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c91137050 0x560c91144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7fab857fa640 0 client.15603 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560c91136690 0x560c91146b80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 --2- 172.21.15.8:0/2456810528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c911354c0 0x560c91146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 >> 172.21.15.8:0/2456810528 conn(0x560c911306a0 msgr2=0x560c91134190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 shutdown_connections 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 1 -- 172.21.15.8:0/2456810528 wait complete. 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba6c4b740 20 client.15603 trim_cache size 0 max 16384 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:22:44.849 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "8cd53f13-3cfd-4dff-9585-cd6587566340", 2024-02-19T16:22:44.850 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:22:44.850 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:44.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab880a2150 msgr2=0x7fab880a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab880a2150 0x7fab880a4610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x560c91136590 tx=0x7fab9402bef0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 msgr2=0x7fab9c1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 0x7fab9c1975a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e9a0 tx=0x7fab9800ee60 comp rx=0 tx=0).stop 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 shutdown_connections 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fab880a2150 0x7fab880a4610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1030d0 0x7fab9c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c102710 0x7fab9c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 --2- 172.21.15.8:0/2761652061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101520 0x7fab9c1975a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 >> 172.21.15.8:0/2761652061 conn(0x7fab9c0fca90 msgr2=0x7fab9c1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 shutdown_connections 2024-02-19T16:22:44.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.843+0000 7faba407b640 1 -- 172.21.15.8:0/2761652061 wait complete. 2024-02-19T16:22:44.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:22:44.855 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 -- 172.21.15.8:0/412284098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 msgr2=0x7fe6dc1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 --2- 172.21.15.8:0/412284098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc1088a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009be0 tx=0x7fe6d402bef0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 -- 172.21.15.8:0/412284098 shutdown_connections 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 --2- 172.21.15.8:0/412284098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc1088a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 --2- 172.21.15.8:0/412284098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 0x7fe6dc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 --2- 172.21.15.8:0/412284098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 -- 172.21.15.8:0/412284098 >> 172.21.15.8:0/412284098 conn(0x7fe6dc0fb0d0 msgr2=0x7fe6dc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 -- 172.21.15.8:0/412284098 shutdown_connections 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.958+0000 7fe6e300e640 1 -- 172.21.15.8:0/412284098 wait complete. 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 Processor -- start 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 -- start start 2024-02-19T16:22:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 0x7fe6dc1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19efd0 con 0x7fe6dc1019f0 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19f140 con 0x7fe6dc101000 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e300e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19f300 con 0x7fe6dc100640 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e0daa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 0x7fe6dc1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49780/0 (socket says 172.21.15.8:49780) 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.959+0000 7fe6e15ab640 1 -- 172.21.15.8:0/368393792 learned_addr learned my addr 172.21.15.8:0/368393792 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6dbfff640 1 -- 172.21.15.8:0/368393792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 msgr2=0x7fe6dc1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 0x7fe6dc1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6dbfff640 1 -- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 msgr2=0x7fe6dc19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6dbfff640 1 -- 172.21.15.8:0/368393792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009490 con 0x7fe6dc101000 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 0x7fe6dc1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc197a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9c0 tx=0x7fe6c800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6d9ffb640 1 -- 172.21.15.8:0/368393792 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800ccc0 con 0x7fe6dc101000 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6d9ffb640 1 -- 172.21.15.8:0/368393792 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c800ce20 con 0x7fe6dc101000 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d4009840 con 0x7fe6dc101000 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6d9ffb640 1 -- 172.21.15.8:0/368393792 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80187e0 con 0x7fe6dc101000 2024-02-19T16:22:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.960+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc19f960 con 0x7fe6dc101000 2024-02-19T16:22:44.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 Processor -- start 2024-02-19T16:22:44.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 -- start start 2024-02-19T16:22:44.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d968eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9693bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d96966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d95fa280 con 0x55c5d968f0d0 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d95f77b0 con 0x55c5d968e6e0 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d95f2f90 con 0x55c5d9694240 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9693bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d968eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9693bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49794/0 (socket says 172.21.15.8:49794) 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 -- 172.21.15.8:0/684537409 learned_addr learned my addr 172.21.15.8:0/684537409 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6d9ffb640 1 -- 172.21.15.8:0/368393792 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 136818+0+0 (secure 0 0 0) 0x7fe6c8010430 con 0x7fe6dc101000 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 -- 172.21.15.8:0/684537409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 msgr2=0x55c5d96966d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d96966d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 -- 172.21.15.8:0/684537409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 msgr2=0x55c5d968eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d968eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 -- 172.21.15.8:0/684537409 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5d95b1570 con 0x55c5d968f0d0 2024-02-19T16:22:44.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.963+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9693bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00114f0 tx=0x7fe6d00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0b9b4e97d4be9e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6c6ffd640 1 -- 172.21.15.8:0/684537409 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0005020 con 0x55c5d968f0d0 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6c6ffd640 1 -- 172.21.15.8:0/684537409 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0005660 con 0x55c5d968f0d0 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6c6ffd640 1 -- 172.21.15.8:0/684537409 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000f8c0 con 0x55c5d968f0d0 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6d9ffb640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6cc0a2150 0x7fe6cc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6cc0a2150 0x7fe6cc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6d9ffb640 1 -- 172.21.15.8:0/368393792 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fe6c8014070 con 0x7fe6dc101000 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6cc0a2150 0x7fe6cc0a4610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55c5d968f930 tx=0x7fe6d4009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6e5beb740 1 -- 172.21.15.8:0/684537409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 msgr2=0x55c5d9693bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.964+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9693bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00114f0 tx=0x7fe6d00119b0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- 172.21.15.8:0/684537409 shutdown_connections 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d96966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9693bd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/684537409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d968eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- 172.21.15.8:0/684537409 >> 172.21.15.8:0/684537409 conn(0x55c5d96857f0 msgr2=0x55c5d9685bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- 172.21.15.8:0/684537409 shutdown_connections 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- 172.21.15.8:0/684537409 wait complete. 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- start start 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 Processor -- start 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d96986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9698c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d969fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d92c6430 con 0x55c5d968f0d0 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9282e00 con 0x55c5d968e6e0 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9577fd0 con 0x55c5d9694240 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6e5beb740 10 client.0 _ll_register_callbacks cb 0x55c5d95cdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.965+0000 7fe6dbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9698c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d96986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6dbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9698c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49808/0 (socket says 172.21.15.8:49808) 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6dbfff640 1 -- 172.21.15.8:0/3611101856 learned_addr learned my addr 172.21.15.8:0/3611101856 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d969fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 -- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 msgr2=0x55c5d969fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d969fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 -- 172.21.15.8:0/3611101856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 msgr2=0x55c5d9698c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9698c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 -- 172.21.15.8:0/3611101856 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d0009cc0 con 0x55c5d968e6e0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d96986d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c5d96867b0 tx=0x7fe6d0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a64d37b395eb785 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6bbfff640 1 -- 172.21.15.8:0/3611101856 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004be0 con 0x55c5d968e6e0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5d92f27e0 con 0x55c5d968e6e0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6bbfff640 1 -- 172.21.15.8:0/3611101856 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fe6d0004020 con 0x55c5d968e6e0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6bbfff640 1 -- 172.21.15.8:0/3611101856 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d00041e0 con 0x55c5d968e6e0 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e5beb740 10 client.25394 fetch_fsmap learned FSMap version 24 2024-02-19T16:22:44.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.966+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c5d95f1610 con 0x55c5d968e6e0 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6bbfff640 1 -- 172.21.15.8:0/3611101856 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5d92f27e0 con 0x55c5d968e6e0 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6bbfff640 1 -- 172.21.15.8:0/3611101856 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe6d0003780 con 0x55c5d968e6e0 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6e5beb740 10 client.25394 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6e5beb740 10 client.25394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6e5beb740 20 client.25394 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d96a4760 0x55c5d96a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6e5beb740 4 client.25394 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5d932f720 con 0x55c5d96a4760 2024-02-19T16:22:44.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d96a4760 0x55c5d96a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6bbfff640 10 client.25394 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.967+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d96a4760 0x55c5d96a6b50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1596da496f4f0d8b server_cookie=6777aaf7251582c6 in_seq=0 out_seq=0 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6bbfff640 1 -- 172.21.15.8:0/3611101856 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5d932f720 con 0x55c5d96a4760 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6bbfff640 10 client.25394 handle_command_reply: tid=0 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 client.25394 shutdown 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 msgr2=0x55c5d96986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d96986d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c5d96867b0 tx=0x7fe6d0005e80 comp rx=0 tx=0).stop 2024-02-19T16:22:44.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 shutdown_connections 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d96a4760 0x55c5d96a6b50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d9694240 0x55c5d969fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6bbfff640 0 client.25394 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d968f0d0 0x55c5d9698c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/3611101856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d96986d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 >> 172.21.15.8:0/3611101856 conn(0x55c5d96857f0 msgr2=0x55c5d9684a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 shutdown_connections 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 1 -- 172.21.15.8:0/3611101856 wait complete. 2024-02-19T16:22:44.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.968+0000 7fe6e5beb740 20 client.25394 trim_cache size 0 max 16384 2024-02-19T16:22:44.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.984+0000 7fe6e5beb740 1 Processor -- start 2024-02-19T16:22:44.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.984+0000 7fe6e5beb740 1 -- start start 2024-02-19T16:22:44.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.984+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d9689110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.984+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968f0d0 0x55c5d969fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.984+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d9697cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d968f7c0 con 0x55c5d9694240 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9681320 con 0x55c5d968e6e0 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d96a1140 con 0x55c5d968f0d0 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d9697cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d9697cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49812/0 (socket says 172.21.15.8:49812) 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 -- 172.21.15.8:0/2394758913 learned_addr learned my addr 172.21.15.8:0/2394758913 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d9689110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968f0d0 0x55c5d969fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:44.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 -- 172.21.15.8:0/2394758913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968f0d0 msgr2=0x55c5d969fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968f0d0 0x55c5d969fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 -- 172.21.15.8:0/2394758913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 msgr2=0x55c5d9689110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d9689110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 -- 172.21.15.8:0/2394758913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5d958dac0 con 0x55c5d9694240 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d9697cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe6d402c430 tx=0x7fe6d402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235334dbca1a417b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6c67fc640 1 -- 172.21.15.8:0/2394758913 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d404e070 con 0x55c5d9694240 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6c67fc640 1 -- 172.21.15.8:0/2394758913 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d4033e50 con 0x55c5d9694240 2024-02-19T16:22:44.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.985+0000 7fe6c67fc640 1 -- 172.21.15.8:0/2394758913 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4034b40 con 0x55c5d9694240 2024-02-19T16:22:45.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 -- 172.21.15.8:0/2394758913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 msgr2=0x55c5d9697cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d9697cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe6d402c430 tx=0x7fe6d402e070 comp rx=0 tx=0).stop 2024-02-19T16:22:45.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 -- 172.21.15.8:0/2394758913 shutdown_connections 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d9697cf0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968f0d0 0x55c5d969fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/2394758913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968e6e0 0x55c5d9689110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 -- 172.21.15.8:0/2394758913 >> 172.21.15.8:0/2394758913 conn(0x55c5d96857f0 msgr2=0x55c5d9684a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 -- 172.21.15.8:0/2394758913 shutdown_connections 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.986+0000 7fe6e5beb740 1 -- 172.21.15.8:0/2394758913 wait complete. 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 -- start start 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 Processor -- start 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 0x55c5d969faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968f0d0 0x55c5d969e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d969e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d969ed10 con 0x55c5d9694240 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d969ee80 con 0x55c5d968f0d0 2024-02-19T16:22:45.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d96fffe0 con 0x55c5d968e6e0 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e5beb740 10 client.0 _ll_register_callbacks cb 0x7fe6c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d969e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e0daa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 0x55c5d969faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968f0d0 0x55c5d969e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e0daa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 0x55c5d969faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:57824/0 (socket says 172.21.15.8:57824) 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e15ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d969e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49826/0 (socket says 172.21.15.8:49826) 2024-02-19T16:22:45.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.987+0000 7fe6e0daa640 1 -- 172.21.15.8:0/1032379837 learned_addr learned my addr 172.21.15.8:0/1032379837 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e0daa640 1 -- 172.21.15.8:0/1032379837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968f0d0 msgr2=0x55c5d969e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968f0d0 0x55c5d969e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e0daa640 1 -- 172.21.15.8:0/1032379837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 msgr2=0x55c5d969e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d969e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e0daa640 1 -- 172.21.15.8:0/1032379837 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc109130 con 0x55c5d968e6e0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e15ab640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d969e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968f0d0 0x55c5d969e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e0daa640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 0x55c5d969faf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0024480 tx=0x7fe6d00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de63a48ac60425c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6bbfff640 1 -- 172.21.15.8:0/1032379837 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004a60 con 0x55c5d968e6e0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6bbfff640 1 -- 172.21.15.8:0/1032379837 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fe6d0000e40 con 0x55c5d968e6e0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6d0005660 con 0x55c5d968e6e0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6bbfff640 1 -- 172.21.15.8:0/1032379837 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d0004bc0 con 0x55c5d968e6e0 2024-02-19T16:22:45.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e5beb740 10 client.25498 fetch_fsmap learned FSMap version 24 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c5d95b36a0 con 0x55c5d968e6e0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6bbfff640 1 -- 172.21.15.8:0/1032379837 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0005660 con 0x55c5d968e6e0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.988+0000 7fe6bbfff640 1 -- 172.21.15.8:0/1032379837 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe6d0024680 con 0x55c5d968e6e0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 10 client.25498 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 20 client.25498 populate_metadata read hostname 'smithi008' 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d97015d0 0x55c5d97039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 4 client.25498 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c5d937e890 con 0x55c5d97015d0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d97015d0 0x55c5d97039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6bbfff640 10 client.25498 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6dbfff640 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d97015d0 0x55c5d97039f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f777165b78e118ec server_cookie=71d8488495a280d2 in_seq=0 out_seq=0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6bbfff640 1 -- 172.21.15.8:0/1032379837 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c5d937e890 con 0x55c5d97015d0 2024-02-19T16:22:45.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6bbfff640 10 client.25498 handle_command_reply: tid=0 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 client.25498 shutdown 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 msgr2=0x55c5d969faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 0x55c5d969faf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0024480 tx=0x7fe6d00119b0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 shutdown_connections 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5d9694240 0x55c5d969e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5d968f0d0 0x55c5d969e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5d97015d0 0x55c5d97039f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 --2- 172.21.15.8:0/1032379837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5d968e6e0 0x55c5d969faf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6bbfff640 0 client.25498 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.989+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 >> 172.21.15.8:0/1032379837 conn(0x55c5d96857f0 msgr2=0x55c5d96ab410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 shutdown_connections 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e5beb740 1 -- 172.21.15.8:0/1032379837 wait complete. 2024-02-19T16:22:45.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e5beb740 20 client.25498 trim_cache size 0 max 16384 2024-02-19T16:22:45.007 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:22:45.007 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-19T16:22:45.007 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:22:45.007 INFO:teuthology.orchestra.run.smithi008.stdout: "8cd53f13-3cfd-4dff-9585-cd6587566340": { 2024-02-19T16:22:45.007 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:22:45.008 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "8cd53f13-3cfd-4dff-9585-cd6587566340", 2024-02-19T16:22:45.008 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:22:45.008 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:22:45.008 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:22:45.008 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6cc0a2150 msgr2=0x7fe6cc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6cc0a2150 0x7fe6cc0a4610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55c5d968f930 tx=0x7fe6d4009280 comp rx=0 tx=0).stop 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 msgr2=0x7fe6dc197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc197a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9c0 tx=0x7fe6c800edc0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 shutdown_connections 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fe6cc0a2150 0x7fe6cc0a4610 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6dc1019f0 0x7fe6dc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6dc101000 0x7fe6dc197a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 --2- 172.21.15.8:0/368393792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6dc100640 0x7fe6dc1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 >> 172.21.15.8:0/368393792 conn(0x7fe6dc0fb0d0 msgr2=0x7fe6dc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 shutdown_connections 2024-02-19T16:22:45.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:44.990+0000 7fe6e300e640 1 -- 172.21.15.8:0/368393792 wait complete. 2024-02-19T16:22:45.011 INFO:tasks.cephfs.filesystem:scrub status for tag:8cd53f13-3cfd-4dff-9585-cd6587566340 - {'path': '/', 'tag': '8cd53f13-3cfd-4dff-9585-cd6587566340', 'options': 'recursive,force'} 2024-02-19T16:22:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.063+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f7a9c02d000 con 0x7f7a9c101360 2024-02-19T16:22:45.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.419+0000 7f7a98ff9640 1 -- 172.21.15.8:0/3882545326 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7f7a8c09a720 con 0x7f7a9c101360 2024-02-19T16:22:45.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.421+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a880a2140 msgr2=0x7f7a880a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.421+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a880a2140 0x7f7a880a4600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c1988b0 tx=0x7f7a84005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.421+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 msgr2=0x7f7a9c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.421+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 0x7f7a9c1973a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00e9c0 tx=0x7f7a8c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 shutdown_connections 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f7a880a2140 0x7f7a880a4600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a9c102f10 0x7f7a9c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102550 0x7f7a9c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 --2- 172.21.15.8:0/3882545326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a9c101360 0x7f7a9c1973a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 >> 172.21.15.8:0/3882545326 conn(0x7f7a9c0fc8d0 msgr2=0x7f7a9c1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 shutdown_connections 2024-02-19T16:22:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.422+0000 7f7a9bfff640 1 -- 172.21.15.8:0/3882545326 wait complete. 2024-02-19T16:22:45.443 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-19T16:22:45.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 -- 172.21.15.8:0/3129092700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 msgr2=0x7fbd70109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/3129092700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70109b10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ca0 tx=0x7fbd6802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 -- 172.21.15.8:0/3129092700 shutdown_connections 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/3129092700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70109b10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/3129092700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd700ff5a0 0x7fbd700ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/3129092700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd70102b30 0x7fbd70102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 -- 172.21.15.8:0/3129092700 >> 172.21.15.8:0/3129092700 conn(0x7fbd700fb140 msgr2=0x7fbd700fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 -- 172.21.15.8:0/3129092700 shutdown_connections 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.552+0000 7fbd76bf7640 1 -- 172.21.15.8:0/3129092700 wait complete. 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 Processor -- start 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 -- start start 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700ff5a0 0x7fbd70102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd70102b30 0x7fbd70100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70101510 con 0x7fbd700fff60 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70101680 con 0x7fbd700ff5a0 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd76bf7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701a14d0 con 0x7fbd70102b30 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd67fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd67fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51580/0 (socket says 172.21.15.8:51580) 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd67fff640 1 -- 172.21.15.8:0/2131117153 learned_addr learned my addr 172.21.15.8:0/2131117153 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd74993640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700ff5a0 0x7fbd70102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd75194640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd70102b30 0x7fbd70100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.553+0000 7fbd67fff640 1 -- 172.21.15.8:0/2131117153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd70102b30 msgr2=0x7fbd70100db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd67fff640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd70102b30 0x7fbd70100db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd67fff640 1 -- 172.21.15.8:0/2131117153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700ff5a0 msgr2=0x7fbd70102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd67fff640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700ff5a0 0x7fbd70102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd67fff640 1 -- 172.21.15.8:0/2131117153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c009490 con 0x7fbd700fff60 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd67fff640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70100870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000ea20 tx=0x7fbd6000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60009720 con 0x7fbd700fff60 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd6000cd80 con 0x7fbd700fff60 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd68009900 con 0x7fbd700fff60 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60010490 con 0x7fbd700fff60 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.554+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd701a1ab0 con 0x7fbd700fff60 2024-02-19T16:22:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.555+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd70108920 con 0x7fbd700fff60 2024-02-19T16:22:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.557+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 20) v1 ==== 136835+0+0 (secure 0 0 0) 0x7fbd60005020 con 0x7fbd700fff60 2024-02-19T16:22:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.557+0000 7fbd65ffb640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd580a2250 0x7fbd580a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.557+0000 7fbd74993640 1 -- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd580a2250 msgr2=0x7fbd580a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.558+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fbd60014070 con 0x7fbd700fff60 2024-02-19T16:22:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.558+0000 7fbd74993640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd580a2250 0x7fbd580a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:22:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.562+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbd60097100 con 0x7fbd700fff60 2024-02-19T16:22:45.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.748+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fbd7010cf40 con 0x7fbd700fff60 2024-02-19T16:22:45.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.752+0000 7fbd65ffb640 1 -- 172.21.15.8:0/2131117153 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd6009a7c0 con 0x7fbd700fff60 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd580a2250 msgr2=0x7fbd580a4710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd580a2250 0x7fbd580a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 msgr2=0x7fbd70100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70100870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000ea20 tx=0x7fbd6000ee20 comp rx=0 tx=0).stop 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 shutdown_connections 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fbd580a2250 0x7fbd580a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd70102b30 0x7fbd70100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd700fff60 0x7fbd70100870 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 --2- 172.21.15.8:0/2131117153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700ff5a0 0x7fbd70102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 >> 172.21.15.8:0/2131117153 conn(0x7fbd700fb140 msgr2=0x7fbd70105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 shutdown_connections 2024-02-19T16:22:45.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.754+0000 7fbd76bf7640 1 -- 172.21.15.8:0/2131117153 wait complete. 2024-02-19T16:22:45.765 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: ignoring --setuser ceph since I am not root 2024-02-19T16:22:45.765 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: ignoring --setgroup ceph since I am not root 2024-02-19T16:22:45.766 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/309727549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:45.766 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:22:45.766 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:22:45.766 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:22:45.766 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:22:45.766 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:45 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3882545326' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-19T16:22:45.767 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-19T16:22:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/309727549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:22:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:22:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:22:45.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:22:45.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3882545326' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-19T16:22:45.777 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: ignoring --setuser ceph since I am not root 2024-02-19T16:22:45.777 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: ignoring --setgroup ceph since I am not root 2024-02-19T16:22:45.777 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:45 smithi191 ceph-mgr[39064]: -- 172.21.15.191:0/265516056 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569120821a0 con 0x556912058c00 2024-02-19T16:22:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/309727549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:22:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:22:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:22:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:22:45.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:22:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3882545326' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-19T16:22:45.830 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: ignoring --setuser ceph since I am not root 2024-02-19T16:22:45.830 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: ignoring --setgroup ceph since I am not root 2024-02-19T16:22:45.830 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-mgr[37887]: -- 172.21.15.156:0/2136228925 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1d555e1a0 con 0x55d1d5534800 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 -- 172.21.15.8:0/3017982497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298101600 msgr2=0x7f8298101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 --2- 172.21.15.8:0/3017982497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298101600 0x7f8298101a80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8288009900 tx=0x7f828802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 -- 172.21.15.8:0/3017982497 shutdown_connections 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 --2- 172.21.15.8:0/3017982497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8298101fc0 0x7f8298108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 --2- 172.21.15.8:0/3017982497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298101600 0x7f8298101a80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 --2- 172.21.15.8:0/3017982497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8298100410 0x7f8298100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 -- 172.21.15.8:0/3017982497 >> 172.21.15.8:0/3017982497 conn(0x7f82980fb980 msgr2=0x7f82980fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:45.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 -- 172.21.15.8:0/3017982497 shutdown_connections 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.871+0000 7f829e812640 1 -- 172.21.15.8:0/3017982497 wait complete. 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 Processor -- start 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 -- start start 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 0x7f82981043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8298101600 0x7f82981029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8298101fc0 0x7f8298102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298103680 con 0x7f8298100410 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82981037f0 con 0x7f8298101fc0 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829e812640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82981a15a0 con 0x7f8298101600 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f8297fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 0x7f82981043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f8297fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 0x7f82981043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51600/0 (socket says 172.21.15.8:51600) 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f8297fff640 1 -- 172.21.15.8:0/1531150169 learned_addr learned my addr 172.21.15.8:0/1531150169 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f829cdaf640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8298101fc0 0x7f8298102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.872+0000 7f82977fe640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8298101600 0x7f82981029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f8297fff640 1 -- 172.21.15.8:0/1531150169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8298101600 msgr2=0x7f82981029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f8297fff640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8298101600 0x7f82981029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f8297fff640 1 -- 172.21.15.8:0/1531150169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8298101fc0 msgr2=0x7f8298102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f8297fff640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8298101fc0 0x7f8298102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f8297fff640 1 -- 172.21.15.8:0/1531150169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009560 con 0x7f8298100410 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f8297fff640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 0x7f82981043a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f828000e9f0 tx=0x7f828000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800096e0 con 0x7f8298100410 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82981a1850 con 0x7f8298100410 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f828000cd80 con 0x7f8298100410 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280010430 con 0x7f8298100410 2024-02-19T16:22:45.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.873+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82981a1ce0 con 0x7f8298100410 2024-02-19T16:22:45.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.874+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829810be20 con 0x7f8298100410 2024-02-19T16:22:45.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.875+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 20) v1 ==== 136835+0+0 (secure 0 0 0) 0x7f8280003680 con 0x7f8298100410 2024-02-19T16:22:45.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.876+0000 7f82957fa640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82840a2250 0x7f82840a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:45.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.876+0000 7f82977fe640 1 -- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82840a2250 msgr2=0x7f82840a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:45.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.876+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f8280014070 con 0x7f8298100410 2024-02-19T16:22:45.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.876+0000 7f82977fe640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82840a2250 0x7f82840a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:22:45.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:45.880+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8280097060 con 0x7f8298100410 2024-02-19T16:22:46.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.065+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f829810bfb0 con 0x7f8298100410 2024-02-19T16:22:46.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.069+0000 7f82957fa640 1 -- 172.21.15.8:0/1531150169 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) v1 ==== 133+0+0 (secure 0 0 0) 0x7f828009a720 con 0x7f8298100410 2024-02-19T16:22:46.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82840a2250 msgr2=0x7f82840a4710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:22:46.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82840a2250 0x7f82840a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 msgr2=0x7f82981043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 0x7f82981043a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f828000e9f0 tx=0x7f828000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 shutdown_connections 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7f82840a2250 0x7f82840a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8298101fc0 0x7f8298102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8298101600 0x7f82981029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 --2- 172.21.15.8:0/1531150169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8298100410 0x7f82981043a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 >> 172.21.15.8:0/1531150169 conn(0x7f82980fb980 msgr2=0x7f82980fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 shutdown_connections 2024-02-19T16:22:46.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.071+0000 7f829e812640 1 -- 172.21.15.8:0/1531150169 wait complete. 2024-02-19T16:22:46.083 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 -- 172.21.15.8:0/65664414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 msgr2=0x7fc934109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 --2- 172.21.15.8:0/65664414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc934109980 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009ca0 tx=0x7fc92c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 -- 172.21.15.8:0/65664414 shutdown_connections 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 --2- 172.21.15.8:0/65664414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc934109980 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 --2- 172.21.15.8:0/65664414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc934102730 0x7fc934102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 --2- 172.21.15.8:0/65664414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc934101540 0x7fc934101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 -- 172.21.15.8:0/65664414 >> 172.21.15.8:0/65664414 conn(0x7fc9340fcab0 msgr2=0x7fc9340feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 -- 172.21.15.8:0/65664414 shutdown_connections 2024-02-19T16:22:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 -- 172.21.15.8:0/65664414 wait complete. 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 Processor -- start 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.190+0000 7fc93a5d0640 1 -- start start 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc93a5d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc934101540 0x7fc934197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc93a5d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc934102730 0x7fc934197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc93a5d0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc93419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc93a5d0640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93419eff0 con 0x7fc9341030f0 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc93a5d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93419f160 con 0x7fc934101540 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc93a5d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93419f320 con 0x7fc934102730 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc93419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc933fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc934101540 0x7fc934197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc93419eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51606/0 (socket says 172.21.15.8:51606) 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 -- 172.21.15.8:0/1391921235 learned_addr learned my addr 172.21.15.8:0/1391921235 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc9337fe640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc934102730 0x7fc934197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc934102730 msgr2=0x7fc934197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc934102730 0x7fc934197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc934101540 msgr2=0x7fc934197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc934101540 0x7fc934197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 -- 172.21.15.8:0/1391921235 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009900 con 0x7fc9341030f0 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc934101540 0x7fc934197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.191+0000 7fc938b6d640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc93419eab0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc92c037040 tx=0x7fc92c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.192+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0329b0 con 0x7fc9341030f0 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.192+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93419f5a0 con 0x7fc9341030f0 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.192+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc92c003e50 con 0x7fc9341030f0 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.192+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0048a0 con 0x7fc9341030f0 2024-02-19T16:22:46.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.192+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93419faa0 con 0x7fc9341030f0 2024-02-19T16:22:46.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.193+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc934108800 con 0x7fc9341030f0 2024-02-19T16:22:46.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.194+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 20) v1 ==== 136835+0+0 (secure 0 0 0) 0x7fc92c00a040 con 0x7fc9341030f0 2024-02-19T16:22:46.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.195+0000 7fc9317fa640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:46.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.195+0000 7fc933fff640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:46.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.195+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fc92c0f1300 con 0x7fc9341030f0 2024-02-19T16:22:46.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.195+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T16:22:46.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.198+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fc92c0b9f10 con 0x7fc9341030f0 2024-02-19T16:22:46.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:45.996+0000 7f772c1cc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:22:46.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:46.095+0000 7f772c1cc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:22:46.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.396+0000 7fc933fff640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:46.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.396+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T16:22:46.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.403+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 --> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc93402d000 con 0x7fc9200a2250 2024-02-19T16:22:46.420 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:46.026+0000 7f05deefe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:22:46.420 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:46.127+0000 7f05deefe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:22:46.420 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:46.062+0000 7fceb59e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-19T16:22:46.420 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:46.183+0000 7fceb59e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-19T16:22:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3882545326' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-19T16:22:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-mon[32573]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-02-19T16:22:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2131117153' entity='client.admin' 2024-02-19T16:22:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1531150169' entity='client.admin' 2024-02-19T16:22:46.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3882545326' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-19T16:22:46.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-mon[37203]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-02-19T16:22:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2131117153' entity='client.admin' 2024-02-19T16:22:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1531150169' entity='client.admin' 2024-02-19T16:22:46.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.797+0000 7fc933fff640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:46.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:46.797+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-19T16:22:46.808 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3882545326' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-19T16:22:46.808 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-mon[36502]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-02-19T16:22:46.808 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2131117153' entity='client.admin' 2024-02-19T16:22:46.808 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1531150169' entity='client.admin' 2024-02-19T16:22:47.068 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:46.806+0000 7f772c1cc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:22:47.069 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:46.921+0000 7f772c1cc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:22:47.137 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:46.796+0000 7f05deefe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:22:47.137 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:46.893+0000 7f05deefe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:22:47.137 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.038+0000 7f05deefe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:22:47.178 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:46 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:46.903+0000 7fceb59e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-19T16:22:47.178 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.027+0000 7fceb59e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-19T16:22:47.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.067+0000 7f772c1cc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:22:47.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.165+0000 7f772c1cc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:22:47.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.250+0000 7f772c1cc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:22:47.495 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.136+0000 7f05deefe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:22:47.495 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.224+0000 7f05deefe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:22:47.526 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.177+0000 7fceb59e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-19T16:22:47.527 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.293+0000 7fceb59e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-19T16:22:47.527 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.406+0000 7fceb59e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-19T16:22:47.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:47.598+0000 7fc933fff640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:47.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:47.598+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-19T16:22:47.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.494+0000 7f05deefe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:22:47.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.607+0000 7f05deefe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:22:47.770 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.697+0000 7f05deefe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:22:47.818 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.520+0000 7f772c1cc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:22:47.819 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.632+0000 7f772c1cc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:22:47.819 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.720+0000 7f772c1cc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:22:48.026 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.763+0000 7fceb59e2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-19T16:22:48.027 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.873+0000 7fceb59e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-19T16:22:48.027 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:47.960+0000 7fceb59e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-19T16:22:48.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.817+0000 7f772c1cc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:22:48.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:47.999+0000 7f772c1cc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:22:48.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.795+0000 7f05deefe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:22:48.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:47.975+0000 7f05deefe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:22:48.526 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:48.074+0000 7fceb59e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-19T16:22:48.527 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:48.294+0000 7fceb59e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-19T16:22:49.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:49.199+0000 7fc933fff640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:49.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:49.199+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-19T16:22:49.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:48.978+0000 7f05deefe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:22:49.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:49.115+0000 7f05deefe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:22:49.329 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:49.029+0000 7f772c1cc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:22:49.330 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:49.161+0000 7f772c1cc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:22:49.759 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:49.459+0000 7f05deefe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:22:49.759 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:49.553+0000 7f05deefe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:22:49.776 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:49.396+0000 7fceb59e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-19T16:22:49.777 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:49.547+0000 7fceb59e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-19T16:22:49.798 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:49.499+0000 7f772c1cc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:22:49.798 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:49.589+0000 7f772c1cc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:22:50.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:49.758+0000 7f05deefe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:22:50.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:49.953+0000 7f05deefe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:22:50.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:49.796+0000 7f772c1cc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:22:50.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:49.988+0000 7f772c1cc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:22:50.219 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:49.921+0000 7fceb59e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-19T16:22:50.219 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:50.011+0000 7fceb59e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-19T16:22:50.515 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:50.250+0000 7f05deefe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:22:50.526 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:50.218+0000 7fceb59e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:22:50.526 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:50.410+0000 7fceb59e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-19T16:22:50.549 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:50.277+0000 7f772c1cc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:22:50.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:50.514+0000 7f05deefe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:22:50.769 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:50.761+0000 7f05deefe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:22:50.829 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:50.547+0000 7f772c1cc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:22:50.829 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:50.789+0000 7f772c1cc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:22:50.988 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:50.721+0000 7fceb59e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-19T16:22:51.276 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:50.985+0000 7fceb59e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-19T16:22:51.277 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:51.225+0000 7fceb59e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-19T16:22:52.019 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:51.604+0000 7f05deefe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:22:52.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:51.655+0000 7f772c1cc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:22:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:52.401+0000 7fc933fff640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/354789085 2024-02-19T16:22:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:52.401+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-19T16:22:52.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.192+0000 7f05deefe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:22:52.519 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.297+0000 7f05deefe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:22:52.526 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:52.058+0000 7fceb59e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-19T16:22:52.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:52.548+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 21) v1 ==== 136453+0+0 (secure 0 0 0) 0x7fc92c004040 con 0x7fc9341030f0 2024-02-19T16:22:52.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:52.549+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 msgr2=0x7fc9200a4710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T16:22:52.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:52.549+0000 7fc9317fa640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/354789085,v1:172.21.15.8:6801/354789085] conn(0x7fc9200a2250 0x7fc9200a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:52.579 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:52.242+0000 7f772c1cc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:22:52.579 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:52.342+0000 7f772c1cc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:22:52.797 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.526+0000 7f05deefe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:22:52.798 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: Active manager daemon x restarted 2024-02-19T16:22:52.798 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: Activating manager daemon x 2024-02-19T16:22:52.798 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: osdmap e96: 12 total, 12 up, 12 in 2024-02-19T16:22:52.798 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: mgrmap e21: x(active, starting, since 0.0116843s), standbys: z, y 2024-02-19T16:22:52.798 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:22:52.798 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:22:52.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:22:52.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T16:22:52.801 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T16:22:52.801 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T16:22:52.801 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-mon[32573]: Manager daemon x is now available 2024-02-19T16:22:52.845 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:52.721+0000 7fceb59e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-19T16:22:52.845 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: Active manager daemon x restarted 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: Activating manager daemon x 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: osdmap e96: 12 total, 12 up, 12 in 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: mgrmap e21: x(active, starting, since 0.0116843s), standbys: z, y 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:22:52.846 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:22:52.847 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T16:22:52.848 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-mon[37203]: Manager daemon x is now available 2024-02-19T16:22:53.079 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y[37883]: 2024-02-19T16:22:52.596+0000 7f772c1cc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: Active manager daemon x restarted 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: Activating manager daemon x 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: osdmap e96: 12 total, 12 up, 12 in 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: mgrmap e21: x(active, starting, since 0.0116843s), standbys: z, y 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-19T16:22:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-19T16:22:53.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T16:22:53.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T16:22:53.083 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:52 smithi156 ceph-mon[36502]: Manager daemon x is now available 2024-02-19T16:22:53.269 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f05418a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f05418a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f05418a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f05418a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f05418a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f053e89b700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f053e89b700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.271 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f053e89b700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.271 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f053e89b700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.271 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:22:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:22:52.796+0000 7f053e89b700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:22:53.277 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:52.844+0000 7fceb59e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-19T16:22:53.277 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z[39059]: 2024-02-19T16:22:53.068+0000 7fceb59e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-19T16:22:53.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.555+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fc92c0ba2e0 con 0x7fc9341030f0 2024-02-19T16:22:53.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.556+0000 7fc9317fa640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc920050d30 0x7fc920051110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:53.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.556+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 --> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc93402d000 con 0x7fc920050d30 2024-02-19T16:22:53.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.559+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc920050d30 0x7fc920051110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:53.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.562+0000 7fc933fff640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc920050d30 0x7fc920051110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc934102590 tx=0x7fc91c005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:53.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.630+0000 7fc9317fa640 1 -- 172.21.15.8:0/1391921235 <== mgr.15633 v2:172.21.15.8:6800/4279763273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc93402d000 con 0x7fc920050d30 2024-02-19T16:22:53.631 INFO:teuthology.orchestra.run.smithi008.stdout:Schedule set for path / 2024-02-19T16:22:53.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc920050d30 msgr2=0x7fc920051110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:53.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc920050d30 0x7fc920051110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc934102590 tx=0x7fc91c005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:53.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 msgr2=0x7fc93419eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 --2- 172.21.15.8:0/1391921235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9341030f0 0x7fc93419eab0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc92c037040 tx=0x7fc92c009360 comp rx=0 tx=0).stop 2024-02-19T16:22:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc938b6d640 1 -- 172.21.15.8:0/1391921235 reap_dead start 2024-02-19T16:22:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 shutdown_connections 2024-02-19T16:22:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 >> 172.21.15.8:0/1391921235 conn(0x7fc9340fcab0 msgr2=0x7fc9341059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 shutdown_connections 2024-02-19T16:22:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.632+0000 7fc93a5d0640 1 -- 172.21.15.8:0/1391921235 wait complete. 2024-02-19T16:22:53.645 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 -- 172.21.15.8:0/4073751543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b401071f0 msgr2=0x7f5b40109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 --2- 172.21.15.8:0/4073751543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b401071f0 0x7f5b40109640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009ca0 tx=0x7f5b3002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 -- 172.21.15.8:0/4073751543 shutdown_connections 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 --2- 172.21.15.8:0/4073751543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b401071f0 0x7f5b40109640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 --2- 172.21.15.8:0/4073751543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 0x7f5b40106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 --2- 172.21.15.8:0/4073751543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b40101f90 0x7f5b40104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 -- 172.21.15.8:0/4073751543 >> 172.21.15.8:0/4073751543 conn(0x7f5b400fb980 msgr2=0x7f5b400fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 -- 172.21.15.8:0/4073751543 shutdown_connections 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 -- 172.21.15.8:0/4073751543 wait complete. 2024-02-19T16:22:53.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.755+0000 7f5b45450640 1 Processor -- start 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 -- start start 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 0x7f5b401974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 0x7f5b40197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b401071f0 0x7f5b4019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019efc0 con 0x7f5b401048c0 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019f130 con 0x7f5b401071f0 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b45450640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019f2f0 con 0x7f5b40101f90 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3e7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 0x7f5b40197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 0x7f5b401974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 0x7f5b401974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:59972/0 (socket says 172.21.15.8:59972) 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 -- 172.21.15.8:0/4035916812 learned_addr learned my addr 172.21.15.8:0/4035916812 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3f7fe640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b401071f0 0x7f5b4019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 -- 172.21.15.8:0/4035916812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b401071f0 msgr2=0x7f5b4019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b401071f0 0x7f5b4019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:53.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 -- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 msgr2=0x7f5b40197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 0x7f5b40197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 -- 172.21.15.8:0/4035916812 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b30009900 con 0x7f5b40101f90 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3e7fc640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 0x7f5b40197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.756+0000 7f5b3effd640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 0x7f5b401974d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e9f0 tx=0x7f5b3800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.757+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38009720 con 0x7f5b40101f90 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.757+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4019f5d0 con 0x7f5b40101f90 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.757+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b3800cd80 con 0x7f5b40101f90 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.757+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38010690 con 0x7f5b40101f90 2024-02-19T16:22:53.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.757+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4019fa90 con 0x7f5b40101f90 2024-02-19T16:22:53.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.758+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b400fd470 con 0x7f5b40101f90 2024-02-19T16:22:53.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.759+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f5b380046c0 con 0x7f5b40101f90 2024-02-19T16:22:53.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.759+0000 7f5b37fff640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5b2c0a1f70 0x7f5b2c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:53.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.759+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f5b38014070 con 0x7f5b40101f90 2024-02-19T16:22:53.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.759+0000 7f5b3e7fc640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5b2c0a1f70 0x7f5b2c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:53.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.760+0000 7f5b3e7fc640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5b2c0a1f70 0x7f5b2c0a4430 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b401989e0 tx=0x7f5b28005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:53.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.763+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5b38096ed0 con 0x7f5b40101f90 2024-02-19T16:22:53.883 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: Standby manager daemon y restarted 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: Standby manager daemon y started 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:53.884 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-19T16:22:53.885 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-19T16:22:53.885 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: Standby manager daemon z restarted 2024-02-19T16:22:53.885 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: Standby manager daemon z started 2024-02-19T16:22:53.885 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-19T16:22:53.885 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:22:53.885 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-19T16:22:53.886 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:22:53.886 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: [19/Feb/2024:16:22:53] ENGINE Bus STARTING 2024-02-19T16:22:53.886 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: [19/Feb/2024:16:22:53] ENGINE Serving on http://172.21.15.8:8765 2024-02-19T16:22:53.886 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:53 smithi191 ceph-mon[37203]: mgrmap e22: x(active, since 1.01818s), standbys: y, z 2024-02-19T16:22:53.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:53.974+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 --> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b4002d070 con 0x7f5b2c0a1f70 2024-02-19T16:22:53.976 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: Standby manager daemon y restarted 2024-02-19T16:22:53.976 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: Standby manager daemon y started 2024-02-19T16:22:53.977 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-19T16:22:53.977 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:22:53.977 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-19T16:22:53.977 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:22:53.977 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:53.977 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:53.978 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-19T16:22:53.978 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-19T16:22:53.978 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: Standby manager daemon z restarted 2024-02-19T16:22:53.978 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: Standby manager daemon z started 2024-02-19T16:22:53.978 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-19T16:22:53.978 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:22:53.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-19T16:22:53.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:22:53.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: [19/Feb/2024:16:22:53] ENGINE Bus STARTING 2024-02-19T16:22:53.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: [19/Feb/2024:16:22:53] ENGINE Serving on http://172.21.15.8:8765 2024-02-19T16:22:53.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:53 smithi008 ceph-mon[32573]: mgrmap e22: x(active, since 1.01818s), standbys: y, z 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: Standby manager daemon y restarted 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: Standby manager daemon y started 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.156:0/3995843185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: Standby manager daemon z restarted 2024-02-19T16:22:53.982 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: Standby manager daemon z started 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: from='mgr.? 172.21.15.191:0/3184412977' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: [19/Feb/2024:16:22:53] ENGINE Bus STARTING 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: [19/Feb/2024:16:22:53] ENGINE Serving on http://172.21.15.8:8765 2024-02-19T16:22:53.983 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:53 smithi156 ceph-mon[36502]: mgrmap e22: x(active, since 1.01818s), standbys: y, z 2024-02-19T16:22:54.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.028+0000 7f5b37fff640 1 -- 172.21.15.8:0/4035916812 <== mgr.15633 v2:172.21.15.8:6800/4279763273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5b4002d070 con 0x7f5b2c0a1f70 2024-02-19T16:22:54.029 INFO:teuthology.orchestra.run.smithi008.stdout:Retention added to path / 2024-02-19T16:22:54.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5b2c0a1f70 msgr2=0x7f5b2c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5b2c0a1f70 0x7f5b2c0a4430 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b401989e0 tx=0x7f5b28005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 msgr2=0x7f5b401974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 0x7f5b401974d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e9f0 tx=0x7f5b3800edf0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 shutdown_connections 2024-02-19T16:22:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b401071f0 0x7f5b4019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5b401048c0 0x7f5b40197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5b2c0a1f70 0x7f5b2c0a4430 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 --2- 172.21.15.8:0/4035916812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b40101f90 0x7f5b401974d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 >> 172.21.15.8:0/4035916812 conn(0x7f5b400fb980 msgr2=0x7f5b40102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:54.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 shutdown_connections 2024-02-19T16:22:54.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.030+0000 7f5b45450640 1 -- 172.21.15.8:0/4035916812 wait complete. 2024-02-19T16:22:54.043 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-19T16:22:54.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/4245836949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc0ff5d0 msgr2=0x7f0fdc0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/4245836949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc0ff5d0 0x7f0fdc0ffa50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009900 tx=0x7f0fd002bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:54.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/4245836949 shutdown_connections 2024-02-19T16:22:54.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/4245836949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fdc0fff90 0x7f0fdc109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/4245836949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc0ff5d0 0x7f0fdc0ffa50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/4245836949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fdc102b60 0x7f0fdc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/4245836949 >> 172.21.15.8:0/4245836949 conn(0x7f0fdc0fb110 msgr2=0x7f0fdc0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:54.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/4245836949 shutdown_connections 2024-02-19T16:22:54.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.148+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/4245836949 wait complete. 2024-02-19T16:22:54.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 Processor -- start 2024-02-19T16:22:54.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 -- start start 2024-02-19T16:22:54.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fdc0ff5d0 0x7f0fdc102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fdc0fff90 0x7f0fdc1008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 0x7f0fdc100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc101540 con 0x7f0fdc102b60 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc1016b0 con 0x7f0fdc0ff5d0 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fe1f9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc1a15c0 con 0x7f0fdc0fff90 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 0x7f0fdc100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 0x7f0fdc100de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51754/0 (socket says 172.21.15.8:51754) 2024-02-19T16:22:54.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 -- 172.21.15.8:0/1748697167 learned_addr learned my addr 172.21.15.8:0/1748697167 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:54.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdaffd640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fdc0fff90 0x7f0fdc1008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdb7fe640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fdc0ff5d0 0x7f0fdc102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 -- 172.21.15.8:0/1748697167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fdc0fff90 msgr2=0x7f0fdc1008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fdc0fff90 0x7f0fdc1008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 -- 172.21.15.8:0/1748697167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fdc0ff5d0 msgr2=0x7f0fdc102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fdc0ff5d0 0x7f0fdc102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.149+0000 7f0fdbfff640 1 -- 172.21.15.8:0/1748697167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd0009560 con 0x7f0fdc102b60 2024-02-19T16:22:54.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.150+0000 7f0fdbfff640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 0x7f0fdc100de0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400eb40 tx=0x7f0fd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:54.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.150+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004b20 con 0x7f0fdc102b60 2024-02-19T16:22:54.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.150+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc1a1870 con 0x7f0fdc102b60 2024-02-19T16:22:54.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.150+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fd4004c80 con 0x7f0fdc102b60 2024-02-19T16:22:54.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.150+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4010670 con 0x7f0fdc102b60 2024-02-19T16:22:54.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.150+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc1a1d00 con 0x7f0fdc102b60 2024-02-19T16:22:54.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.151+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc108a20 con 0x7f0fdc102b60 2024-02-19T16:22:54.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.152+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f0fd4005020 con 0x7f0fdc102b60 2024-02-19T16:22:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.152+0000 7f0fd8ff9640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0fc40a1f70 0x7f0fc40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.152+0000 7f0fdb7fe640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0fc40a1f70 0x7f0fc40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.152+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f0fd4014070 con 0x7f0fdc102b60 2024-02-19T16:22:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.153+0000 7f0fdb7fe640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0fc40a1f70 0x7f0fc40a4430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc069a50 tx=0x7f0fcc005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.157+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f0fd4097130 con 0x7f0fdc102b60 2024-02-19T16:22:54.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.386+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 --> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fdc02d000 con 0x7f0fc40a1f70 2024-02-19T16:22:54.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.395+0000 7f0fd8ff9640 1 -- 172.21.15.8:0/1748697167 <== mgr.15633 v2:172.21.15.8:6800/4279763273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f0fdc02d000 con 0x7f0fc40a1f70 2024-02-19T16:22:54.396 INFO:teuthology.orchestra.run.smithi008.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-19T00:00:00", "created": "2024-02-19T16:22:53", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-19T16:22:54.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0fc40a1f70 msgr2=0x7f0fc40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0fc40a1f70 0x7f0fc40a4430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc069a50 tx=0x7f0fcc005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:54.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 msgr2=0x7f0fdc100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 0x7f0fdc100de0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400eb40 tx=0x7f0fd400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 shutdown_connections 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fdc102b60 0x7f0fdc100de0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0fc40a1f70 0x7f0fc40a4430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fdc0fff90 0x7f0fdc1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 --2- 172.21.15.8:0/1748697167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fdc0ff5d0 0x7f0fdc102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 >> 172.21.15.8:0/1748697167 conn(0x7f0fdc0fb110 msgr2=0x7f0fdc105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 shutdown_connections 2024-02-19T16:22:54.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.397+0000 7f0fe1f9a640 1 -- 172.21.15.8:0/1748697167 wait complete. 2024-02-19T16:22:54.410 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-19T16:22:54.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 -- 172.21.15.8:0/1515880413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 msgr2=0x7f2ebc101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/1515880413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc101ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009900 tx=0x7f2ea802bc30 comp rx=0 tx=0).stop 2024-02-19T16:22:54.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 -- 172.21.15.8:0/1515880413 shutdown_connections 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/1515880413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 0x7f2ebc108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/1515880413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc101ac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/1515880413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 -- 172.21.15.8:0/1515880413 >> 172.21.15.8:0/1515880413 conn(0x7f2ebc0fb9c0 msgr2=0x7f2ebc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 -- 172.21.15.8:0/1515880413 shutdown_connections 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.520+0000 7f2ec2df4640 1 -- 172.21.15.8:0/1515880413 wait complete. 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 Processor -- start 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 -- start start 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 0x7f2ebc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc19eff0 con 0x7f2ebc100450 2024-02-19T16:22:54.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc19f160 con 0x7f2ebc101640 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec2df4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc19f320 con 0x7f2ebc102000 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec0b90640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec0b90640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51770/0 (socket says 172.21.15.8:51770) 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec0b90640 1 -- 172.21.15.8:0/2830279977 learned_addr learned my addr 172.21.15.8:0/2830279977 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ec1391640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 0x7f2ebc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.521+0000 7f2ebbfff640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec1391640 1 -- 172.21.15.8:0/2830279977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 msgr2=0x7f2ebc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec1391640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec1391640 1 -- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 msgr2=0x7f2ebc197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec1391640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec1391640 1 -- 172.21.15.8:0/2830279977 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009560 con 0x7f2ebc102000 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec0b90640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ebbfff640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:22:54.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec1391640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 0x7f2ebc19eab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009fd0 tx=0x7f2eb400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:54.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400ce20 con 0x7f2ebc102000 2024-02-19T16:22:54.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc19f600 con 0x7f2ebc102000 2024-02-19T16:22:54.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4005020 con 0x7f2ebc102000 2024-02-19T16:22:54.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4010690 con 0x7f2ebc102000 2024-02-19T16:22:54.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.522+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc19fb30 con 0x7f2ebc102000 2024-02-19T16:22:54.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.523+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc1076a0 con 0x7f2ebc102000 2024-02-19T16:22:54.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.524+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f2eb4005500 con 0x7f2ebc102000 2024-02-19T16:22:54.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.525+0000 7f2eb9ffb640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2eac0a1f70 0x7f2eac0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:22:54.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.525+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f2eb4014070 con 0x7f2ebc102000 2024-02-19T16:22:54.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.525+0000 7f2ec0b90640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2eac0a1f70 0x7f2eac0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:22:54.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.525+0000 7f2ec0b90640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2eac0a1f70 0x7f2eac0a4430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00058c0 tx=0x7f2eb0005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:22:54.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.528+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2eb4097260 con 0x7f2ebc102000 2024-02-19T16:22:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:54 smithi008 ceph-mon[32573]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:54 smithi008 ceph-mon[32573]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:54 smithi008 ceph-mon[32573]: [19/Feb/2024:16:22:53] ENGINE Serving on https://172.21.15.8:7150 2024-02-19T16:22:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:54 smithi008 ceph-mon[32573]: [19/Feb/2024:16:22:53] ENGINE Bus STARTED 2024-02-19T16:22:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:54 smithi008 ceph-mon[32573]: from='client.25573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:54 smithi008 ceph-mon[32573]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:54.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.783+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 --> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2ebc02d070 con 0x7f2eac0a1f70 2024-02-19T16:22:54.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.790+0000 7f2eb9ffb640 1 -- 172.21.15.8:0/2830279977 <== mgr.15633 v2:172.21.15.8:6800/4279763273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2ebc02d070 con 0x7f2eac0a1f70 2024-02-19T16:22:54.792 INFO:teuthology.orchestra.run.smithi008.stdout:/ 1m 6m3h 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2eac0a1f70 msgr2=0x7f2eac0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2eac0a1f70 0x7f2eac0a4430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2eb00058c0 tx=0x7f2eb0005f90 comp rx=0 tx=0).stop 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 msgr2=0x7f2ebc19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 0x7f2ebc19eab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009fd0 tx=0x7f2eb400ef10 comp rx=0 tx=0).stop 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 shutdown_connections 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2eac0a1f70 0x7f2eac0a4430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ebc102000 0x7f2ebc19eab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ebc101640 0x7f2ebc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 --2- 172.21.15.8:0/2830279977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ebc100450 0x7f2ebc197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 >> 172.21.15.8:0/2830279977 conn(0x7f2ebc0fb9c0 msgr2=0x7f2ebc104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 shutdown_connections 2024-02-19T16:22:54.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:22:54.792+0000 7f2ec2df4640 1 -- 172.21.15.8:0/2830279977 wait complete. 2024-02-19T16:22:54.804 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-19T16:22:54.832 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-19T16:22:54.855 INFO:teuthology.run_tasks:Running task workunit... 2024-02-19T16:22:55.002 INFO:tasks.workunit:Pulling workunits from ref 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:22:55.002 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-19T16:22:55.002 INFO:tasks.workunit:timeout=3h 2024-02-19T16:22:55.003 INFO:tasks.workunit:cleanup=True 2024-02-19T16:22:55.003 DEBUG:teuthology.orchestra.run.smithi008:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-02-19 16:20:42.724827772 +0000 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-02-19 16:20:42.724827772 +0000 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-02-19 16:22:36.164795193 +0000 2024-02-19T16:22:55.018 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2024-02-19 16:20:42.724827772 +0000 2024-02-19T16:22:55.019 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-19T16:22:55.019 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-19T16:22:55.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:54 smithi191 ceph-mon[37203]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:55.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:54 smithi191 ceph-mon[37203]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:54 smithi191 ceph-mon[37203]: [19/Feb/2024:16:22:53] ENGINE Serving on https://172.21.15.8:7150 2024-02-19T16:22:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:54 smithi191 ceph-mon[37203]: [19/Feb/2024:16:22:53] ENGINE Bus STARTED 2024-02-19T16:22:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:54 smithi191 ceph-mon[37203]: from='client.25573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:54 smithi191 ceph-mon[37203]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:55.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:54 smithi156 ceph-mon[36502]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:55.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:54 smithi156 ceph-mon[36502]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:55.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:54 smithi156 ceph-mon[36502]: [19/Feb/2024:16:22:53] ENGINE Serving on https://172.21.15.8:7150 2024-02-19T16:22:55.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:54 smithi156 ceph-mon[36502]: [19/Feb/2024:16:22:53] ENGINE Bus STARTED 2024-02-19T16:22:55.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:54 smithi156 ceph-mon[36502]: from='client.25573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:55.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:54 smithi156 ceph-mon[36502]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:55.093 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9f8908eed2b2d7e8c316bd5be7ed6e49024931ee 2024-02-19T16:22:55.147 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-19T16:22:56.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:56.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='client.25585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:55 smithi008 ceph-mon[32573]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-02-19T16:22:56.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='client.25585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:55 smithi191 ceph-mon[37203]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-02-19T16:22:56.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:56.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='client.25585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:22:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:55 smithi156 ceph-mon[36502]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-02-19T16:22:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi008 to 177.2M 2024-02-19T16:22:57.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Unable to set osd_memory_target on smithi008 to 185894195: error parsing value: Value '185894195' is below minimum 939524096 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Unable to set osd_memory_target on smithi191 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:22:57.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:56 smithi008 ceph-mon[32573]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi008 to 177.2M 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Unable to set osd_memory_target on smithi008 to 185894195: error parsing value: Value '185894195' is below minimum 939524096 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Unable to set osd_memory_target on smithi191 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:22:57.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:22:57.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:22:57.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:22:57.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:56 smithi191 ceph-mon[37203]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:57.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi008 to 177.2M 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Unable to set osd_memory_target on smithi008 to 185894195: error parsing value: Value '185894195' is below minimum 939524096 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Unable to set osd_memory_target on smithi191 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi008:/etc/ceph/ceph.conf 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T16:22:57.332 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:56 smithi156 ceph-mon[36502]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:22:58.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:57 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:58.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:22:58.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:22:58.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:57 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-19T16:22:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: Updating smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: Updating smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: Updating smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:22:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:22:58.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:57 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:59 smithi008 ceph-mon[32573]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:23:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:22:59 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:23:00.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:23:00 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:23:00.038+0000 7f051f4dd700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:23:00.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:23:00 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:23:00.038+0000 7f051f4dd700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:23:00.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:23:00 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:23:00.038+0000 7f051f4dd700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:23:00.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:23:00 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:23:00.038+0000 7f051f4dd700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:23:00.270 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:23:00 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:23:00.038+0000 7f051f4dd700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-19T16:23:00.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:59 smithi191 ceph-mon[37203]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:23:00.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:22:59 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:23:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:59 smithi156 ceph-mon[36502]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:23:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:22:59 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:23:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:01 smithi008 ceph-mon[32573]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-02-19T16:23:02.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:01 smithi008 ceph-mon[32573]: mgrmap e25: x(active, since 8s), standbys: y, z 2024-02-19T16:23:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:01 smithi191 ceph-mon[37203]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-02-19T16:23:02.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:01 smithi191 ceph-mon[37203]: mgrmap e25: x(active, since 8s), standbys: y, z 2024-02-19T16:23:02.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:01 smithi156 ceph-mon[36502]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-02-19T16:23:02.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:01 smithi156 ceph-mon[36502]: mgrmap e25: x(active, since 8s), standbys: y, z 2024-02-19T16:23:04.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:03 smithi008 ceph-mon[32573]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-02-19T16:23:04.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:03 smithi191 ceph-mon[37203]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-02-19T16:23:04.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:03 smithi156 ceph-mon[36502]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-02-19T16:23:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:05 smithi008 ceph-mon[32573]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-02-19T16:23:06.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:05 smithi191 ceph-mon[37203]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-02-19T16:23:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:05 smithi156 ceph-mon[36502]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-02-19T16:23:08.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:07 smithi008 ceph-mon[32573]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-02-19T16:23:08.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:07 smithi191 ceph-mon[37203]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-02-19T16:23:08.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:07 smithi156 ceph-mon[36502]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-02-19T16:23:10.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:09 smithi008 ceph-mon[32573]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-02-19T16:23:10.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:09 smithi191 ceph-mon[37203]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-02-19T16:23:10.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:09 smithi156 ceph-mon[36502]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-02-19T16:23:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:11 smithi008 ceph-mon[32573]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-02-19T16:23:12.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:11 smithi191 ceph-mon[37203]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-02-19T16:23:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:11 smithi156 ceph-mon[36502]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-02-19T16:23:14.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:13 smithi008 ceph-mon[32573]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-02-19T16:23:14.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:13 smithi191 ceph-mon[37203]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-02-19T16:23:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:13 smithi156 ceph-mon[36502]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-02-19T16:23:15.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:23:15.012 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:23:15.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 -- 172.21.15.8:0/838658133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec100fa0 msgr2=0x7f84ec104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 --2- 172.21.15.8:0/838658133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec100fa0 0x7f84ec104fe0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009930 tx=0x7f84e002bc30 comp rx=0 tx=0).stop 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 -- 172.21.15.8:0/838658133 shutdown_connections 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 --2- 172.21.15.8:0/838658133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84ec1056c0 0x7f84ec10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 --2- 172.21.15.8:0/838658133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec100fa0 0x7f84ec104fe0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 --2- 172.21.15.8:0/838658133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 0x7f84ec1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 -- 172.21.15.8:0/838658133 >> 172.21.15.8:0/838658133 conn(0x7f84ec0774e0 msgr2=0x7f84ec0778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 -- 172.21.15.8:0/838658133 shutdown_connections 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.118+0000 7f84f232c640 1 -- 172.21.15.8:0/838658133 wait complete. 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 Processor -- start 2024-02-19T16:23:15.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 -- start start 2024-02-19T16:23:15.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 0x7f84ec19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84ec1056c0 0x7f84ec19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 0x7f84ec197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec1982e0 con 0x7f84ec1978f0 2024-02-19T16:23:15.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec198450 con 0x7f84ec1056c0 2024-02-19T16:23:15.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f232c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec198610 con 0x7f84ec1005f0 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.119+0000 7f84f08c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 0x7f84ec197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 0x7f84ec19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84f08c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 0x7f84ec197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43584/0 (socket says 172.21.15.8:43584) 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84f08c9640 1 -- 172.21.15.8:0/565471316 learned_addr learned my addr 172.21.15.8:0/565471316 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84ec1056c0 0x7f84ec19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 -- 172.21.15.8:0/565471316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84ec1056c0 msgr2=0x7f84ec19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84ec1056c0 0x7f84ec19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 -- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 msgr2=0x7f84ec197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 0x7f84ec197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 -- 172.21.15.8:0/565471316 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc009490 con 0x7f84ec1005f0 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84f08c9640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 0x7f84ec197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84ebfff640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 0x7f84ec19d740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009ce0 tx=0x7f84dc00b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84e97fa640 1 -- 172.21.15.8:0/565471316 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc015070 con 0x7f84ec1005f0 2024-02-19T16:23:15.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e0009560 con 0x7f84ec1005f0 2024-02-19T16:23:15.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84e97fa640 1 -- 172.21.15.8:0/565471316 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84dc009e90 con 0x7f84ec1005f0 2024-02-19T16:23:15.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84e97fa640 1 -- 172.21.15.8:0/565471316 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc019410 con 0x7f84ec1005f0 2024-02-19T16:23:15.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.120+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec102350 con 0x7f84ec1005f0 2024-02-19T16:23:15.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.122+0000 7f84e97fa640 1 -- 172.21.15.8:0/565471316 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f84dc019570 con 0x7f84ec1005f0 2024-02-19T16:23:15.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 Processor -- start 2024-02-19T16:23:15.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 -- start start 2024-02-19T16:23:15.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 0x55b131d9f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84e97fa640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f84d40a2350 0x7f84d40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9fbd0 0x55b131da8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131da87b0 0x55b131daaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131d0e280 con 0x55b131d9f2b0 2024-02-19T16:23:15.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131d0b7b0 con 0x55b131da87b0 2024-02-19T16:23:15.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131d06f90 con 0x55b131d9fbd0 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84ebfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 0x55b131d9f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84ebfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 0x55b131d9f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43594/0 (socket says 172.21.15.8:43594) 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84ebfff640 1 -- 172.21.15.8:0/1915569784 learned_addr learned my addr 172.21.15.8:0/1915569784 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f08c9640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f84d40a2350 0x7f84d40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9fbd0 0x55b131da8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84f08c9640 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131da87b0 0x55b131daaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84e97fa640 1 -- 172.21.15.8:0/565471316 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f84dc0d1fa0 con 0x7f84ec1005f0 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1915569784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131da87b0 msgr2=0x55b131daaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131da87b0 0x55b131daaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1915569784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 msgr2=0x55b131d9f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 0x55b131d9f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1915569784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b131cc5570 con 0x55b131d9fbd0 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.123+0000 7f84ebfff640 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 0x55b131d9f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f08c9640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f84d40a2350 0x7f84d40a4810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009fd0 tx=0x7f84e4013460 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9fbd0 0x55b131da8270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f84e002c170 tx=0x7f84e002c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244ecb9dd0f91c91 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84d27fc640 1 -- 172.21.15.8:0/1915569784 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004740 con 0x55b131d9fbd0 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84d27fc640 1 -- 172.21.15.8:0/1915569784 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e00048a0 con 0x55b131d9fbd0 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84d27fc640 1 -- 172.21.15.8:0/1915569784 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0005050 con 0x55b131d9fbd0 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1915569784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9fbd0 msgr2=0x55b131da8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9fbd0 0x55b131da8270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f84e002c170 tx=0x7f84e002c5d0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1915569784 shutdown_connections 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131da87b0 0x55b131daaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9fbd0 0x55b131da8270 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1915569784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9f2b0 0x55b131d9f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.124+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1915569784 >> 172.21.15.8:0/1915569784 conn(0x55b131d99990 msgr2=0x55b131d99d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1915569784 shutdown_connections 2024-02-19T16:23:15.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1915569784 wait complete. 2024-02-19T16:23:15.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 -- start start 2024-02-19T16:23:15.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 Processor -- start 2024-02-19T16:23:15.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9f2b0 0x55b131da81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 0x55b131da5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131da87b0 0x55b131da5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1319da430 con 0x55b131d9fbd0 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131996e00 con 0x55b131d9f2b0 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131c8bfd0 con 0x55b131da87b0 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84f4fe7740 10 client.0 _ll_register_callbacks cb 0x55b131ce1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84eb7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 0x55b131da5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84eb7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 0x55b131da5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43596/0 (socket says 172.21.15.8:43596) 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1105377826 learned_addr learned my addr 172.21.15.8:0/1105377826 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.125+0000 7f84ebfff640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9f2b0 0x55b131da81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84f08c9640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131da87b0 0x55b131da5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131da87b0 msgr2=0x55b131da5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131da87b0 0x55b131da5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1105377826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9f2b0 msgr2=0x55b131da81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9f2b0 0x55b131da81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84eb7fe640 1 -- 172.21.15.8:0/1105377826 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec10daa0 con 0x55b131d9fbd0 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 0x55b131da5550 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f84e0036040 tx=0x7f84e0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9e3b1829fe0f52 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84cb7fe640 1 -- 172.21.15.8:0/1105377826 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004ff0 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84cb7fe640 1 -- 172.21.15.8:0/1105377826 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f84e00044a0 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b131a067e0 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84cb7fe640 1 -- 172.21.15.8:0/1105377826 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e0005150 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84f4fe7740 10 client.15735 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b131d05610 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.126+0000 7f84cb7fe640 1 -- 172.21.15.8:0/1105377826 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b131a067e0 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84cb7fe640 1 -- 172.21.15.8:0/1105377826 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f84e003c050 con 0x55b131d9fbd0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84f4fe7740 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84f4fe7740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84f4fe7740 20 client.15735 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131db84f0 0x55b131dba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84f4fe7740 4 client.15735 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b131a43720 con 0x55b131db84f0 2024-02-19T16:23:15.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84ebfff640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131db84f0 0x55b131dba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84ebfff640 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131db84f0 0x55b131dba8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0709bc74060d74 server_cookie=404c1c67088a3353 in_seq=0 out_seq=0 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.127+0000 7f84cb7fe640 10 client.15735 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84cb7fe640 1 -- 172.21.15.8:0/1105377826 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b131a43720 con 0x55b131db84f0 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84cb7fe640 10 client.15735 handle_command_reply: tid=0 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 client.15735 shutdown 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 msgr2=0x55b131da5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 0x55b131da5550 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f84e0036040 tx=0x7f84e0005c00 comp rx=0 tx=0).stop 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 shutdown_connections 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131db84f0 0x55b131dba8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131da87b0 0x55b131da5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131d9fbd0 0x55b131da5550 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84cb7fe640 0 client.15735 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1105377826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9f2b0 0x55b131da81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 >> 172.21.15.8:0/1105377826 conn(0x55b131d99990 msgr2=0x55b131d9b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 shutdown_connections 2024-02-19T16:23:15.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.128+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1105377826 wait complete. 2024-02-19T16:23:15.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.129+0000 7f84f4fe7740 20 client.15735 trim_cache size 0 max 16384 2024-02-19T16:23:15.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 Processor -- start 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 -- start start 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131d988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131da7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131da81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131db4db0 con 0x55b131da87b0 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131d997f0 con 0x55b131d9fbd0 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131d9bcf0 con 0x55b131d9f2b0 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131da81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131da81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43606/0 (socket says 172.21.15.8:43606) 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 -- 172.21.15.8:0/1827003359 learned_addr learned my addr 172.21.15.8:0/1827003359 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84ebfff640 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131d988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131da7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 -- 172.21.15.8:0/1827003359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 msgr2=0x55b131d988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131d988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 -- 172.21.15.8:0/1827003359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 msgr2=0x55b131da7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131da7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 -- 172.21.15.8:0/1827003359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b131ca1ac0 con 0x55b131da87b0 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84f08c9640 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131da81c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f84e4013800 tx=0x7f84e4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea7a915a800973d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.145+0000 7f84d1ffb640 1 -- 172.21.15.8:0/1827003359 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4018050 con 0x55b131da87b0 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84d1ffb640 1 -- 172.21.15.8:0/1827003359 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e400ed80 con 0x55b131da87b0 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84d1ffb640 1 -- 172.21.15.8:0/1827003359 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4026070 con 0x55b131da87b0 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1827003359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 msgr2=0x55b131da81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131da81c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f84e4013800 tx=0x7f84e4021000 comp rx=0 tx=0).stop 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1827003359 shutdown_connections 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131da81c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131da7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/1827003359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131d988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1827003359 >> 172.21.15.8:0/1827003359 conn(0x55b131d99990 msgr2=0x55b131d981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1827003359 shutdown_connections 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 -- 172.21.15.8:0/1827003359 wait complete. 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.146+0000 7f84f4fe7740 1 -- start start 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 Processor -- start 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131dab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131dabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131dac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131e13d90 con 0x55b131da87b0 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131e13ed0 con 0x55b131d9fbd0 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131e14010 con 0x55b131d9f2b0 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f4fe7740 10 client.0 _ll_register_callbacks cb 0x7f84dc0038f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f08c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131dac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84eb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131dabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f08c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131dac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43614/0 (socket says 172.21.15.8:43614) 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f08c9640 1 -- 172.21.15.8:0/4140681474 learned_addr learned my addr 172.21.15.8:0/4140681474 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84ebfff640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131dab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84eb7fe640 1 -- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 msgr2=0x55b131dab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131dab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84eb7fe640 1 -- 172.21.15.8:0/4140681474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 msgr2=0x55b131dac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131dac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84eb7fe640 1 -- 172.21.15.8:0/4140681474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e401b040 con 0x55b131d9fbd0 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84f08c9640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131dac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.147+0000 7f84ebfff640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131dab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84eb7fe640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131dabb10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84e003ffd0 tx=0x7f84e00097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eecc7f2fac58364 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84cb7fe640 1 -- 172.21.15.8:0/4140681474 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004450 con 0x55b131d9fbd0 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84e00048a0 con 0x55b131d9fbd0 2024-02-19T16:23:15.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84cb7fe640 1 -- 172.21.15.8:0/4140681474 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f84e0000e40 con 0x55b131d9fbd0 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84cb7fe640 1 -- 172.21.15.8:0/4140681474 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e00046c0 con 0x55b131d9fbd0 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 10 client.25514 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b131cc76a0 con 0x55b131d9fbd0 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84cb7fe640 1 -- 172.21.15.8:0/4140681474 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e00048a0 con 0x55b131d9fbd0 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84cb7fe640 1 -- 172.21.15.8:0/4140681474 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f84e003c040 con 0x55b131d9fbd0 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 10 client.25514 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 10 client.25514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 20 client.25514 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131e15350 0x55b131e17770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 4 client.25514 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.148+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b131a92890 con 0x55b131e15350 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84ebfff640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131e15350 0x55b131e17770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84cb7fe640 10 client.25514 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84ebfff640 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131e15350 0x55b131e17770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2efbb04e60d6a1 server_cookie=833ef9efaa1c1fe6 in_seq=0 out_seq=0 2024-02-19T16:23:15.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84cb7fe640 1 -- 172.21.15.8:0/4140681474 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b131a92890 con 0x55b131e15350 2024-02-19T16:23:15.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84cb7fe640 10 client.25514 handle_command_reply: tid=0 2024-02-19T16:23:15.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 client.25514 shutdown 2024-02-19T16:23:15.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 msgr2=0x55b131dabb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131dabb10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84e003ffd0 tx=0x7f84e00097a0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 shutdown_connections 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b131da87b0 0x55b131dac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131d9fbd0 0x55b131dabb10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b131e15350 0x55b131e17770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 --2- 172.21.15.8:0/4140681474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b131d9f2b0 0x55b131dab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84cb7fe640 0 client.25514 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.149+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 >> 172.21.15.8:0/4140681474 conn(0x55b131d99990 msgr2=0x55b131d9b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 shutdown_connections 2024-02-19T16:23:15.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f4fe7740 1 -- 172.21.15.8:0/4140681474 wait complete. 2024-02-19T16:23:15.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f4fe7740 20 client.25514 trim_cache size 0 max 16384 2024-02-19T16:23:15.161 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:23:15.161 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:23:15.161 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:23:15.161 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f84d40a2350 msgr2=0x7f84d40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f84d40a2350 0x7f84d40a4810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009fd0 tx=0x7f84e4013460 comp rx=0 tx=0).stop 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 msgr2=0x7f84ec19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 0x7f84ec19d740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009ce0 tx=0x7f84dc00b690 comp rx=0 tx=0).stop 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 shutdown_connections 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84ec1978f0 0x7f84ec197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84ec1056c0 0x7f84ec19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f84d40a2350 0x7f84d40a4810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 --2- 172.21.15.8:0/565471316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84ec1005f0 0x7f84ec19d740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 >> 172.21.15.8:0/565471316 conn(0x7f84ec0774e0 msgr2=0x7f84ec069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 shutdown_connections 2024-02-19T16:23:15.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.150+0000 7f84f232c640 1 -- 172.21.15.8:0/565471316 wait complete. 2024-02-19T16:23:15.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:23:15.164 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.266+0000 7efcb9d6c640 1 -- 172.21.15.8:0/3683653350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb41003f0 msgr2=0x7efcb4100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.266+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/3683653350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb41003f0 0x7efcb4100870 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efca8009930 tx=0x7efca802bc30 comp rx=0 tx=0).stop 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- 172.21.15.8:0/3683653350 shutdown_connections 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/3683653350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 0x7efcb410d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/3683653350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb41003f0 0x7efcb4100870 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/3683653350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb4106490 0x7efcb4106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- 172.21.15.8:0/3683653350 >> 172.21.15.8:0/3683653350 conn(0x7efcb40774f0 msgr2=0x7efcb4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- 172.21.15.8:0/3683653350 shutdown_connections 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- 172.21.15.8:0/3683653350 wait complete. 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 Processor -- start 2024-02-19T16:23:15.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- start start 2024-02-19T16:23:15.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb41003f0 0x7efcb419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 0x7efcb419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 0x7efcb4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb4198060 con 0x7efcb4106490 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb41981d0 con 0x7efcb41003f0 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.267+0000 7efcb9d6c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb4198390 con 0x7efcb4100db0 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 0x7efcb4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 0x7efcb4197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43626/0 (socket says 172.21.15.8:43626) 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb41003f0 0x7efcb419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb3fff640 1 -- 172.21.15.8:0/2297603401 learned_addr learned my addr 172.21.15.8:0/2297603401 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 0x7efcb419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 -- 172.21.15.8:0/2297603401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb41003f0 msgr2=0x7efcb419d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb41003f0 0x7efcb419d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 -- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 msgr2=0x7efcb4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 0x7efcb4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 -- 172.21.15.8:0/2297603401 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009490 con 0x7efcb4100db0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb3fff640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 0x7efcb4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb37fe640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb41003f0 0x7efcb419d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb2ffd640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 0x7efcb419dda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efca8009230 tx=0x7efca802c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80042d0 con 0x7efcb4100db0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efca8004430 con 0x7efcb4100db0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca4009840 con 0x7efcb4100db0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8003bd0 con 0x7efcb4100db0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.268+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca8009560 con 0x7efcb4100db0 2024-02-19T16:23:15.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.269+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb40682e0 con 0x7efcb4100db0 2024-02-19T16:23:15.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.271+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7efca8003d30 con 0x7efcb4100db0 2024-02-19T16:23:15.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.271+0000 7efcb0ff9640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc9c0a23d0 0x7efc9c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.271+0000 7efcb37fe640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc9c0a23d0 0x7efc9c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.271+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7efca80f0890 con 0x7efcb4100db0 2024-02-19T16:23:15.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.272+0000 7efcb37fe640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc9c0a23d0 0x7efc9c0a4890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efca4009fd0 tx=0x7efca4009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.275+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7efca80b9030 con 0x7efcb4100db0 2024-02-19T16:23:15.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.495+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcb4198f40 con 0x7efcb4100db0 2024-02-19T16:23:15.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.496+0000 7efcb0ff9640 1 -- 172.21.15.8:0/2297603401 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7efca80bc6f0 con 0x7efcb4100db0 2024-02-19T16:23:15.497 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:23:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:22:36.842920+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:23:15.498 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.498+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc9c0a23d0 msgr2=0x7efc9c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.498+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc9c0a23d0 0x7efc9c0a4890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efca4009fd0 tx=0x7efca4009500 comp rx=0 tx=0).stop 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.498+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 msgr2=0x7efcb419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.498+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 0x7efcb419dda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efca8009230 tx=0x7efca802c5d0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 shutdown_connections 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcb4106490 0x7efcb4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc9c0a23d0 0x7efc9c0a4890 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcb4100db0 0x7efcb419dda0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 --2- 172.21.15.8:0/2297603401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb41003f0 0x7efcb419d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 >> 172.21.15.8:0/2297603401 conn(0x7efcb40774f0 msgr2=0x7efcb40689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 shutdown_connections 2024-02-19T16:23:15.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.499+0000 7efcb9d6c640 1 -- 172.21.15.8:0/2297603401 wait complete. 2024-02-19T16:23:15.510 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:23:15.511 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 -- 172.21.15.8:0/587673150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00686f0 msgr2=0x7f41b0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 --2- 172.21.15.8:0/587673150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00686f0 0x7f41b0068b70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009890 tx=0x7f41a402bb70 comp rx=0 tx=0).stop 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 -- 172.21.15.8:0/587673150 shutdown_connections 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 --2- 172.21.15.8:0/587673150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00690b0 0x7f41b010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 --2- 172.21.15.8:0/587673150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00686f0 0x7f41b0068b70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 --2- 172.21.15.8:0/587673150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b01074f0 0x7f41b01078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 -- 172.21.15.8:0/587673150 >> 172.21.15.8:0/587673150 conn(0x7f41b00774f0 msgr2=0x7f41b0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.614+0000 7f41b6091640 1 -- 172.21.15.8:0/587673150 shutdown_connections 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 -- 172.21.15.8:0/587673150 wait complete. 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 Processor -- start 2024-02-19T16:23:15.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 -- start start 2024-02-19T16:23:15.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00686f0 0x7f41b019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 0x7f41b019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b01074f0 0x7f41b01978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b0197f10 con 0x7f41b00690b0 2024-02-19T16:23:15.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b0198080 con 0x7f41b01074f0 2024-02-19T16:23:15.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.615+0000 7f41b6091640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b0198240 con 0x7f41b00686f0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 0x7f41b019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41af7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00686f0 0x7f41b019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 0x7f41b019dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40972/0 (socket says 172.21.15.8:40972) 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 -- 172.21.15.8:0/108817711 learned_addr learned my addr 172.21.15.8:0/108817711 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41affff640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b01074f0 0x7f41b01978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 -- 172.21.15.8:0/108817711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00686f0 msgr2=0x7f41b019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00686f0 0x7f41b019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 -- 172.21.15.8:0/108817711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b01074f0 msgr2=0x7f41b01978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b01074f0 0x7f41b01978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 -- 172.21.15.8:0/108817711 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4009490 con 0x7f41b00690b0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41af7fe640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00686f0 0x7f41b019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41aeffd640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 0x7f41b019dc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009860 tx=0x7f41a4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41acff9640 1 -- 172.21.15.8:0/108817711 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4034950 con 0x7f41b00690b0 2024-02-19T16:23:15.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41acff9640 1 -- 172.21.15.8:0/108817711 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41a402c9f0 con 0x7f41b00690b0 2024-02-19T16:23:15.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b01984c0 con 0x7f41b00690b0 2024-02-19T16:23:15.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41acff9640 1 -- 172.21.15.8:0/108817711 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4032600 con 0x7f41b00690b0 2024-02-19T16:23:15.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.616+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b01a47b0 con 0x7f41b00690b0 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.618+0000 7f41acff9640 1 -- 172.21.15.8:0/108817711 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f41a403b040 con 0x7f41b00690b0 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 Processor -- start 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41acff9640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41980a2350 0x7f41980a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 -- start start 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41affff640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41980a2350 0x7f41980a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41acff9640 1 -- 172.21.15.8:0/108817711 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f41a40f00b0 con 0x7f41b00690b0 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e30c30 0x5627a4e31010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e3e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e3eb60 0x5627a4e40f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4dec960 con 0x5627a4e315e0 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4da0310 con 0x5627a4e3eb60 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4d9d840 con 0x5627a4e30c30 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e3e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e3e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40974/0 (socket says 172.21.15.8:40974) 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 -- 172.21.15.8:0/1536071940 learned_addr learned my addr 172.21.15.8:0/1536071940 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41af7fe640 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e30c30 0x5627a4e31010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41affff640 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e3eb60 0x5627a4e40f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 -- 172.21.15.8:0/1536071940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e30c30 msgr2=0x5627a4e31010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e30c30 0x5627a4e31010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 -- 172.21.15.8:0/1536071940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e3eb60 msgr2=0x5627a4e40f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e3eb60 0x5627a4e40f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41aeffd640 1 -- 172.21.15.8:0/1536071940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a4d5b060 con 0x5627a4e315e0 2024-02-19T16:23:15.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41affff640 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e3eb60 0x5627a4e40f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.619+0000 7f41affff640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41980a2350 0x7f41980a4810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41a8005fd0 tx=0x7f41a80095c0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41aeffd640 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e3e620 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f41a40c47c0 tx=0x7f41a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96819c43d4a8035 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f4195ffb640 1 -- 172.21.15.8:0/1536071940 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40c06a0 con 0x5627a4e315e0 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f4195ffb640 1 -- 172.21.15.8:0/1536071940 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41a40c4cf0 con 0x5627a4e315e0 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f4195ffb640 1 -- 172.21.15.8:0/1536071940 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a402c680 con 0x5627a4e315e0 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 -- 172.21.15.8:0/1536071940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 msgr2=0x5627a4e3e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e3e620 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f41a40c47c0 tx=0x7f41a4005f90 comp rx=0 tx=0).stop 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 -- 172.21.15.8:0/1536071940 shutdown_connections 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e3eb60 0x5627a4e40f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e3e620 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/1536071940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e30c30 0x5627a4e31010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 -- 172.21.15.8:0/1536071940 >> 172.21.15.8:0/1536071940 conn(0x5627a4e2b980 msgr2=0x5627a4e2bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 -- 172.21.15.8:0/1536071940 shutdown_connections 2024-02-19T16:23:15.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.620+0000 7f41b8c5f740 1 -- 172.21.15.8:0/1536071940 wait complete. 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 -- start start 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 Processor -- start 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 0x5627a4e3bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e315e0 0x5627a4e3c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e3c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4d99020 con 0x5627a4e30c30 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4a6c430 con 0x5627a4e315e0 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4a28e00 con 0x5627a4e3eb60 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41b8c5f740 10 client.0 _ll_register_callbacks cb 0x5627a4d40cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41af7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 0x5627a4e3bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41af7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 0x5627a4e3bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40988/0 (socket says 172.21.15.8:40988) 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41af7fe640 1 -- 172.21.15.8:0/3446320024 learned_addr learned my addr 172.21.15.8:0/3446320024 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41aeffd640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e315e0 0x5627a4e3c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.621+0000 7f41affff640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e3c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41af7fe640 1 -- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 msgr2=0x5627a4e3c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41af7fe640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e3c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41af7fe640 1 -- 172.21.15.8:0/3446320024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e315e0 msgr2=0x5627a4e3c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41af7fe640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e315e0 0x5627a4e3c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41af7fe640 1 -- 172.21.15.8:0/3446320024 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b01a47b0 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41af7fe640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 0x5627a4e3bb40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f41a0012d50 tx=0x7f41a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27226de16e2ed320 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f418effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f418effd640 1 -- 172.21.15.8:0/3446320024 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f418effd640 1 -- 172.21.15.8:0/3446320024 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f41a0012110 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627a4a987e0 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f418effd640 1 -- 172.21.15.8:0/3446320024 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a00122d0 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 10 client.15759 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5627a4d976a0 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f418effd640 1 -- 172.21.15.8:0/3446320024 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5627a4a987e0 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f418effd640 1 -- 172.21.15.8:0/3446320024 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f41a0010430 con 0x5627a4e30c30 2024-02-19T16:23:15.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 10 client.15759 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 20 client.15759 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4e4ec90 0x5627a4e510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 4 client.15759 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.622+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627a4ad5720 con 0x5627a4e4ec90 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.623+0000 7f41aeffd640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4e4ec90 0x5627a4e510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.623+0000 7f41aeffd640 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4e4ec90 0x5627a4e510b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=971e01df09a40e03 server_cookie=75615328bbb1c580 in_seq=0 out_seq=0 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.623+0000 7f418effd640 10 client.15759 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f418effd640 1 -- 172.21.15.8:0/3446320024 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5627a4ad5720 con 0x5627a4e4ec90 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f418effd640 10 client.15759 handle_command_reply: tid=0 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 client.15759 shutdown 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 msgr2=0x5627a4e3bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 0x5627a4e3bb40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f41a0012d50 tx=0x7f41a000f790 comp rx=0 tx=0).stop 2024-02-19T16:23:15.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 shutdown_connections 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4e4ec90 0x5627a4e510b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e3c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e315e0 0x5627a4e3c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f418effd640 0 client.15759 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3446320024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e30c30 0x5627a4e3bb40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 >> 172.21.15.8:0/3446320024 conn(0x5627a4e2b980 msgr2=0x5627a4e2d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 shutdown_connections 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3446320024 wait complete. 2024-02-19T16:23:15.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.624+0000 7f41b8c5f740 20 client.15759 trim_cache size 0 max 16384 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.636+0000 7f41b8c5f740 1 Processor -- start 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 -- start start 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e2d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e315e0 0x5627a4e3c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 0x5627a4e2aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4d1e130 con 0x5627a4e3eb60 2024-02-19T16:23:15.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4e27320 con 0x5627a4e30c30 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4e40f80 con 0x5627a4e315e0 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 0x5627a4e2aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 0x5627a4e2aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41002/0 (socket says 172.21.15.8:41002) 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 -- 172.21.15.8:0/3887065879 learned_addr learned my addr 172.21.15.8:0/3887065879 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41aeffd640 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e315e0 0x5627a4e3c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41af7fe640 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e2d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 -- 172.21.15.8:0/3887065879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e315e0 msgr2=0x5627a4e3c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e315e0 0x5627a4e3c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 -- 172.21.15.8:0/3887065879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 msgr2=0x5627a4e2d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e2d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 -- 172.21.15.8:0/3887065879 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a4d33c20 con 0x5627a4e3eb60 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41affff640 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 0x5627a4e2aa00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f41a8003eb0 tx=0x7f41a801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e006f2b6521c66 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41957fa640 1 -- 172.21.15.8:0/3887065879 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8013040 con 0x5627a4e3eb60 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41957fa640 1 -- 172.21.15.8:0/3887065879 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41a800de50 con 0x5627a4e3eb60 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.637+0000 7f41957fa640 1 -- 172.21.15.8:0/3887065879 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a801b3f0 con 0x5627a4e3eb60 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3887065879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 msgr2=0x5627a4e2aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 0x5627a4e2aa00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f41a8003eb0 tx=0x7f41a801c000 comp rx=0 tx=0).stop 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3887065879 shutdown_connections 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e3eb60 0x5627a4e2aa00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e315e0 0x5627a4e3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/3887065879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e2d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3887065879 >> 172.21.15.8:0/3887065879 conn(0x5627a4e2b980 msgr2=0x5627a4e2a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3887065879 shutdown_connections 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 -- 172.21.15.8:0/3887065879 wait complete. 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.638+0000 7f41b8c5f740 1 -- start start 2024-02-19T16:23:15.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 Processor -- start 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e42ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e43000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e43540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4eaa590 con 0x5627a4e315e0 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4eaa6d0 con 0x5627a4e30c30 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a4eaa810 con 0x5627a4e3eb60 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 10 client.0 _ll_register_callbacks cb 0x7f41a0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e43000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e43000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41012/0 (socket says 172.21.15.8:41012) 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 -- 172.21.15.8:0/4021579222 learned_addr learned my addr 172.21.15.8:0/4021579222 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41affff640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e43540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41af7fe640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e42ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 -- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 msgr2=0x5627a4e43540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e43540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 -- 172.21.15.8:0/4021579222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 msgr2=0x5627a4e42ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e42ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 -- 172.21.15.8:0/4021579222 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a8003b60 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41aeffd640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e43000 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f41a4000c00 tx=0x7f41a40b8200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d9d7c99467e99e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f418effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f418effd640 1 -- 172.21.15.8:0/4021579222 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40ed2c0 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f418effd640 1 -- 172.21.15.8:0/4021579222 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f41a40c06a0 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.639+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41a40c4cf0 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f418effd640 1 -- 172.21.15.8:0/4021579222 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a402cb60 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 10 client.15765 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5627a4d598c0 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f418effd640 1 -- 172.21.15.8:0/4021579222 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41a40c4cf0 con 0x5627a4e315e0 2024-02-19T16:23:15.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f418effd640 1 -- 172.21.15.8:0/4021579222 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f41a40bc860 con 0x5627a4e315e0 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 10 client.15765 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 20 client.15765 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4eabce0 0x5627a4eae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 4 client.15765 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627a4b24890 con 0x5627a4eabce0 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41af7fe640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4eabce0 0x5627a4eae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f418effd640 10 client.15765 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.640+0000 7f41af7fe640 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4eabce0 0x5627a4eae100 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc4536b05702fc1 server_cookie=802aa7e3603fffa0 in_seq=0 out_seq=0 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f418effd640 1 -- 172.21.15.8:0/4021579222 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627a4b24890 con 0x5627a4eabce0 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f418effd640 10 client.15765 handle_command_reply: tid=0 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 client.15765 shutdown 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 msgr2=0x5627a4e43000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e43000 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f41a4000c00 tx=0x7f41a40b8200 comp rx=0 tx=0).stop 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 shutdown_connections 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627a4eabce0 0x5627a4eae100 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627a4e3eb60 0x5627a4e43540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627a4e315e0 0x5627a4e43000 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 --2- 172.21.15.8:0/4021579222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a4e30c30 0x5627a4e42ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f418effd640 0 client.15765 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 >> 172.21.15.8:0/4021579222 conn(0x5627a4e2b980 msgr2=0x5627a4e2ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 shutdown_connections 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 1 -- 172.21.15.8:0/4021579222 wait complete. 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b8c5f740 20 client.15765 trim_cache size 0 max 16384 2024-02-19T16:23:15.648 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:23:15.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41980a2350 msgr2=0x7f41980a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41980a2350 0x7f41980a4810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41a8005fd0 tx=0x7f41a80095c0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 msgr2=0x7f41b019dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 0x7f41b019dc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009860 tx=0x7f41a4004530 comp rx=0 tx=0).stop 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 shutdown_connections 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b01074f0 0x7f41b01978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41b00690b0 0x7f41b019dc50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41980a2350 0x7f41980a4810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 --2- 172.21.15.8:0/108817711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41b00686f0 0x7f41b019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.641+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 >> 172.21.15.8:0/108817711 conn(0x7f41b00774f0 msgr2=0x7f41b00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.642+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 shutdown_connections 2024-02-19T16:23:15.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.642+0000 7f41b6091640 1 -- 172.21.15.8:0/108817711 wait complete. 2024-02-19T16:23:15.653 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:23:15.653 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:23:15.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 -- 172.21.15.8:0/2585833984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 msgr2=0x7fa96010d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 --2- 172.21.15.8:0/2585833984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96010d2e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa958009cd0 tx=0x7fa95802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 -- 172.21.15.8:0/2585833984 shutdown_connections 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 --2- 172.21.15.8:0/2585833984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96010d2e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 --2- 172.21.15.8:0/2585833984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa960100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 --2- 172.21.15.8:0/2585833984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9601064b0 0x7fa960106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 -- 172.21.15.8:0/2585833984 >> 172.21.15.8:0/2585833984 conn(0x7fa960077550 msgr2=0x7fa960077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 -- 172.21.15.8:0/2585833984 shutdown_connections 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.757+0000 7fa967dd7640 1 -- 172.21.15.8:0/2585833984 wait complete. 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 Processor -- start 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 -- start start 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa96019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9601064b0 0x7fa960197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960197fd0 con 0x7fa960100410 2024-02-19T16:23:15.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960198140 con 0x7fa9601064b0 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa967dd7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960198300 con 0x7fa960100dd0 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa96019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa96019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41022/0 (socket says 172.21.15.8:41022) 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 -- 172.21.15.8:0/3302631405 learned_addr learned my addr 172.21.15.8:0/3302631405 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa966374640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9601064b0 0x7fa960197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965372640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 -- 172.21.15.8:0/3302631405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 msgr2=0x7fa96019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 -- 172.21.15.8:0/3302631405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9601064b0 msgr2=0x7fa960197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9601064b0 0x7fa960197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965b73640 1 -- 172.21.15.8:0/3302631405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009900 con 0x7fa960100410 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.758+0000 7fa965372640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.759+0000 7fa965b73640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa96019d7d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00e9f0 tx=0x7fa95c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.759+0000 7fa956ffd640 1 -- 172.21.15.8:0/3302631405 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c009720 con 0x7fa960100410 2024-02-19T16:23:15.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.759+0000 7fa956ffd640 1 -- 172.21.15.8:0/3302631405 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa95c00cd80 con 0x7fa960100410 2024-02-19T16:23:15.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.759+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9601985e0 con 0x7fa960100410 2024-02-19T16:23:15.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.759+0000 7fa956ffd640 1 -- 172.21.15.8:0/3302631405 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c010690 con 0x7fa960100410 2024-02-19T16:23:15.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.759+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601a4870 con 0x7fa960100410 2024-02-19T16:23:15.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.761+0000 7fa956ffd640 1 -- 172.21.15.8:0/3302631405 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fa95c0046c0 con 0x7fa960100410 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.761+0000 7fa96a9b3740 1 Processor -- start 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 -- start start 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa956ffd640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa94c0a2350 0x7fa94c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda56d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda57a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda57cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda528960 con 0x55fcda56cf80 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda4dc310 con 0x55fcda56d8a0 2024-02-19T16:23:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda4d9840 con 0x55fcda57ab60 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa966374640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa94c0a2350 0x7fa94c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda56d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda56d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41024/0 (socket says 172.21.15.8:41024) 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 -- 172.21.15.8:0/2142400478 learned_addr learned my addr 172.21.15.8:0/2142400478 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965372640 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda57a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa956ffd640 1 -- 172.21.15.8:0/3302631405 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa95c014070 con 0x7fa960100410 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa966374640 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda57cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 -- 172.21.15.8:0/2142400478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 msgr2=0x55fcda57cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda57cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 -- 172.21.15.8:0/2142400478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 msgr2=0x55fcda57a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda57a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965b73640 1 -- 172.21.15.8:0/2142400478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcda497060 con 0x55fcda56cf80 2024-02-19T16:23:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.762+0000 7fa965372640 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda57a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa965b73640 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda56d360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0a3120 tx=0x7fa95c014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a23806f78fdf442 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa966374640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa94c0a2350 0x7fa94c0a4810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa958005cc0 tx=0x7fa958032f00 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa94bfff640 1 -- 172.21.15.8:0/2142400478 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0a0520 con 0x55fcda56cf80 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa94bfff640 1 -- 172.21.15.8:0/2142400478 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa95c09ff20 con 0x55fcda56cf80 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa94bfff640 1 -- 172.21.15.8:0/2142400478 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c09f000 con 0x55fcda56cf80 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2142400478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 msgr2=0x55fcda56d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda56d360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0a3120 tx=0x7fa95c014f10 comp rx=0 tx=0).stop 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2142400478 shutdown_connections 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda57cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda57a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2142400478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda56d360 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.763+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2142400478 >> 172.21.15.8:0/2142400478 conn(0x55fcda567980 msgr2=0x55fcda567d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2142400478 shutdown_connections 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2142400478 wait complete. 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 -- start start 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 Processor -- start 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda56cf80 0x55fcda577bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda578110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda57ab60 0x55fcda578650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda4d5020 con 0x55fcda56d8a0 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda1a8430 con 0x55fcda57ab60 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda164e00 con 0x55fcda56cf80 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa96a9b3740 10 client.0 _ll_register_callbacks cb 0x55fcda47ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.764+0000 7fa965372640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda578110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965b73640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda56cf80 0x55fcda577bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda578110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41030/0 (socket says 172.21.15.8:41030) 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 -- 172.21.15.8:0/4163482360 learned_addr learned my addr 172.21.15.8:0/4163482360 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa966374640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda57ab60 0x55fcda578650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 -- 172.21.15.8:0/4163482360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda56cf80 msgr2=0x55fcda577bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda56cf80 0x55fcda577bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 -- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda57ab60 msgr2=0x55fcda578650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda57ab60 0x55fcda578650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 -- 172.21.15.8:0/4163482360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601a4870 con 0x55fcda56d8a0 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965b73640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda56cf80 0x55fcda577bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa966374640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda57ab60 0x55fcda578650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa965372640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda578110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa950013710 tx=0x7fa950013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f47fd76c1c0831 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa9497fa640 1 -- 172.21.15.8:0/4163482360 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950004020 con 0x55fcda56d8a0 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa9497fa640 1 -- 172.21.15.8:0/4163482360 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa950004180 con 0x55fcda56d8a0 2024-02-19T16:23:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcda1d47e0 con 0x55fcda56d8a0 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa9497fa640 1 -- 172.21.15.8:0/4163482360 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa950004340 con 0x55fcda56d8a0 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa96a9b3740 10 client.15777 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fcda4d36a0 con 0x55fcda56d8a0 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.765+0000 7fa9497fa640 1 -- 172.21.15.8:0/4163482360 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fcda1d47e0 con 0x55fcda56d8a0 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa9497fa640 1 -- 172.21.15.8:0/4163482360 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa95001b510 con 0x55fcda56d8a0 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa96a9b3740 10 client.15777 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa96a9b3740 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa96a9b3740 20 client.15777 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda58b110 0x55fcda58d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa96a9b3740 4 client.15777 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcda211720 con 0x55fcda58b110 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa965b73640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda58b110 0x55fcda58d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa9497fa640 10 client.15777 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.766+0000 7fa965b73640 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda58b110 0x55fcda58d530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c87bb94f43a3975 server_cookie=964740a366aebf74 in_seq=0 out_seq=0 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa9497fa640 1 -- 172.21.15.8:0/4163482360 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fcda211720 con 0x55fcda58b110 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa9497fa640 10 client.15777 handle_command_reply: tid=0 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 client.15777 shutdown 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 msgr2=0x55fcda578110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda578110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa950013710 tx=0x7fa950013bd0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 shutdown_connections 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda58b110 0x55fcda58d530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda57ab60 0x55fcda578650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda578110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa9497fa640 0 client.15777 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/4163482360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda56cf80 0x55fcda577bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 >> 172.21.15.8:0/4163482360 conn(0x55fcda567980 msgr2=0x55fcda5696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 shutdown_connections 2024-02-19T16:23:15.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 1 -- 172.21.15.8:0/4163482360 wait complete. 2024-02-19T16:23:15.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.767+0000 7fa96a9b3740 20 client.15777 trim_cache size 0 max 16384 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 Processor -- start 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 -- start start 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56cf80 0x55fcda578470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda569470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda585ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda45a130 con 0x55fcda56d8a0 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda5671f0 con 0x55fcda56cf80 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda5737f0 con 0x55fcda57ab60 2024-02-19T16:23:15.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda569470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda569470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41034/0 (socket says 172.21.15.8:41034) 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 -- 172.21.15.8:0/2177276675 learned_addr learned my addr 172.21.15.8:0/2177276675 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa966374640 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda585ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965b73640 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56cf80 0x55fcda578470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 -- 172.21.15.8:0/2177276675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 msgr2=0x55fcda585ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda585ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 -- 172.21.15.8:0/2177276675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56cf80 msgr2=0x55fcda578470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56cf80 0x55fcda578470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 -- 172.21.15.8:0/2177276675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcda46fc20 con 0x55fcda56d8a0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa966374640 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda585ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.780+0000 7fa965372640 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda569470 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa950000c00 tx=0x7fa950011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b6b0c039445fb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa94b7fe640 1 -- 172.21.15.8:0/2177276675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950009e70 con 0x55fcda56d8a0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa94b7fe640 1 -- 172.21.15.8:0/2177276675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa950005550 con 0x55fcda56d8a0 2024-02-19T16:23:15.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa94b7fe640 1 -- 172.21.15.8:0/2177276675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950003850 con 0x55fcda56d8a0 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2177276675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 msgr2=0x55fcda569470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda569470 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa950000c00 tx=0x7fa950011810 comp rx=0 tx=0).stop 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2177276675 shutdown_connections 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda585ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56d8a0 0x55fcda569470 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/2177276675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56cf80 0x55fcda578470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2177276675 >> 172.21.15.8:0/2177276675 conn(0x55fcda567980 msgr2=0x55fcda568d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2177276675 shutdown_connections 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 -- 172.21.15.8:0/2177276675 wait complete. 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.781+0000 7fa96a9b3740 1 -- start start 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 Processor -- start 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda574910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda574e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda5e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda575570 con 0x55fcda56cf80 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda5e5f00 con 0x55fcda56d8a0 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda5e6070 con 0x55fcda57ab60 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa96a9b3740 10 client.0 _ll_register_callbacks cb 0x7fa950009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda574910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda574910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41044/0 (socket says 172.21.15.8:41044) 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 -- 172.21.15.8:0/449784229 learned_addr learned my addr 172.21.15.8:0/449784229 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa966374640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda5e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965372640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda574e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 -- 172.21.15.8:0/449784229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 msgr2=0x55fcda5e59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda5e59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 -- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 msgr2=0x55fcda574e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda574e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.782+0000 7fa965b73640 1 -- 172.21.15.8:0/449784229 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96010dbe0 con 0x55fcda56cf80 2024-02-19T16:23:15.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa965b73640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda574910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa95c097310 tx=0x7fa95c00cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f413165551f601b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa9497fa640 1 -- 172.21.15.8:0/449784229 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c09b5b0 con 0x55fcda56cf80 2024-02-19T16:23:15.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa9497fa640 1 -- 172.21.15.8:0/449784229 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa95c0a3650 con 0x55fcda56cf80 2024-02-19T16:23:15.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa95c09ff20 con 0x55fcda56cf80 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa9497fa640 1 -- 172.21.15.8:0/449784229 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa95c09f000 con 0x55fcda56cf80 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 10 client.15783 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fcda4d36a0 con 0x55fcda56cf80 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa9497fa640 1 -- 172.21.15.8:0/449784229 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa95c0a7080 con 0x55fcda56cf80 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa9497fa640 1 -- 172.21.15.8:0/449784229 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa95c01f060 con 0x55fcda56cf80 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 10 client.15783 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 20 client.15783 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda5e75b0 0x55fcda5e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 4 client.15783 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:23:15.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fcda260890 con 0x55fcda5e75b0 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.783+0000 7fa965372640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda5e75b0 0x55fcda5e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa9497fa640 10 client.15783 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa965372640 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda5e75b0 0x55fcda5e99a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6e12a0a159d145 server_cookie=cac416d37460341d in_seq=0 out_seq=0 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa9497fa640 1 -- 172.21.15.8:0/449784229 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fcda260890 con 0x55fcda5e75b0 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa9497fa640 10 client.15783 handle_command_reply: tid=0 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 client.15783 shutdown 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 msgr2=0x55fcda574910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda574910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa95c097310 tx=0x7fa95c00cfa0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 shutdown_connections 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcda5e75b0 0x55fcda5e99a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcda57ab60 0x55fcda5e59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa9497fa640 0 client.15783 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcda56d8a0 0x55fcda574e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 --2- 172.21.15.8:0/449784229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcda56cf80 0x55fcda574910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 >> 172.21.15.8:0/449784229 conn(0x55fcda567980 msgr2=0x55fcda5695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 shutdown_connections 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 1 -- 172.21.15.8:0/449784229 wait complete. 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.784+0000 7fa96a9b3740 20 client.15783 trim_cache size 0 max 16384 2024-02-19T16:23:15.793 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:23:15.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa94c0a2350 msgr2=0x7fa94c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa94c0a2350 0x7fa94c0a4810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa958005cc0 tx=0x7fa958032f00 comp rx=0 tx=0).stop 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 msgr2=0x7fa96019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa96019d7d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00e9f0 tx=0x7fa95c00edf0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 shutdown_connections 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9601064b0 0x7fa960197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa94c0a2350 0x7fa94c0a4810 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa960100dd0 0x7fa96019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 --2- 172.21.15.8:0/3302631405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa960100410 0x7fa96019d7d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 >> 172.21.15.8:0/3302631405 conn(0x7fa960077550 msgr2=0x7fa9600ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 shutdown_connections 2024-02-19T16:23:15.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.785+0000 7fa967dd7640 1 -- 172.21.15.8:0/3302631405 wait complete. 2024-02-19T16:23:15.796 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:23:15.797 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:23:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 -- 172.21.15.8:0/856409929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1198069a60 msgr2=0x7f1198069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 --2- 172.21.15.8:0/856409929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1198069a60 0x7f1198069ee0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002bc30 comp rx=0 tx=0).stop 2024-02-19T16:23:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 -- 172.21.15.8:0/856409929 shutdown_connections 2024-02-19T16:23:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 --2- 172.21.15.8:0/856409929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f119806a420 0x7f119810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 --2- 172.21.15.8:0/856409929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1198069a60 0x7f1198069ee0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 --2- 172.21.15.8:0/856409929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1198107560 0x7f1198107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.900+0000 7f119d3bc640 1 -- 172.21.15.8:0/856409929 >> 172.21.15.8:0/856409929 conn(0x7f11980773d0 msgr2=0x7f11980777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.901+0000 7f119d3bc640 1 -- 172.21.15.8:0/856409929 shutdown_connections 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.901+0000 7f119d3bc640 1 -- 172.21.15.8:0/856409929 wait complete. 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.901+0000 7f119d3bc640 1 Processor -- start 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.901+0000 7f119d3bc640 1 -- start start 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.901+0000 7f119d3bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1198069a60 0x7f119819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.901+0000 7f119d3bc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 0x7f119819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f119d3bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1198107560 0x7f1198197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f119d3bc640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1198197eb0 con 0x7f119806a420 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f119d3bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1198198020 con 0x7f1198069a60 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f119d3bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11981981e0 con 0x7f1198107560 2024-02-19T16:23:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 0x7f119819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 0x7f119819dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41054/0 (socket says 172.21.15.8:41054) 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 -- 172.21.15.8:0/2145316192 learned_addr learned my addr 172.21.15.8:0/2145316192 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11977fe640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1198107560 0x7f1198197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f1196ffd640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1198069a60 0x7f119819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 -- 172.21.15.8:0/2145316192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1198107560 msgr2=0x7f1198197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1198107560 0x7f1198197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 -- 172.21.15.8:0/2145316192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1198069a60 msgr2=0x7f119819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1198069a60 0x7f119819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 -- 172.21.15.8:0/2145316192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009560 con 0x7f119806a420 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f11967fc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 0x7f119819dbf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f118002c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f118ffff640 1 -- 172.21.15.8:0/2145316192 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118002dea0 con 0x7f119806a420 2024-02-19T16:23:15.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1198198460 con 0x7f119806a420 2024-02-19T16:23:15.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f118ffff640 1 -- 172.21.15.8:0/2145316192 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1180004410 con 0x7f119806a420 2024-02-19T16:23:15.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f118ffff640 1 -- 172.21.15.8:0/2145316192 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180003bb0 con 0x7f119806a420 2024-02-19T16:23:15.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.902+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11981a4750 con 0x7f119806a420 2024-02-19T16:23:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f118ffff640 1 -- 172.21.15.8:0/2145316192 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f1180003d10 con 0x7f119806a420 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 Processor -- start 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 -- start start 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d60c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d615210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f118ffff640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11840a2350 0x7f11840a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d617b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d5c7960 con 0x55fd7d60cb70 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d57b310 con 0x55fd7d615750 2024-02-19T16:23:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f119ff96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d578840 con 0x55fd7d60c250 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d615210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.905+0000 7f11977fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d60c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d615210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41060/0 (socket says 172.21.15.8:41060) 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f11977fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d60c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51980/0 (socket says 172.21.15.8:51980) 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 -- 172.21.15.8:0/1955894043 learned_addr learned my addr 172.21.15.8:0/1955894043 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f11967fc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11840a2350 0x7f11840a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f118ffff640 1 -- 172.21.15.8:0/2145316192 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f11800f0bd0 con 0x7f119806a420 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f11977fe640 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d617b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 -- 172.21.15.8:0/1955894043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 msgr2=0x55fd7d60c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d60c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 -- 172.21.15.8:0/1955894043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 msgr2=0x55fd7d617b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d617b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 -- 172.21.15.8:0/1955894043 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7d536060 con 0x55fd7d60cb70 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f11967fc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11840a2350 0x7f11840a4810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f11800edf60 tx=0x7f1180005e20 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f1196ffd640 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d615210 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1190009c90 tx=0x7f1190011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff23ce8d1339805 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f118d7fa640 1 -- 172.21.15.8:0/1955894043 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190005220 con 0x55fd7d60cb70 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f118d7fa640 1 -- 172.21.15.8:0/1955894043 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1190004c80 con 0x55fd7d60cb70 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.906+0000 7f118d7fa640 1 -- 172.21.15.8:0/1955894043 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190019670 con 0x55fd7d60cb70 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 -- 172.21.15.8:0/1955894043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 msgr2=0x55fd7d615210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d615210 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1190009c90 tx=0x7f1190011810 comp rx=0 tx=0).stop 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 -- 172.21.15.8:0/1955894043 shutdown_connections 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d617b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d615210 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 --2- 172.21.15.8:0/1955894043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d60c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 -- 172.21.15.8:0/1955894043 >> 172.21.15.8:0/1955894043 conn(0x55fd7d606980 msgr2=0x55fd7d606d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 -- 172.21.15.8:0/1955894043 shutdown_connections 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 -- 172.21.15.8:0/1955894043 wait complete. 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 -- start start 2024-02-19T16:23:15.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.907+0000 7f119ff96740 1 Processor -- start 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d612760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60cb70 0x55fd7d612ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d6131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d574020 con 0x55fd7d60c250 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d247430 con 0x55fd7d615750 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d203e00 con 0x55fd7d60cb70 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 10 client.0 _ll_register_callbacks cb 0x55fd7d51bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d612760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d612760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41068/0 (socket says 172.21.15.8:41068) 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 -- 172.21.15.8:0/14450938 learned_addr learned my addr 172.21.15.8:0/14450938 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f1196ffd640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60cb70 0x55fd7d612ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d6131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 -- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60cb70 msgr2=0x55fd7d612ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60cb70 0x55fd7d612ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 -- 172.21.15.8:0/14450938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 msgr2=0x55fd7d6131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d6131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 -- 172.21.15.8:0/14450938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11900098f0 con 0x55fd7d60c250 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f1196ffd640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60cb70 0x55fd7d612ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f11977fe640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d612760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1188011c40 tx=0x7f1188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9f86bd5e12b8ec server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f117e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f117e7fc640 1 -- 172.21.15.8:0/14450938 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188016070 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f117e7fc640 1 -- 172.21.15.8:0/14450938 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f1188010cc0 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd7d2737e0 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f117e7fc640 1 -- 172.21.15.8:0/14450938 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1188010e80 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 10 client.15795 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.908+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fd7d5726a0 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f117e7fc640 1 -- 172.21.15.8:0/14450938 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd7d2737e0 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f117e7fc640 1 -- 172.21.15.8:0/14450938 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f118800e5a0 con 0x55fd7d60c250 2024-02-19T16:23:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f119ff96740 10 client.15795 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f119ff96740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f119ff96740 20 client.15795 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f119ff96740 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d625bd0 0x55fd7d627ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f119ff96740 4 client.15795 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd7d2b0720 con 0x55fd7d625bd0 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f1196ffd640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d625bd0 0x55fd7d627ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f117e7fc640 10 client.15795 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.909+0000 7f1196ffd640 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d625bd0 0x55fd7d627ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bef5e30306580d5 server_cookie=31b3b67d545675 in_seq=0 out_seq=0 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f117e7fc640 1 -- 172.21.15.8:0/14450938 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fd7d2b0720 con 0x55fd7d625bd0 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f117e7fc640 10 client.15795 handle_command_reply: tid=0 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 client.15795 shutdown 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 msgr2=0x55fd7d612760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d612760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1188011c40 tx=0x7f1188004a40 comp rx=0 tx=0).stop 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 shutdown_connections 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d6131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d625bd0 0x55fd7d627ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60cb70 0x55fd7d612ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 --2- 172.21.15.8:0/14450938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d612760 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f117e7fc640 0 client.15795 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 >> 172.21.15.8:0/14450938 conn(0x55fd7d606980 msgr2=0x55fd7d6086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 shutdown_connections 2024-02-19T16:23:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 1 -- 172.21.15.8:0/14450938 wait complete. 2024-02-19T16:23:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.910+0000 7f119ff96740 20 client.15795 trim_cache size 0 max 16384 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 Processor -- start 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 -- start start 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d614e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d605a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d620ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d4f9130 con 0x55fd7d60cb70 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d6025c0 con 0x55fd7d615750 2024-02-19T16:23:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f119ff96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d622640 con 0x55fd7d60c250 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f11977fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d614e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d605a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f11977fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d614e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51986/0 (socket says 172.21.15.8:51986) 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f11977fe640 1 -- 172.21.15.8:0/1901562076 learned_addr learned my addr 172.21.15.8:0/1901562076 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f11977fe640 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d620ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 -- 172.21.15.8:0/1901562076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 msgr2=0x55fd7d614e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d614e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 -- 172.21.15.8:0/1901562076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 msgr2=0x55fd7d620ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d620ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 -- 172.21.15.8:0/1901562076 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7d50ec20 con 0x55fd7d60cb70 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f11977fe640 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d614e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.923+0000 7f1196ffd640 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d605a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1190011d10 tx=0x7f11900056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c09bcc05668d89f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f118cff9640 1 -- 172.21.15.8:0/1901562076 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190029040 con 0x55fd7d60cb70 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f118cff9640 1 -- 172.21.15.8:0/1901562076 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1190011d40 con 0x55fd7d60cb70 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f118cff9640 1 -- 172.21.15.8:0/1901562076 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119002d600 con 0x55fd7d60cb70 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 -- 172.21.15.8:0/1901562076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 msgr2=0x55fd7d605a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d605a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1190011d10 tx=0x7f11900056b0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 -- 172.21.15.8:0/1901562076 shutdown_connections 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d615750 0x55fd7d620ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60cb70 0x55fd7d605a00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 --2- 172.21.15.8:0/1901562076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d60c250 0x55fd7d614e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 -- 172.21.15.8:0/1901562076 >> 172.21.15.8:0/1901562076 conn(0x55fd7d606980 msgr2=0x55fd7d6052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 -- 172.21.15.8:0/1901562076 shutdown_connections 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.924+0000 7f119ff96740 1 -- 172.21.15.8:0/1901562076 wait complete. 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 -- start start 2024-02-19T16:23:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 Processor -- start 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d67a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d60cb70 0x55fd7d67ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d615750 0x55fd7d60f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d60fb40 con 0x55fd7d60c250 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d60fcb0 con 0x55fd7d60cb70 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d60fe70 con 0x55fd7d615750 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f119ff96740 10 client.0 _ll_register_callbacks cb 0x7f1188011120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d67a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d67a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41086/0 (socket says 172.21.15.8:41086) 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 -- 172.21.15.8:0/3288574321 learned_addr learned my addr 172.21.15.8:0/3288574321 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f1196ffd640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d60cb70 0x55fd7d67ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d615750 0x55fd7d60f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 -- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d615750 msgr2=0x55fd7d60f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d615750 0x55fd7d60f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 -- 172.21.15.8:0/3288574321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d60cb70 msgr2=0x55fd7d67ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d60cb70 0x55fd7d67ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 -- 172.21.15.8:0/3288574321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd7d5726a0 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f11977fe640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d67a7f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f1188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bb74b41ce8d832 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.925+0000 7f117e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f117e7fc640 1 -- 172.21.15.8:0/3288574321 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188016040 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f117e7fc640 1 -- 172.21.15.8:0/3288574321 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f118800ed30 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd7d2ff890 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f117e7fc640 1 -- 172.21.15.8:0/3288574321 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118800e3e0 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 10 client.15801 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fd7d536060 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f117e7fc640 1 -- 172.21.15.8:0/3288574321 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd7d2ff890 con 0x55fd7d60c250 2024-02-19T16:23:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f117e7fc640 1 -- 172.21.15.8:0/3288574321 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1188022680 con 0x55fd7d60c250 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 10 client.15801 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 20 client.15801 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d682540 0x55fd7d684960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 4 client.15801 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd7d21f700 con 0x55fd7d682540 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.926+0000 7f1196ffd640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d682540 0x55fd7d684960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f117e7fc640 10 client.15801 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f1196ffd640 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d682540 0x55fd7d684960 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e144c1574cf5bff server_cookie=767b01c9b17aa42b in_seq=0 out_seq=0 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f117e7fc640 1 -- 172.21.15.8:0/3288574321 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd7d21f700 con 0x55fd7d682540 2024-02-19T16:23:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f117e7fc640 10 client.15801 handle_command_reply: tid=0 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 client.15801 shutdown 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 msgr2=0x55fd7d67a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d67a7f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f1188004a40 comp rx=0 tx=0).stop 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 shutdown_connections 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55fd7d682540 0x55fd7d684960 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd7d615750 0x55fd7d60f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd7d60cb70 0x55fd7d67ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f117e7fc640 0 client.15801 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 --2- 172.21.15.8:0/3288574321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd7d60c250 0x55fd7d67a7f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 >> 172.21.15.8:0/3288574321 conn(0x55fd7d606980 msgr2=0x55fd7d610c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 shutdown_connections 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 1 -- 172.21.15.8:0/3288574321 wait complete. 2024-02-19T16:23:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.927+0000 7f119ff96740 20 client.15801 trim_cache size 0 max 16384 2024-02-19T16:23:15.937 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:23:15.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11840a2350 msgr2=0x7f11840a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11840a2350 0x7f11840a4810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f11800edf60 tx=0x7f1180005e20 comp rx=0 tx=0).stop 2024-02-19T16:23:15.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 msgr2=0x7f119819dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:15.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 0x7f119819dbf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f118002c6b0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 shutdown_connections 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11840a2350 0x7f11840a4810 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1198107560 0x7f1198197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f119806a420 0x7f119819dbf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 --2- 172.21.15.8:0/2145316192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1198069a60 0x7f119819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 >> 172.21.15.8:0/2145316192 conn(0x7f11980773d0 msgr2=0x7f11980ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 shutdown_connections 2024-02-19T16:23:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:15.928+0000 7f119d3bc640 1 -- 172.21.15.8:0/2145316192 wait complete. 2024-02-19T16:23:16.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:15 smithi008 ceph-mon[32573]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-19T16:23:16.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:15 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2297603401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:23:16.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:15 smithi191 ceph-mon[37203]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-19T16:23:16.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:15 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2297603401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:23:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:15 smithi156 ceph-mon[36502]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-02-19T16:23:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:15 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2297603401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:23:16.941 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:23:16.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:23:16.941 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:23:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 -- 172.21.15.8:0/1763011929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 msgr2=0x7fb4a810d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/1763011929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a810d2e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009cd0 tx=0x7fb4a002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 -- 172.21.15.8:0/1763011929 shutdown_connections 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/1763011929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a810d2e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/1763011929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 0x7fb4a8100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/1763011929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 0x7fb4a8106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 -- 172.21.15.8:0/1763011929 >> 172.21.15.8:0/1763011929 conn(0x7fb4a8077550 msgr2=0x7fb4a8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 -- 172.21.15.8:0/1763011929 shutdown_connections 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.061+0000 7fb4af6ed640 1 -- 172.21.15.8:0/1763011929 wait complete. 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 Processor -- start 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 -- start start 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 0x7fb4a819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 0x7fb4a81968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8196ef0 con 0x7fb4a8100dd0 2024-02-19T16:23:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8197060 con 0x7fb4a81064b0 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4af6ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8197220 con 0x7fb4a8100410 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4acc88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4ad489640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 0x7fb4a819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4acc88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41112/0 (socket says 172.21.15.8:41112) 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4acc88640 1 -- 172.21.15.8:0/2961199737 learned_addr learned my addr 172.21.15.8:0/2961199737 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.062+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 0x7fb4a81968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4acc88640 1 -- 172.21.15.8:0/2961199737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 msgr2=0x7fb4a819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4acc88640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 0x7fb4a819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4acc88640 1 -- 172.21.15.8:0/2961199737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 msgr2=0x7fb4a81968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4acc88640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 0x7fb4a81968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4acc88640 1 -- 172.21.15.8:0/2961199737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0009900 con 0x7fb4a8100dd0 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 0x7fb4a81968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4ad489640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 0x7fb4a819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4acc88640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a819cc30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb49800e9f0 tx=0x7fb49800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb49e7fc640 1 -- 172.21.15.8:0/2961199737 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498009720 con 0x7fb4a8100dd0 2024-02-19T16:23:17.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb49e7fc640 1 -- 172.21.15.8:0/2961199737 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb49800cd80 con 0x7fb4a8100dd0 2024-02-19T16:23:17.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb49e7fc640 1 -- 172.21.15.8:0/2961199737 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498010690 con 0x7fb4a8100dd0 2024-02-19T16:23:17.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a8197500 con 0x7fb4a8100dd0 2024-02-19T16:23:17.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.063+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a81a3790 con 0x7fb4a8100dd0 2024-02-19T16:23:17.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.065+0000 7fb49e7fc640 1 -- 172.21.15.8:0/2961199737 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fb4980046c0 con 0x7fb4a8100dd0 2024-02-19T16:23:17.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 Processor -- start 2024-02-19T16:23:17.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 -- start start 2024-02-19T16:23:17.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb49e7fc640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4940a2350 0x7fb4940a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 0x5569eab19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19d30 0x5569eab222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eaa87f90 con 0x5569eab19380 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eaa854c0 con 0x5569eab228b0 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eaa80ca0 con 0x5569eab19d30 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4ad489640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 0x5569eab19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4940a2350 0x7fb4940a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4ad489640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 0x5569eab19760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41126/0 (socket says 172.21.15.8:41126) 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4ad489640 1 -- 172.21.15.8:0/272658556 learned_addr learned my addr 172.21.15.8:0/272658556 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb49e7fc640 1 -- 172.21.15.8:0/2961199737 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb498014070 con 0x7fb4a8100dd0 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.066+0000 7fb4acc88640 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19d30 0x5569eab222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4ad489640 1 -- 172.21.15.8:0/272658556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19d30 msgr2=0x5569eab222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4ad489640 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19d30 0x5569eab222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4ad489640 1 -- 172.21.15.8:0/272658556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 msgr2=0x5569eab24d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4ad489640 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab24d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4ad489640 1 -- 172.21.15.8:0/272658556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569eaa430b0 con 0x5569eab19380 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4940a2350 0x7fb4940a4810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0005cc0 tx=0x7fb4a0005ef0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4ad489640 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 0x5569eab19760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4011470 tx=0x7fb4a4011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1990cf907e134d25 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4937fe640 1 -- 172.21.15.8:0/272658556 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4004030 con 0x5569eab19380 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4937fe640 1 -- 172.21.15.8:0/272658556 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4004190 con 0x5569eab19380 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.067+0000 7fb4937fe640 1 -- 172.21.15.8:0/272658556 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400f800 con 0x5569eab19380 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 -- 172.21.15.8:0/272658556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 msgr2=0x5569eab19760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 0x5569eab19760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4011470 tx=0x7fb4a4011930 comp rx=0 tx=0).stop 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 -- 172.21.15.8:0/272658556 shutdown_connections 2024-02-19T16:23:17.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab24d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19d30 0x5569eab222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/272658556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19380 0x5569eab19760 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 -- 172.21.15.8:0/272658556 >> 172.21.15.8:0/272658556 conn(0x5569eab13840 msgr2=0x5569eab13c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 -- 172.21.15.8:0/272658556 shutdown_connections 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 -- 172.21.15.8:0/272658556 wait complete. 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 -- start start 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.068+0000 7fb4b23a3740 1 Processor -- start 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab1fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab20280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eaa25f40 con 0x5569eab19d30 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ea7544b0 con 0x5569eab228b0 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eaa06670 con 0x5569eab19380 2024-02-19T16:23:17.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 10 client.0 _ll_register_callbacks cb 0x5569eaa73f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab1fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab1fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41138/0 (socket says 172.21.15.8:41138) 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 -- 172.21.15.8:0/1120944758 learned_addr learned my addr 172.21.15.8:0/1120944758 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4ad489640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab20280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 -- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 msgr2=0x5569eab1f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab1f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 -- 172.21.15.8:0/1120944758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 msgr2=0x5569eab20280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab20280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 -- 172.21.15.8:0/1120944758 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a4009c40 con 0x5569eab19d30 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4acc88640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab1fd40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb49800c6a0 tx=0x7fb49806dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0e9222fcfe837f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb490ff9640 1 -- 172.21.15.8:0/1120944758 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49809b2d0 con 0x5569eab19d30 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb490ff9640 1 -- 172.21.15.8:0/1120944758 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb49806d390 con 0x5569eab19d30 2024-02-19T16:23:17.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.069+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569eaa5af40 con 0x5569eab19d30 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb490ff9640 1 -- 172.21.15.8:0/1120944758 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb49806da20 con 0x5569eab19d30 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 10 client.15813 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569eaa7f320 con 0x5569eab19d30 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb490ff9640 1 -- 172.21.15.8:0/1120944758 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4980b7030 con 0x5569eab19d30 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb490ff9640 1 -- 172.21.15.8:0/1120944758 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb49801f060 con 0x5569eab19d30 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 10 client.15813 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 20 client.15813 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab32ea0 0x5569eab352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 4 client.15813 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569eaa77310 con 0x5569eab32ea0 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.070+0000 7fb4ad489640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab32ea0 0x5569eab352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb490ff9640 10 client.15813 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4ad489640 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab32ea0 0x5569eab352c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90a156085dd49fa server_cookie=262fb18bf1a4a0f0 in_seq=0 out_seq=0 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb490ff9640 1 -- 172.21.15.8:0/1120944758 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5569eaa77310 con 0x5569eab32ea0 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb490ff9640 10 client.15813 handle_command_reply: tid=0 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 client.15813 shutdown 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 msgr2=0x5569eab1fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab1fd40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb49800c6a0 tx=0x7fb49806dc10 comp rx=0 tx=0).stop 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 shutdown_connections 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab20280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab1fd40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab32ea0 0x5569eab352c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/1120944758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab1f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 >> 172.21.15.8:0/1120944758 conn(0x5569eab13840 msgr2=0x5569eab157a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.071+0000 7fb490ff9640 0 client.15813 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.072+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 shutdown_connections 2024-02-19T16:23:17.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.072+0000 7fb4b23a3740 1 -- 172.21.15.8:0/1120944758 wait complete. 2024-02-19T16:23:17.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.072+0000 7fb4b23a3740 20 client.15813 trim_cache size 0 max 16384 2024-02-19T16:23:17.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 Processor -- start 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 -- start start 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab21f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eab37b70 con 0x5569eab19d30 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eab35920 con 0x5569eab228b0 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eab186a0 con 0x5569eab19380 2024-02-19T16:23:17.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41150/0 (socket says 172.21.15.8:41150) 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 -- 172.21.15.8:0/544165787 learned_addr learned my addr 172.21.15.8:0/544165787 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4ad489640 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab21f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 -- 172.21.15.8:0/544165787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 msgr2=0x5569eab21f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab21f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 -- 172.21.15.8:0/544165787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 msgr2=0x5569eab2ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab2ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.088+0000 7fb4acc88640 1 -- 172.21.15.8:0/544165787 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569eaa1bc00 con 0x5569eab19d30 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4acc88640 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab200a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb49809ae00 tx=0x7fb498004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160a290291ec7a7b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb492ffd640 1 -- 172.21.15.8:0/544165787 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49806da20 con 0x5569eab19d30 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb492ffd640 1 -- 172.21.15.8:0/544165787 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4980b7070 con 0x5569eab19d30 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb492ffd640 1 -- 172.21.15.8:0/544165787 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49809b330 con 0x5569eab19d30 2024-02-19T16:23:17.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 -- 172.21.15.8:0/544165787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 msgr2=0x5569eab200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab200a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb49809ae00 tx=0x7fb498004a30 comp rx=0 tx=0).stop 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 -- 172.21.15.8:0/544165787 shutdown_connections 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab228b0 0x5569eab2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab19d30 0x5569eab200a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/544165787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab21f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 -- 172.21.15.8:0/544165787 >> 172.21.15.8:0/544165787 conn(0x5569eab13840 msgr2=0x5569eab128e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 -- 172.21.15.8:0/544165787 shutdown_connections 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.089+0000 7fb4b23a3740 1 -- 172.21.15.8:0/544165787 wait complete. 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 -- start start 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 Processor -- start 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab2a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab19d30 0x5569eab36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 0x5569eab37080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eab2ac00 con 0x5569eab228b0 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eab375c0 con 0x5569eab19d30 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569eab37780 con 0x5569eab19380 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4b23a3740 10 client.0 _ll_register_callbacks cb 0x7fb49809acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4adc8a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 0x5569eab37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4ad489640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab2a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4adc8a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 0x5569eab37080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41162/0 (socket says 172.21.15.8:41162) 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4adc8a640 1 -- 172.21.15.8:0/76498133 learned_addr learned my addr 172.21.15.8:0/76498133 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.090+0000 7fb4acc88640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab19d30 0x5569eab36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4adc8a640 1 -- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 msgr2=0x5569eab2a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab2a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4adc8a640 1 -- 172.21.15.8:0/76498133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab19d30 msgr2=0x5569eab36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab19d30 0x5569eab36b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4adc8a640 1 -- 172.21.15.8:0/76498133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569eaa7f320 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4acc88640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab19d30 0x5569eab36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4adc8a640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 0x5569eab37080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009cd0 tx=0x7fb4a002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123536427e260b0d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb490ff9640 1 -- 172.21.15.8:0/76498133 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004650 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb490ff9640 1 -- 172.21.15.8:0/76498133 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb4a00047b0 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569eaaa6900 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb490ff9640 1 -- 172.21.15.8:0/76498133 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4a0042450 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4b23a3740 10 client.15819 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569eaa41910 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb490ff9640 1 -- 172.21.15.8:0/76498133 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5569eaaa6900 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb490ff9640 1 -- 172.21.15.8:0/76498133 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb4a0044810 con 0x5569eab228b0 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4b23a3740 10 client.15819 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4b23a3740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.091+0000 7fb4b23a3740 20 client.15819 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:17.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.092+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab94020 0x5569eab96440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.092+0000 7fb4b23a3740 4 client.15819 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.092+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5569ea780860 con 0x5569eab94020 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.092+0000 7fb4ad489640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab94020 0x5569eab96440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.092+0000 7fb490ff9640 10 client.15819 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.092+0000 7fb4ad489640 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab94020 0x5569eab96440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbca7ab5a5560c8d server_cookie=125dcd6ecf3d8394 in_seq=0 out_seq=0 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb490ff9640 1 -- 172.21.15.8:0/76498133 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5569ea780860 con 0x5569eab94020 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb490ff9640 10 client.15819 handle_command_reply: tid=0 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 client.15819 shutdown 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 msgr2=0x5569eab37080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 0x5569eab37080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009cd0 tx=0x7fb4a002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 shutdown_connections 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5569eab228b0 0x5569eab37080 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569eab19d30 0x5569eab36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5569eab94020 0x5569eab96440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 --2- 172.21.15.8:0/76498133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569eab19380 0x5569eab2a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 >> 172.21.15.8:0/76498133 conn(0x5569eab13840 msgr2=0x5569eab39d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb490ff9640 0 client.15819 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.093+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 shutdown_connections 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4b23a3740 1 -- 172.21.15.8:0/76498133 wait complete. 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4b23a3740 20 client.15819 trim_cache size 0 max 16384 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:23:17.101 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:23:17.102 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "d48c4b01-5418-44aa-9046-6559877ea731", 2024-02-19T16:23:17.102 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:23:17.102 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4940a2350 msgr2=0x7fb4940a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4940a2350 0x7fb4940a4810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0005cc0 tx=0x7fb4a0005ef0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 msgr2=0x7fb4a819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a819cc30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb49800e9f0 tx=0x7fb49800edf0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 shutdown_connections 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4a81064b0 0x7fb4a81968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4a8100dd0 0x7fb4a819cc30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4940a2350 0x7fb4940a4810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 --2- 172.21.15.8:0/2961199737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8100410 0x7fb4a819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 >> 172.21.15.8:0/2961199737 conn(0x7fb4a8077550 msgr2=0x7fb4a8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 shutdown_connections 2024-02-19T16:23:17.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.094+0000 7fb4af6ed640 1 -- 172.21.15.8:0/2961199737 wait complete. 2024-02-19T16:23:17.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:23:17.106 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:23:17.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 -- 172.21.15.8:0/708126035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6284107600 msgr2=0x7f62841079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 --2- 172.21.15.8:0/708126035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6284107600 0x7f62841079e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6274009930 tx=0x7f627402bcd0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 -- 172.21.15.8:0/708126035 shutdown_connections 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 --2- 172.21.15.8:0/708126035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628406a4e0 0x7f628410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 --2- 172.21.15.8:0/708126035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6284069b20 0x7f6284069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 --2- 172.21.15.8:0/708126035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6284107600 0x7f62841079e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 -- 172.21.15.8:0/708126035 >> 172.21.15.8:0/708126035 conn(0x7f6284077190 msgr2=0x7f62840775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 -- 172.21.15.8:0/708126035 shutdown_connections 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 -- 172.21.15.8:0/708126035 wait complete. 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 Processor -- start 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.210+0000 7f628a09d640 1 -- start start 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6284069b20 0x7f628419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 0x7f628419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6284107600 0x7f62841978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284197f20 con 0x7f628406a4e0 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284198090 con 0x7f6284069b20 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284198250 con 0x7f6284107600 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 0x7f628419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 0x7f628419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41186/0 (socket says 172.21.15.8:41186) 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 -- 172.21.15.8:0/1066840979 learned_addr learned my addr 172.21.15.8:0/1066840979 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6283fff640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6284107600 0x7f62841978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f62837fe640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6284069b20 0x7f628419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 -- 172.21.15.8:0/1066840979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6284107600 msgr2=0x7f62841978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6284107600 0x7f62841978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 -- 172.21.15.8:0/1066840979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6284069b20 msgr2=0x7f628419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6284069b20 0x7f628419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 -- 172.21.15.8:0/1066840979 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009560 con 0x7f628406a4e0 2024-02-19T16:23:17.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6282ffd640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 0x7f628419dc60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f627800e9f0 tx=0x7f627800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6280ff9640 1 -- 172.21.15.8:0/1066840979 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278009720 con 0x7f628406a4e0 2024-02-19T16:23:17.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6280ff9640 1 -- 172.21.15.8:0/1066840979 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f627800cd80 con 0x7f628406a4e0 2024-02-19T16:23:17.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6284198530 con 0x7f628406a4e0 2024-02-19T16:23:17.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f6280ff9640 1 -- 172.21.15.8:0/1066840979 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278010690 con 0x7f628406a4e0 2024-02-19T16:23:17.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.211+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62841a47c0 con 0x7f628406a4e0 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f6280ff9640 1 -- 172.21.15.8:0/1066840979 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f62780046c0 con 0x7f628406a4e0 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 Processor -- start 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 -- start start 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f6280ff9640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f626c0a2350 0x7f626c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 0x5593dc97e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97ebd0 0x5593dc987270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc989ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc8ed280 con 0x5593dc9877b0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc8ea7b0 con 0x5593dc97ebd0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f628cc7e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc8e5f90 con 0x5593dc97e2b0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f6283fff640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f626c0a2350 0x7f626c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f62837fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 0x5593dc97e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f6283fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc989ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f62837fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 0x5593dc97e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:52084/0 (socket says 172.21.15.8:52084) 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f6280ff9640 1 -- 172.21.15.8:0/1066840979 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6278014070 con 0x7f628406a4e0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f62837fe640 1 -- 172.21.15.8:0/1878885889 learned_addr learned my addr 172.21.15.8:0/1878885889 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.214+0000 7f6282ffd640 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97ebd0 0x5593dc987270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f62837fe640 1 -- 172.21.15.8:0/1878885889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97ebd0 msgr2=0x5593dc987270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f62837fe640 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97ebd0 0x5593dc987270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f62837fe640 1 -- 172.21.15.8:0/1878885889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 msgr2=0x5593dc989ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f62837fe640 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc989ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f62837fe640 1 -- 172.21.15.8:0/1878885889 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593dc8a4570 con 0x5593dc97e2b0 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f6282ffd640 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97ebd0 0x5593dc987270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f6283fff640 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc989ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f6283fff640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f626c0a2350 0x7f626c0a4810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f627c005fd0 tx=0x7f627c009f10 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f62837fe640 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 0x5593dc97e690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6274000c00 tx=0x7f627402c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ca6998a88e4aef server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f6269ffb640 1 -- 172.21.15.8:0/1878885889 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274004840 con 0x5593dc97e2b0 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f6269ffb640 1 -- 172.21.15.8:0/1878885889 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62740049a0 con 0x5593dc97e2b0 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.215+0000 7f6269ffb640 1 -- 172.21.15.8:0/1878885889 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274005050 con 0x5593dc97e2b0 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 -- 172.21.15.8:0/1878885889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 msgr2=0x5593dc97e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 0x5593dc97e690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6274000c00 tx=0x7f627402c780 comp rx=0 tx=0).stop 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 -- 172.21.15.8:0/1878885889 shutdown_connections 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc989ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97ebd0 0x5593dc987270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 --2- 172.21.15.8:0/1878885889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97e2b0 0x5593dc97e690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 -- 172.21.15.8:0/1878885889 >> 172.21.15.8:0/1878885889 conn(0x5593dc978990 msgr2=0x5593dc978d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 -- 172.21.15.8:0/1878885889 shutdown_connections 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 -- 172.21.15.8:0/1878885889 wait complete. 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 -- start start 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.216+0000 7f628cc7e740 1 Processor -- start 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc987160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc9844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc984a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc5b9430 con 0x5593dc97ebd0 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc575e00 con 0x5593dc97e2b0 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc86afd0 con 0x5593dc9877b0 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 10 client.0 _ll_register_callbacks cb 0x5593dc8c0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc9844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc9844f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41208/0 (socket says 172.21.15.8:41208) 2024-02-19T16:23:17.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 -- 172.21.15.8:0/2654953756 learned_addr learned my addr 172.21.15.8:0/2654953756 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6283fff640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc984a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f62837fe640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc987160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 -- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 msgr2=0x5593dc984a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc984a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 -- 172.21.15.8:0/2654953756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 msgr2=0x5593dc987160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc987160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 -- 172.21.15.8:0/2654953756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628410eba0 con 0x5593dc97ebd0 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f62837fe640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc987160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6283fff640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc984a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6282ffd640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc9844f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6278097840 tx=0x7f627807ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d3165dd376e936 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6262ffd640 1 -- 172.21.15.8:0/2654953756 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627809efc0 con 0x5593dc97ebd0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f6262ffd640 1 -- 172.21.15.8:0/2654953756 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f62780a3610 con 0x5593dc97ebd0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.217+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593dc5e57e0 con 0x5593dc97ebd0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f6262ffd640 1 -- 172.21.15.8:0/2654953756 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62780b9200 con 0x5593dc97ebd0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 10 client.15831 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5593dc8e4610 con 0x5593dc97ebd0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f6262ffd640 1 -- 172.21.15.8:0/2654953756 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5593dc5e57e0 con 0x5593dc97ebd0 2024-02-19T16:23:17.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f6262ffd640 1 -- 172.21.15.8:0/2654953756 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f627801f060 con 0x5593dc97ebd0 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 10 client.15831 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 20 client.15831 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc997900 0x5593dc999d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 4 client.15831 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593dc622720 con 0x5593dc997900 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.218+0000 7f62837fe640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc997900 0x5593dc999d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f6262ffd640 10 client.15831 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f62837fe640 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc997900 0x5593dc999d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ea260f1529f685 server_cookie=101f427fa810c98b in_seq=0 out_seq=0 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f6262ffd640 1 -- 172.21.15.8:0/2654953756 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5593dc622720 con 0x5593dc997900 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f6262ffd640 10 client.15831 handle_command_reply: tid=0 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 client.15831 shutdown 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 msgr2=0x5593dc9844f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc9844f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6278097840 tx=0x7f627807ebd0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 shutdown_connections 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc997900 0x5593dc999d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc984a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f6262ffd640 0 client.15831 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc9844f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 --2- 172.21.15.8:0/2654953756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc987160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 >> 172.21.15.8:0/2654953756 conn(0x5593dc978990 msgr2=0x5593dc97a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 shutdown_connections 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.219+0000 7f628cc7e740 1 -- 172.21.15.8:0/2654953756 wait complete. 2024-02-19T16:23:17.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.220+0000 7f628cc7e740 20 client.15831 trim_cache size 0 max 16384 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 Processor -- start 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 -- start start 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc984490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc986c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc987160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc9787f0 con 0x5593dc97ebd0 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc97df10 con 0x5593dc97e2b0 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f628cc7e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc97d540 con 0x5593dc9877b0 2024-02-19T16:23:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc986c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc986c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41216/0 (socket says 172.21.15.8:41216) 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 -- 172.21.15.8:0/669610593 learned_addr learned my addr 172.21.15.8:0/669610593 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6283fff640 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc987160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f62837fe640 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc984490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 -- 172.21.15.8:0/669610593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 msgr2=0x5593dc987160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc987160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 -- 172.21.15.8:0/669610593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 msgr2=0x5593dc984490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc984490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.236+0000 7f6282ffd640 1 -- 172.21.15.8:0/669610593 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593dc880ac0 con 0x5593dc97ebd0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f6282ffd640 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc986c20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f62780cbdb0 tx=0x7f6278097240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33476a675e187051 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f62697fa640 1 -- 172.21.15.8:0/669610593 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780b9400 con 0x5593dc97ebd0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f62697fa640 1 -- 172.21.15.8:0/669610593 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62780cc230 con 0x5593dc97ebd0 2024-02-19T16:23:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f62697fa640 1 -- 172.21.15.8:0/669610593 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a36a0 con 0x5593dc97ebd0 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 -- 172.21.15.8:0/669610593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 msgr2=0x5593dc986c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc986c20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f62780cbdb0 tx=0x7f6278097240 comp rx=0 tx=0).stop 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 -- 172.21.15.8:0/669610593 shutdown_connections 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc9877b0 0x5593dc987160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc97ebd0 0x5593dc986c20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 --2- 172.21.15.8:0/669610593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc984490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 -- 172.21.15.8:0/669610593 >> 172.21.15.8:0/669610593 conn(0x5593dc978990 msgr2=0x5593dc983d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 -- 172.21.15.8:0/669610593 shutdown_connections 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.237+0000 7f628cc7e740 1 -- 172.21.15.8:0/669610593 wait complete. 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 -- start start 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 Processor -- start 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc988270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 0x5593dc9887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc98e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc988f50 con 0x5593dc9877b0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc9890c0 con 0x5593dc97e2b0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593dc98eb50 con 0x5593dc97ebd0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f628cc7e740 10 client.0 _ll_register_callbacks cb 0x7f627800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6283fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc98e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 0x5593dc9887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6283fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc98e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41222/0 (socket says 172.21.15.8:41222) 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 0x5593dc9887b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:52106/0 (socket says 172.21.15.8:52106) 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6283fff640 1 -- 172.21.15.8:0/243453656 learned_addr learned my addr 172.21.15.8:0/243453656 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f62837fe640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc988270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 -- 172.21.15.8:0/243453656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 msgr2=0x5593dc988270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc988270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 -- 172.21.15.8:0/243453656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 msgr2=0x5593dc98e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc98e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6282ffd640 1 -- 172.21.15.8:0/243453656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593dc8e4610 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f6283fff640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc98e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.238+0000 7f62837fe640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc988270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6282ffd640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 0x5593dc9887b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5593dc622860 tx=0x7f62780a3430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e37d6bcaf3ebfa server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6262ffd640 1 -- 172.21.15.8:0/243453656 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a3680 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62740049a0 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6262ffd640 1 -- 172.21.15.8:0/243453656 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f62780cc070 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6262ffd640 1 -- 172.21.15.8:0/243453656 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f627809efc0 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 10 client.25711 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5593dc8a66a0 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6262ffd640 1 -- 172.21.15.8:0/243453656 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62740049a0 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f6262ffd640 1 -- 172.21.15.8:0/243453656 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f627801f070 con 0x5593dc97ebd0 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 10 client.25711 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:17.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 20 client.25711 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc9f39e0 0x5593dc9f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 4 client.25711 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.239+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5593dc671890 con 0x5593dc9f39e0 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f62837fe640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc9f39e0 0x5593dc9f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f62837fe640 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc9f39e0 0x5593dc9f5dd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e80bb7f712a3b5c server_cookie=80dbf2d24cd7f66c in_seq=0 out_seq=0 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f6262ffd640 10 client.25711 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f6262ffd640 1 -- 172.21.15.8:0/243453656 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5593dc671890 con 0x5593dc9f39e0 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f6262ffd640 10 client.25711 handle_command_reply: tid=0 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 client.25711 shutdown 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 msgr2=0x5593dc9887b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 0x5593dc9887b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5593dc622860 tx=0x7f62780a3430 comp rx=0 tx=0).stop 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 shutdown_connections 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5593dc9877b0 0x5593dc98e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5593dc9f39e0 0x5593dc9f5dd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593dc97ebd0 0x5593dc9887b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 --2- 172.21.15.8:0/243453656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593dc97e2b0 0x5593dc988270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f6262ffd640 0 client.25711 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 >> 172.21.15.8:0/243453656 conn(0x5593dc978990 msgr2=0x5593dc9771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 shutdown_connections 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 1 -- 172.21.15.8:0/243453656 wait complete. 2024-02-19T16:23:17.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.240+0000 7f628cc7e740 20 client.25711 trim_cache size 0 max 16384 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: "d48c4b01-5418-44aa-9046-6559877ea731": { 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "d48c4b01-5418-44aa-9046-6559877ea731", 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:23:17.250 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:23:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f626c0a2350 msgr2=0x7f626c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f626c0a2350 0x7f626c0a4810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f627c005fd0 tx=0x7f627c009f10 comp rx=0 tx=0).stop 2024-02-19T16:23:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 msgr2=0x7f628419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 0x7f628419dc60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f627800e9f0 tx=0x7f627800edf0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 shutdown_connections 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f626c0a2350 0x7f626c0a4810 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6284107600 0x7f62841978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f628406a4e0 0x7f628419dc60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 --2- 172.21.15.8:0/1066840979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6284069b20 0x7f628419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 >> 172.21.15.8:0/1066840979 conn(0x7f6284077190 msgr2=0x7f62840ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 shutdown_connections 2024-02-19T16:23:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:17.241+0000 7f628a09d640 1 -- 172.21.15.8:0/1066840979 wait complete. 2024-02-19T16:23:17.253 INFO:tasks.cephfs.filesystem:scrub status for tag:d48c4b01-5418-44aa-9046-6559877ea731 - {'path': '/', 'tag': 'd48c4b01-5418-44aa-9046-6559877ea731', 'options': 'recursive,force'} 2024-02-19T16:23:18.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:17 smithi008 ceph-mon[32573]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-02-19T16:23:18.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:17 smithi191 ceph-mon[37203]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-02-19T16:23:18.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:17 smithi156 ceph-mon[36502]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-02-19T16:23:19.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:18 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:23:19.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:19.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:18 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:23:19.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:19.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:18 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:23:19.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:19.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:18 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:23:19.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:19.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:18 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:23:19.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:19.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:18 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:23:19.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:20.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:19 smithi008 ceph-mon[32573]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-02-19T16:23:20.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:19 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:23:20.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:19 smithi191 ceph-mon[37203]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-02-19T16:23:20.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:19 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:23:20.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:19 smithi156 ceph-mon[36502]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-02-19T16:23:20.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:19 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:23:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:21 smithi008 ceph-mon[32573]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-02-19T16:23:22.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:21 smithi191 ceph-mon[37203]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-02-19T16:23:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:21 smithi156 ceph-mon[36502]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-02-19T16:23:24.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:23 smithi008 ceph-mon[32573]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:23:24.318 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:23 smithi191 ceph-mon[37203]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:23:24.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:23 smithi156 ceph-mon[36502]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:23:26.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:25 smithi008 ceph-mon[32573]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:23:26.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:25 smithi191 ceph-mon[37203]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:23:26.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:25 smithi156 ceph-mon[36502]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:23:28.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:27 smithi008 ceph-mon[32573]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:23:28.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:27 smithi191 ceph-mon[37203]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:23:28.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:27 smithi156 ceph-mon[36502]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:23:30.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:30 smithi008 ceph-mon[32573]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-19T16:23:30.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:30 smithi191 ceph-mon[37203]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-19T16:23:30.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:30 smithi156 ceph-mon[36502]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-02-19T16:23:32.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:32 smithi008 ceph-mon[32573]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-02-19T16:23:32.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:32 smithi191 ceph-mon[37203]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-02-19T16:23:32.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:32 smithi156 ceph-mon[36502]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-02-19T16:23:34.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:34 smithi008 ceph-mon[32573]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:34.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:34 smithi191 ceph-mon[37203]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:34.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:34 smithi156 ceph-mon[36502]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:36.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:36 smithi008 ceph-mon[32573]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:36.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:36 smithi191 ceph-mon[37203]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:36.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:36 smithi156 ceph-mon[36502]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:38 smithi008 ceph-mon[32573]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-19T16:23:38.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:38 smithi191 ceph-mon[37203]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-19T16:23:38.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:38 smithi156 ceph-mon[36502]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-19T16:23:40.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:40 smithi008 ceph-mon[32573]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:40.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:40 smithi191 ceph-mon[37203]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:40.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:40 smithi156 ceph-mon[36502]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:42.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:42 smithi008 ceph-mon[32573]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:42.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:42 smithi191 ceph-mon[37203]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:42.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:42 smithi156 ceph-mon[36502]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:44.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:44 smithi008 ceph-mon[32573]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-19T16:23:44.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:44 smithi191 ceph-mon[37203]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-19T16:23:44.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:44 smithi156 ceph-mon[36502]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-19T16:23:46.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:46 smithi008 ceph-mon[32573]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:46.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:46 smithi191 ceph-mon[37203]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:46.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:46 smithi156 ceph-mon[36502]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-19T16:23:47.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:23:47.255 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.364+0000 7f62092d4640 1 -- 172.21.15.8:0/2594092981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620406a480 msgr2=0x7f620410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.364+0000 7f62092d4640 1 --2- 172.21.15.8:0/2594092981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620406a480 0x7f620410d950 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009860 tx=0x7f61f802bc00 comp rx=0 tx=0).stop 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.364+0000 7f62092d4640 1 -- 172.21.15.8:0/2594092981 shutdown_connections 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.364+0000 7f62092d4640 1 --2- 172.21.15.8:0/2594092981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 0x7f6204110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.364+0000 7f62092d4640 1 --2- 172.21.15.8:0/2594092981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620406a480 0x7f620410d950 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.364+0000 7f62092d4640 1 --2- 172.21.15.8:0/2594092981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6204069b60 0x7f6204069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.365+0000 7f62092d4640 1 -- 172.21.15.8:0/2594092981 >> 172.21.15.8:0/2594092981 conn(0x7f62040774e0 msgr2=0x7f62040778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.365+0000 7f62092d4640 1 -- 172.21.15.8:0/2594092981 shutdown_connections 2024-02-19T16:23:47.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.365+0000 7f62092d4640 1 -- 172.21.15.8:0/2594092981 wait complete. 2024-02-19T16:23:47.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.365+0000 7f62092d4640 1 Processor -- start 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 -- start start 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 0x7f620419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f620406a480 0x7f620419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 0x7f6204199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419a190 con 0x7f6204069b60 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419a300 con 0x7f620410de90 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 0x7f620419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62092d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419a4c0 con 0x7f620406a480 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 0x7f620419f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49520/0 (socket says 172.21.15.8:49520) 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 -- 172.21.15.8:0/1329136592 learned_addr learned my addr 172.21.15.8:0/1329136592 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62037fe640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 0x7f6204199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62027fc640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f620406a480 0x7f620419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 -- 172.21.15.8:0/1329136592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f620406a480 msgr2=0x7f620419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f620406a480 0x7f620419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 -- 172.21.15.8:0/1329136592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 msgr2=0x7f6204199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 0x7f6204199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f6202ffd640 1 -- 172.21.15.8:0/1329136592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc009840 con 0x7f6204069b60 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62027fc640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f620406a480 0x7f620419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.366+0000 7f62037fe640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 0x7f6204199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.367+0000 7f6202ffd640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 0x7f620419f940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f61f400d950 tx=0x7f61f400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.367+0000 7f61ebfff640 1 -- 172.21.15.8:0/1329136592 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400de80 con 0x7f6204069b60 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.367+0000 7f61ebfff640 1 -- 172.21.15.8:0/1329136592 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61f4004020 con 0x7f6204069b60 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.367+0000 7f61ebfff640 1 -- 172.21.15.8:0/1329136592 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4003d60 con 0x7f6204069b60 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.367+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8009490 con 0x7f6204069b60 2024-02-19T16:23:47.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.367+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62041a6800 con 0x7f6204069b60 2024-02-19T16:23:47.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.370+0000 7f620bf86740 1 Processor -- start 2024-02-19T16:23:47.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.370+0000 7f620bf86740 1 -- start start 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.370+0000 7f61ebfff640 1 -- 172.21.15.8:0/1329136592 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f61f4004190 con 0x7f6204069b60 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f620bf86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afeb4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f620bf86740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afebd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f620bf86740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f620bf86740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afe23280 con 0x5573afeb4bd0 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f620bf86740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afe207b0 con 0x5573afebd7b0 2024-02-19T16:23:47.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f620bf86740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afe1bf90 con 0x5573afeb42b0 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afeb4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f6202ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afebd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afeb4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39298/0 (socket says 172.21.15.8:39298) 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 -- 172.21.15.8:0/618507657 learned_addr learned my addr 172.21.15.8:0/618507657 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62027fc640 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 -- 172.21.15.8:0/618507657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 msgr2=0x5573afebfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 -- 172.21.15.8:0/618507657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 msgr2=0x5573afebd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afebd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 -- 172.21.15.8:0/618507657 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573afdda570 con 0x5573afeb42b0 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f61ebfff640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61ec0a23d0 0x7f61ec0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.371+0000 7f62037fe640 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afeb4690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e890 tx=0x7f61fc00ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9017df7dde7b05c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f61e97fa640 1 -- 172.21.15.8:0/618507657 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc011ce0 con 0x5573afeb42b0 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f61ebfff640 1 -- 172.21.15.8:0/1329136592 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f61f40ce4e0 con 0x7f6204069b60 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f61e97fa640 1 -- 172.21.15.8:0/618507657 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61fc003e50 con 0x5573afeb42b0 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f6202ffd640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61ec0a23d0 0x7f61ec0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f61e97fa640 1 -- 172.21.15.8:0/618507657 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc019430 con 0x5573afeb42b0 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f620bf86740 1 -- 172.21.15.8:0/618507657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 msgr2=0x5573afeb4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f6202ffd640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61ec0a23d0 0x7f61ec0a4890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cb760 tx=0x7f61f40d4060 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.372+0000 7f620bf86740 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afeb4690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e890 tx=0x7f61fc00ed50 comp rx=0 tx=0).stop 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 -- 172.21.15.8:0/618507657 shutdown_connections 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afebd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 --2- 172.21.15.8:0/618507657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afeb4690 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 -- 172.21.15.8:0/618507657 >> 172.21.15.8:0/618507657 conn(0x5573afeae990 msgr2=0x5573afeaed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 -- 172.21.15.8:0/618507657 shutdown_connections 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 -- 172.21.15.8:0/618507657 wait complete. 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 -- start start 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 Processor -- start 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.373+0000 7f620bf86740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afeba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f620bf86740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62027fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afeba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39314/0 (socket says 172.21.15.8:39314) 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 -- 172.21.15.8:0/1866390039 learned_addr learned my addr 172.21.15.8:0/1866390039 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afaef430 con 0x5573afeb4bd0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afaabe00 con 0x5573afebd7b0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afda0fd0 con 0x5573afeb42b0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f620bf86740 10 client.0 _ll_register_callbacks cb 0x5573afdf6fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 -- 172.21.15.8:0/1866390039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 msgr2=0x5573afebab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 -- 172.21.15.8:0/1866390039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 msgr2=0x5573afeba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afeba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc00e540 con 0x5573afeb42b0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62027fc640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afeba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f62037fe640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebd1c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61fc01b040 tx=0x7f61fc004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a64f6d54b810421 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.374+0000 7f61e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:23:47.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f61e6ff5640 1 -- 172.21.15.8:0/1866390039 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc003fc0 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f61e6ff5640 1 -- 172.21.15.8:0/1866390039 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f61fc004120 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573afb1b7e0 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f61e6ff5640 1 -- 172.21.15.8:0/1866390039 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61fc0042e0 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 10 client.25720 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5573afe1a610 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f61e6ff5640 1 -- 172.21.15.8:0/1866390039 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61fc005040 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f61e6ff5640 1 -- 172.21.15.8:0/1866390039 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f61fc027470 con 0x5573afeb42b0 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 10 client.25720 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:47.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 20 client.25720 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:47.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573afecd8f0 0x5573afecfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 4 client.25720 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:47.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.375+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573afb1b7e0 con 0x5573afecd8f0 2024-02-19T16:23:47.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.376+0000 7f62027fc640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573afecd8f0 0x5573afecfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.376+0000 7f62027fc640 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573afecd8f0 0x5573afecfd10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4ac80ce010b server_cookie=994a79156b7ee64 in_seq=0 out_seq=0 2024-02-19T16:23:47.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.376+0000 7f61e6ff5640 10 client.25720 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:47.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f61e6ff5640 1 -- 172.21.15.8:0/1866390039 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5573afb1b7e0 con 0x5573afecd8f0 2024-02-19T16:23:47.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f61e6ff5640 10 client.25720 handle_command_reply: tid=0 2024-02-19T16:23:47.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 client.25720 shutdown 2024-02-19T16:23:47.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 msgr2=0x5573afebd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebd1c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61fc01b040 tx=0x7f61fc004560 comp rx=0 tx=0).stop 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 shutdown_connections 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afebd7b0 0x5573afebab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afeb4bd0 0x5573afeba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573afecd8f0 0x5573afecfd10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f61e6ff5640 0 client.25720 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 --2- 172.21.15.8:0/1866390039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebd1c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 >> 172.21.15.8:0/1866390039 conn(0x5573afeae990 msgr2=0x5573afeb0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 shutdown_connections 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 1 -- 172.21.15.8:0/1866390039 wait complete. 2024-02-19T16:23:47.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.377+0000 7f620bf86740 20 client.25720 trim_cache size 0 max 16384 2024-02-19T16:23:47.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 Processor -- start 2024-02-19T16:23:47.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 -- start start 2024-02-19T16:23:47.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb4bd0 0x5573afebd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afec8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afeaf9e0 con 0x5573afebd7b0 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afeaf790 con 0x5573afeb4bd0 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.393+0000 7f620bf86740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afea8e20 con 0x5573afeb42b0 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afec8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62027fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb4bd0 0x5573afebd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afec8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49542/0 (socket says 172.21.15.8:49542) 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 -- 172.21.15.8:0/2682174848 learned_addr learned my addr 172.21.15.8:0/2682174848 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 -- 172.21.15.8:0/2682174848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 msgr2=0x5573afebcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 -- 172.21.15.8:0/2682174848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb4bd0 msgr2=0x5573afebd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb4bd0 0x5573afebd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 -- 172.21.15.8:0/2682174848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573afdb6ac0 con 0x5573afebd7b0 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62027fc640 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb4bd0 0x5573afebd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f62037fe640 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afec8bd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f61fc027800 tx=0x7f61fc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568d574d58894448 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f61e8ff9640 1 -- 172.21.15.8:0/2682174848 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00e760 con 0x5573afebd7b0 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.394+0000 7f61e8ff9640 1 -- 172.21.15.8:0/2682174848 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61fc011810 con 0x5573afebd7b0 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f61e8ff9640 1 -- 172.21.15.8:0/2682174848 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc004b30 con 0x5573afebd7b0 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 -- 172.21.15.8:0/2682174848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 msgr2=0x5573afec8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afec8bd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f61fc027800 tx=0x7f61fc009500 comp rx=0 tx=0).stop 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 -- 172.21.15.8:0/2682174848 shutdown_connections 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afec8bd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb4bd0 0x5573afebd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 --2- 172.21.15.8:0/2682174848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb42b0 0x5573afebcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 -- 172.21.15.8:0/2682174848 >> 172.21.15.8:0/2682174848 conn(0x5573afeae990 msgr2=0x5573afeb5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.395+0000 7f620bf86740 1 -- 172.21.15.8:0/2682174848 shutdown_connections 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 -- 172.21.15.8:0/2682174848 wait complete. 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 -- start start 2024-02-19T16:23:47.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 Processor -- start 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb42b0 0x5573afeb67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb4bd0 0x5573afeb6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afeb7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afebf9d0 con 0x5573afebd7b0 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f62037fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb42b0 0x5573afeb67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f62027fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb4bd0 0x5573afeb6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f62037fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afeb7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f62027fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb4bd0 0x5573afeb6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39318/0 (socket says 172.21.15.8:39318) 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f62027fc640 1 -- 172.21.15.8:0/4020893502 learned_addr learned my addr 172.21.15.8:0/4020893502 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f62037fe640 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afeb7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49550/0 (socket says 172.21.15.8:49550) 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afeb77a0 con 0x5573afeb42b0 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.396+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573afeb7960 con 0x5573afeb4bd0 2024-02-19T16:23:47.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f620bf86740 10 client.0 _ll_register_callbacks cb 0x7f61f40d1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f62037fe640 1 -- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb4bd0 msgr2=0x5573afeb6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f62037fe640 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb4bd0 0x5573afeb6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f62037fe640 1 -- 172.21.15.8:0/4020893502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb42b0 msgr2=0x5573afeb67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f62037fe640 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb42b0 0x5573afeb67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f62037fe640 1 -- 172.21.15.8:0/4020893502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6204110de0 con 0x5573afebd7b0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f62037fe640 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afeb7260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61fc027920 tx=0x7f61fc003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9496e1302914bec server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f61e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f61e6ff5640 1 -- 172.21.15.8:0/4020893502 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0043d0 con 0x5573afebd7b0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f61e6ff5640 1 -- 172.21.15.8:0/4020893502 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f61fc004840 con 0x5573afebd7b0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61fc003e50 con 0x5573afebd7b0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f61e6ff5640 1 -- 172.21.15.8:0/4020893502 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61fc004a00 con 0x5573afebd7b0 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f620bf86740 10 client.15855 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:47.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.397+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5573afe1a610 con 0x5573afebd7b0 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f61e6ff5640 1 -- 172.21.15.8:0/4020893502 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61fc04f080 con 0x5573afebd7b0 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f61e6ff5640 1 -- 172.21.15.8:0/4020893502 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f61fc041070 con 0x5573afebd7b0 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 10 client.15855 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 20 client.15855 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573aff2a5f0 0x5573aff2c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 4 client.15855 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f61fc003e50 con 0x5573aff2a5f0 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f62027fc640 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573aff2a5f0 0x5573aff2c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f61e6ff5640 10 client.15855 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f62027fc640 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573aff2a5f0 0x5573aff2c9e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=811110210a31dc79 server_cookie=9fd789757fa74bf7 in_seq=0 out_seq=0 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f61e6ff5640 1 -- 172.21.15.8:0/4020893502 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f61fc003e50 con 0x5573aff2a5f0 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f61e6ff5640 10 client.15855 handle_command_reply: tid=0 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.398+0000 7f620bf86740 1 client.15855 shutdown 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 msgr2=0x5573afeb7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afeb7260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61fc027920 tx=0x7f61fc003d90 comp rx=0 tx=0).stop 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 shutdown_connections 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5573afebd7b0 0x5573afeb7260 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5573aff2a5f0 0x5573aff2c9e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573afeb4bd0 0x5573afeb6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 --2- 172.21.15.8:0/4020893502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573afeb42b0 0x5573afeb67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f61e6ff5640 0 client.15855 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:47.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 >> 172.21.15.8:0/4020893502 conn(0x5573afeae990 msgr2=0x5573afead1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 shutdown_connections 2024-02-19T16:23:47.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 1 -- 172.21.15.8:0/4020893502 wait complete. 2024-02-19T16:23:47.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f620bf86740 20 client.15855 trim_cache size 0 max 16384 2024-02-19T16:23:47.407 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:23:47.408 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:23:47.408 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:23:47.408 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:23:47.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61ec0a23d0 msgr2=0x7f61ec0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61ec0a23d0 0x7f61ec0a4890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cb760 tx=0x7f61f40d4060 comp rx=0 tx=0).stop 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 msgr2=0x7f620419f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 0x7f620419f940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f61f400d950 tx=0x7f61f400dd50 comp rx=0 tx=0).stop 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 shutdown_connections 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f620410de90 0x7f6204199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61ec0a23d0 0x7f61ec0a4890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f620406a480 0x7f620419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 --2- 172.21.15.8:0/1329136592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6204069b60 0x7f620419f940 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.399+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 >> 172.21.15.8:0/1329136592 conn(0x7f62040774e0 msgr2=0x7f620410f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.400+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 shutdown_connections 2024-02-19T16:23:47.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.400+0000 7f62092d4640 1 -- 172.21.15.8:0/1329136592 wait complete. 2024-02-19T16:23:47.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:23:47.416 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:23:47.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.564+0000 7f804962a640 1 -- 172.21.15.8:0/4093699884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044100240 msgr2=0x7f8044100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.564+0000 7f804962a640 1 --2- 172.21.15.8:0/4093699884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044100240 0x7f8044100620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8034009860 tx=0x7f803402bb70 comp rx=0 tx=0).stop 2024-02-19T16:23:47.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 -- 172.21.15.8:0/4093699884 shutdown_connections 2024-02-19T16:23:47.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 --2- 172.21.15.8:0/4093699884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8044107a10 0x7f804410e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 --2- 172.21.15.8:0/4093699884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8044100bf0 0x7f8044107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 --2- 172.21.15.8:0/4093699884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044100240 0x7f8044100620 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 -- 172.21.15.8:0/4093699884 >> 172.21.15.8:0/4093699884 conn(0x7f8044075710 msgr2=0x7f8044075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 -- 172.21.15.8:0/4093699884 shutdown_connections 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 -- 172.21.15.8:0/4093699884 wait complete. 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.565+0000 7f804962a640 1 Processor -- start 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 -- start start 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8044100240 0x7f804419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8044107a10 0x7f804419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 0x7f8044197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044198340 con 0x7f8044100240 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441984b0 con 0x7f8044107a10 2024-02-19T16:23:47.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f804962a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044198670 con 0x7f8044197950 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f8042ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8044100240 0x7f804419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 0x7f8044197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 0x7f8044197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39348/0 (socket says 172.21.15.8:39348) 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 -- 172.21.15.8:0/3334557168 learned_addr learned my addr 172.21.15.8:0/3334557168 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80427fc640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8044107a10 0x7f804419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 -- 172.21.15.8:0/3334557168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8044107a10 msgr2=0x7f804419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8044107a10 0x7f804419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 -- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8044100240 msgr2=0x7f804419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8044100240 0x7f804419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.566+0000 7f80437fe640 1 -- 172.21.15.8:0/3334557168 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009560 con 0x7f8044197950 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.567+0000 7f80437fe640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 0x7f8044197e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f803c00eb10 tx=0x7f803c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.567+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c004b20 con 0x7f8044197950 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.567+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f803c004c80 con 0x7f8044197950 2024-02-19T16:23:47.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.567+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c010650 con 0x7f8044197950 2024-02-19T16:23:47.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.567+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8034009490 con 0x7f8044197950 2024-02-19T16:23:47.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.567+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8044101410 con 0x7f8044197950 2024-02-19T16:23:47.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.568+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044067a50 con 0x7f8044197950 2024-02-19T16:23:47.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.569+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f803c005020 con 0x7f8044197950 2024-02-19T16:23:47.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.570+0000 7f802bfff640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f802c0a2350 0x7f802c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.570+0000 7f8042ffd640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f802c0a2350 0x7f802c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.570+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f803c014070 con 0x7f8044197950 2024-02-19T16:23:47.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.570+0000 7f8042ffd640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f802c0a2350 0x7f802c0a4810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8034009830 tx=0x7f80340056d0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.574+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f803c0972f0 con 0x7f8044197950 2024-02-19T16:23:47.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.798+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8044109a70 con 0x7f8044197950 2024-02-19T16:23:47.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.799+0000 7f802bfff640 1 -- 172.21.15.8:0/3334557168 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f8044109a70 con 0x7f8044197950 2024-02-19T16:23:47.800 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:23:47.800 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:22:36.842920+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:23:47.800 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2024-02-19T16:23:47.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f802c0a2350 msgr2=0x7f802c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f802c0a2350 0x7f802c0a4810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8034009830 tx=0x7f80340056d0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 msgr2=0x7f8044197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 0x7f8044197e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f803c00eb10 tx=0x7f803c00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 shutdown_connections 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f802c0a2350 0x7f802c0a4810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8044197950 0x7f8044197e00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8044107a10 0x7f804419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 --2- 172.21.15.8:0/3334557168 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8044100240 0x7f804419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 >> 172.21.15.8:0/3334557168 conn(0x7f8044075710 msgr2=0x7f80441055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 shutdown_connections 2024-02-19T16:23:47.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.801+0000 7f804962a640 1 -- 172.21.15.8:0/3334557168 wait complete. 2024-02-19T16:23:47.814 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:23:47.814 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.925+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1681781323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 msgr2=0x7fdf4810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.925+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1681781323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4810d950 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009900 tx=0x7fdf3c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.925+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1681781323 shutdown_connections 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.925+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1681781323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf4810de90 0x7fdf48110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.925+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1681781323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4810d950 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.925+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1681781323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf480686d0 0x7fdf48068ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.926+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1681781323 >> 172.21.15.8:0/1681781323 conn(0x7fdf480774f0 msgr2=0x7fdf48077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.926+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1681781323 shutdown_connections 2024-02-19T16:23:47.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.926+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1681781323 wait complete. 2024-02-19T16:23:47.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.926+0000 7fdf4d08f640 1 Processor -- start 2024-02-19T16:23:47.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 -- start start 2024-02-19T16:23:47.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf480686d0 0x7fdf48199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf4810de90 0x7fdf4819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4819ebf0 con 0x7fdf48068ff0 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4819ed60 con 0x7fdf4810de90 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4d08f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4819ef20 con 0x7fdf480686d0 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49580/0 (socket says 172.21.15.8:49580) 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4659c640 1 -- 172.21.15.8:0/1870353093 learned_addr learned my addr 172.21.15.8:0/1870353093 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf480686d0 0x7fdf48199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.927+0000 7fdf4759e640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf4810de90 0x7fdf4819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4759e640 1 -- 172.21.15.8:0/1870353093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf480686d0 msgr2=0x7fdf48199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4759e640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf480686d0 0x7fdf48199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4759e640 1 -- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 msgr2=0x7fdf4819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4759e640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4759e640 1 -- 172.21.15.8:0/1870353093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38009490 con 0x7fdf4810de90 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4759e640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf4810de90 0x7fdf4819e5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005fd0 tx=0x7fdf4000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf2ffff640 1 -- 172.21.15.8:0/1870353093 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000cdf0 con 0x7fdf4810de90 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf2ffff640 1 -- 172.21.15.8:0/1870353093 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf40005020 con 0x7fdf4810de90 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c009560 con 0x7fdf4810de90 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf2ffff640 1 -- 172.21.15.8:0/1870353093 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40010690 con 0x7fdf4810de90 2024-02-19T16:23:47.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.928+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf481a68a0 con 0x7fdf4810de90 2024-02-19T16:23:47.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.930+0000 7fdf2ffff640 1 -- 172.21.15.8:0/1870353093 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fdf40005190 con 0x7fdf4810de90 2024-02-19T16:23:47.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.931+0000 7fdf2ffff640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdf300a23d0 0x7fdf300a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.931+0000 7fdf2ffff640 1 -- 172.21.15.8:0/1870353093 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fdf40014070 con 0x7fdf4810de90 2024-02-19T16:23:47.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.931+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdf300a23d0 0x7fdf300a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.932+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdf300a23d0 0x7fdf300a4890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009cf0 tx=0x7fdf38005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.932+0000 7fdf4fc67740 1 Processor -- start 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.932+0000 7fdf4fc67740 1 -- start start 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.932+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 0x557f1584b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.932+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584bb70 0x557f15854210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f15856b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 0x557f1584b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 0x557f1584b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52412/0 (socket says 172.21.15.8:52412) 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584bb70 0x557f15854210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584bb70 0x557f15854210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39376/0 (socket says 172.21.15.8:39376) 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 -- 172.21.15.8:0/999628192 learned_addr learned my addr 172.21.15.8:0/999628192 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4759e640 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f15856b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f15806960 con 0x557f15854750 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f157ba310 con 0x557f1584b250 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f157b7840 con 0x557f1584bb70 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 -- 172.21.15.8:0/999628192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584bb70 msgr2=0x557f15854210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584bb70 0x557f15854210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 -- 172.21.15.8:0/999628192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 msgr2=0x557f15856b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f15856b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 -- 172.21.15.8:0/999628192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f15775060 con 0x557f1584b250 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf4759e640 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f15856b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.933+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 0x557f1584b630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf38012530 tx=0x7fdf38014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b7333f9a30b605 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf2d7fa640 1 -- 172.21.15.8:0/999628192 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38005220 con 0x557f1584b250 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf2d7fa640 1 -- 172.21.15.8:0/999628192 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf38016070 con 0x557f1584b250 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf2d7fa640 1 -- 172.21.15.8:0/999628192 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3801f7e0 con 0x557f1584b250 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 msgr2=0x557f1584b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 0x557f1584b630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf38012530 tx=0x7fdf38014000 comp rx=0 tx=0).stop 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 shutdown_connections 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f15856b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584bb70 0x557f15854210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/999628192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584b250 0x557f1584b630 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 >> 172.21.15.8:0/999628192 conn(0x557f15845980 msgr2=0x557f15845d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.934+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 shutdown_connections 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 -- 172.21.15.8:0/999628192 wait complete. 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 -- start start 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 Processor -- start 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f158517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 0x557f15851ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f15854750 0x557f15852220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f157b3020 con 0x557f1584bb70 2024-02-19T16:23:47.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f15486430 con 0x557f15854750 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f15442e00 con 0x557f1584b250 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4fc67740 10 client.0 _ll_register_callbacks cb 0x557f1575acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 0x557f15851ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4759e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f15854750 0x557f15852220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf46d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f158517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 0x557f15851ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49608/0 (socket says 172.21.15.8:49608) 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.935+0000 7fdf4659c640 1 -- 172.21.15.8:0/3345278317 learned_addr learned my addr 172.21.15.8:0/3345278317 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4659c640 1 -- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 msgr2=0x557f158517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4659c640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f158517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4659c640 1 -- 172.21.15.8:0/3345278317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f15854750 msgr2=0x557f15852220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4659c640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f15854750 0x557f15852220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4659c640 1 -- 172.21.15.8:0/3345278317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf38009de0 con 0x557f1584bb70 2024-02-19T16:23:47.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4759e640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f15854750 0x557f15852220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f158517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4659c640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 0x557f15851ce0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c036040 tx=0x7fdf3c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb33536b40b05a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf2aff5640 1 -- 172.21.15.8:0/3345278317 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c02cc00 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f154b27e0 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf2aff5640 1 -- 172.21.15.8:0/3345278317 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fdf3c004260 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf2aff5640 1 -- 172.21.15.8:0/3345278317 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf3c004420 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4fc67740 10 client.15873 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.936+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557f157b16a0 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf2aff5640 1 -- 172.21.15.8:0/3345278317 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557f154b27e0 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf2aff5640 1 -- 172.21.15.8:0/3345278317 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf3c03c040 con 0x557f1584bb70 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf4fc67740 10 client.15873 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:47.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf4fc67740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf4fc67740 20 client.15873 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f15864900 0x557f15866d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf4fc67740 4 client.15873 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f154ef720 con 0x557f15864900 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.937+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f15864900 0x557f15866d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.938+0000 7fdf2aff5640 10 client.15873 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.938+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f15864900 0x557f15866d20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1914d6f2ea3f902 server_cookie=a3949859d2ec8b5a in_seq=0 out_seq=0 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.938+0000 7fdf2aff5640 1 -- 172.21.15.8:0/3345278317 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557f154ef720 con 0x557f15864900 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.938+0000 7fdf2aff5640 10 client.15873 handle_command_reply: tid=0 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.938+0000 7fdf4fc67740 1 client.15873 shutdown 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 msgr2=0x557f15851ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 0x557f15851ce0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c036040 tx=0x7fdf3c02ca10 comp rx=0 tx=0).stop 2024-02-19T16:23:47.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 shutdown_connections 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f15864900 0x557f15866d20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f15854750 0x557f15852220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf2aff5640 0 client.15873 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584bb70 0x557f15851ce0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/3345278317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f158517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 >> 172.21.15.8:0/3345278317 conn(0x557f15845980 msgr2=0x557f158476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 shutdown_connections 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 1 -- 172.21.15.8:0/3345278317 wait complete. 2024-02-19T16:23:47.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.939+0000 7fdf4fc67740 20 client.15873 trim_cache size 0 max 16384 2024-02-19T16:23:47.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.951+0000 7fdf4fc67740 1 Processor -- start 2024-02-19T16:23:47.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.951+0000 7fdf4fc67740 1 -- start start 2024-02-19T16:23:47.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.951+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f15852040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.951+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1585fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f1584ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f15738130 con 0x557f15854750 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f15846980 con 0x557f1584bb70 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f158451f0 con 0x557f1584b250 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f1584ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1585fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f1584ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49616/0 (socket says 172.21.15.8:49616) 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4759e640 1 -- 172.21.15.8:0/2656834464 learned_addr learned my addr 172.21.15.8:0/2656834464 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f15852040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 -- 172.21.15.8:0/2656834464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 msgr2=0x557f15852040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f15852040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 -- 172.21.15.8:0/2656834464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 msgr2=0x557f1584ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f1584ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 -- 172.21.15.8:0/2656834464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1574dc20 con 0x557f1584bb70 2024-02-19T16:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4759e640 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f1584ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f15852040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:23:47.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf4659c640 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1585fd20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02c780 tx=0x7fdf3c004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f4f552745bdc49 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.952+0000 7fdf2cff9640 1 -- 172.21.15.8:0/2656834464 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c005d50 con 0x557f1584bb70 2024-02-19T16:23:47.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf2cff9640 1 -- 172.21.15.8:0/2656834464 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf3c003e70 con 0x557f1584bb70 2024-02-19T16:23:47.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf2cff9640 1 -- 172.21.15.8:0/2656834464 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0052d0 con 0x557f1584bb70 2024-02-19T16:23:47.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 -- 172.21.15.8:0/2656834464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 msgr2=0x557f1585fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1585fd20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02c780 tx=0x7fdf3c004a60 comp rx=0 tx=0).stop 2024-02-19T16:23:47.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 -- 172.21.15.8:0/2656834464 shutdown_connections 2024-02-19T16:23:47.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f15854750 0x557f1584ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1585fd20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/2656834464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f1584b250 0x557f15852040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 -- 172.21.15.8:0/2656834464 >> 172.21.15.8:0/2656834464 conn(0x557f15845980 msgr2=0x557f1585f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 -- 172.21.15.8:0/2656834464 shutdown_connections 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.953+0000 7fdf4fc67740 1 -- 172.21.15.8:0/2656834464 wait complete. 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 -- start start 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 Processor -- start 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 0x557f15869c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1586a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f15854750 0x557f158bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1586a910 con 0x557f1584b250 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1586aa80 con 0x557f1584bb70 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f158bf600 con 0x557f15854750 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4fc67740 10 client.0 _ll_register_callbacks cb 0x7fdf3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:47.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 0x557f15869c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 0x557f15869c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49632/0 (socket says 172.21.15.8:49632) 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 -- 172.21.15.8:0/1981638737 learned_addr learned my addr 172.21.15.8:0/1981638737 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4759e640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f15854750 0x557f158bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1586a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 -- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f15854750 msgr2=0x557f158bf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f15854750 0x557f158bf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 -- 172.21.15.8:0/1981638737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 msgr2=0x557f1586a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1586a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf46d9d640 1 -- 172.21.15.8:0/1981638737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c009560 con 0x557f1584b250 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.954+0000 7fdf4759e640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f15854750 0x557f158bf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf4659c640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1586a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf46d9d640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 0x557f15869c30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdf38012010 tx=0x7fdf380096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d6be50b379e240 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf2aff5640 1 -- 172.21.15.8:0/1981638737 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3802d040 con 0x557f1584b250 2024-02-19T16:23:47.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf2aff5640 1 -- 172.21.15.8:0/1981638737 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fdf38018070 con 0x557f1584b250 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf38016070 con 0x557f1584b250 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf2aff5640 1 -- 172.21.15.8:0/1981638737 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf38035250 con 0x557f1584b250 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf4fc67740 10 client.15879 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557f157b16a0 con 0x557f1584b250 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.955+0000 7fdf2aff5640 1 -- 172.21.15.8:0/1981638737 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf3802e040 con 0x557f1584b250 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf2aff5640 1 -- 172.21.15.8:0/1981638737 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf38025030 con 0x557f1584b250 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4fc67740 10 client.15879 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4fc67740 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4fc67740 20 client.15879 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:47.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f158c0df0 0x557f158c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4fc67740 4 client.15879 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fdf38016070 con 0x557f158c0df0 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4659c640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f158c0df0 0x557f158c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf2aff5640 10 client.15879 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.956+0000 7fdf4659c640 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f158c0df0 0x557f158c3210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa0e59cb201ce9f5 server_cookie=216c305306011693 in_seq=0 out_seq=0 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf2aff5640 1 -- 172.21.15.8:0/1981638737 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fdf38016070 con 0x557f158c0df0 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf2aff5640 10 client.15879 handle_command_reply: tid=0 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 client.15879 shutdown 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 msgr2=0x557f15869c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 0x557f15869c30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdf38012010 tx=0x7fdf380096c0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 shutdown_connections 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x557f158c0df0 0x557f158c3210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f15854750 0x557f158bf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf2aff5640 0 client.15879 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f1584bb70 0x557f1586a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 --2- 172.21.15.8:0/1981638737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f1584b250 0x557f15869c30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 >> 172.21.15.8:0/1981638737 conn(0x557f15845980 msgr2=0x557f1585fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 shutdown_connections 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 1 -- 172.21.15.8:0/1981638737 wait complete. 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.957+0000 7fdf4fc67740 20 client.15879 trim_cache size 0 max 16384 2024-02-19T16:23:47.964 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:23:47.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdf300a23d0 msgr2=0x7fdf300a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdf300a23d0 0x7fdf300a4890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009cf0 tx=0x7fdf38005f90 comp rx=0 tx=0).stop 2024-02-19T16:23:47.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf4810de90 msgr2=0x7fdf4819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:47.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf4810de90 0x7fdf4819e5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005fd0 tx=0x7fdf4000ef10 comp rx=0 tx=0).stop 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 shutdown_connections 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf4810de90 0x7fdf4819e5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf48068ff0 0x7fdf4819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdf300a23d0 0x7fdf300a4890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 --2- 172.21.15.8:0/1870353093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf480686d0 0x7fdf48199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 >> 172.21.15.8:0/1870353093 conn(0x7fdf480774f0 msgr2=0x7fdf4810f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 shutdown_connections 2024-02-19T16:23:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:47.958+0000 7fdf4d08f640 1 -- 172.21.15.8:0/1870353093 wait complete. 2024-02-19T16:23:47.970 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:23:47.971 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 -- 172.21.15.8:0/253044581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100450 msgr2=0x7f6c481008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/253044581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100450 0x7f6c481008d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009900 tx=0x7f6c3c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 -- 172.21.15.8:0/253044581 shutdown_connections 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/253044581 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c48100e10 0x7f6c4810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/253044581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100450 0x7f6c481008d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/253044581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c48106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 -- 172.21.15.8:0/253044581 >> 172.21.15.8:0/253044581 conn(0x7f6c48077550 msgr2=0x7f6c48077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 -- 172.21.15.8:0/253044581 shutdown_connections 2024-02-19T16:23:48.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.079+0000 7f6c4ca76640 1 -- 172.21.15.8:0/253044581 wait complete. 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 Processor -- start 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 -- start start 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c48100450 0x7f6c48199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100e10 0x7f6c4819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c4819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c48100450 0x7f6c48199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c46d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c4819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c46d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c4819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39430/0 (socket says 172.21.15.8:39430) 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100e10 0x7f6c4819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c46d9d640 1 -- 172.21.15.8:0/2959630991 learned_addr learned my addr 172.21.15.8:0/2959630991 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4819ec40 con 0x7f6c48100450 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4819edb0 con 0x7f6c48100e10 2024-02-19T16:23:48.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.080+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4819ef70 con 0x7f6c481064a0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c46d9d640 1 -- 172.21.15.8:0/2959630991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100e10 msgr2=0x7f6c4819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c46d9d640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100e10 0x7f6c4819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c46d9d640 1 -- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c48100450 msgr2=0x7f6c48199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c46d9d640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c48100450 0x7f6c48199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c46d9d640 1 -- 172.21.15.8:0/2959630991 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c40009840 con 0x7f6c481064a0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c46d9d640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c4819e640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000efd0 tx=0x7f6c4000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c377fe640 1 -- 172.21.15.8:0/2959630991 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40009b40 con 0x7f6c481064a0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c377fe640 1 -- 172.21.15.8:0/2959630991 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c40003b70 con 0x7f6c481064a0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c377fe640 1 -- 172.21.15.8:0/2959630991 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400108f0 con 0x7f6c481064a0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c38009490 con 0x7f6c481064a0 2024-02-19T16:23:48.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.081+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c009560 con 0x7f6c481064a0 2024-02-19T16:23:48.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.084+0000 7f6c4f653740 1 Processor -- start 2024-02-19T16:23:48.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.084+0000 7f6c377fe640 1 -- 172.21.15.8:0/2959630991 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f6c40003ce0 con 0x7f6c481064a0 2024-02-19T16:23:48.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.084+0000 7f6c4f653740 1 -- start start 2024-02-19T16:23:48.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 0x55f1adbd3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3b70 0x55f1adbdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbdc750 0x55f1adbdeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4f653740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adb8e960 con 0x55f1adbd3b70 2024-02-19T16:23:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4f653740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adb42310 con 0x55f1adbdc750 2024-02-19T16:23:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4f653740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adb3f840 con 0x55f1adbd3250 2024-02-19T16:23:48.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 0x55f1adbd3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 0x55f1adbd3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39444/0 (socket says 172.21.15.8:39444) 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c45d9b640 1 -- 172.21.15.8:0/1724413578 learned_addr learned my addr 172.21.15.8:0/1724413578 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4659c640 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3b70 0x55f1adbdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbdc750 0x55f1adbdeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c377fe640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c300a2350 0x7f6c300a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4659c640 1 -- 172.21.15.8:0/1724413578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 msgr2=0x55f1adbd3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4659c640 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 0x55f1adbd3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4659c640 1 -- 172.21.15.8:0/1724413578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbdc750 msgr2=0x55f1adbdeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4659c640 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbdc750 0x55f1adbdeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c46d9d640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c300a2350 0x7f6c300a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c4659c640 1 -- 172.21.15.8:0/1724413578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1adafd060 con 0x55f1adbd3b70 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 0x55f1adbd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.085+0000 7f6c377fe640 1 -- 172.21.15.8:0/2959630991 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6c40014070 con 0x7f6c481064a0 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.086+0000 7f6c4659c640 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3b70 0x55f1adbdc210 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009db0 tx=0x7f6c38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8659e40ad656d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.086+0000 7f6c34ff9640 1 -- 172.21.15.8:0/1724413578 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38004230 con 0x55f1adbd3b70 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.086+0000 7f6c46d9d640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c300a2350 0x7f6c300a4810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cbf60 tx=0x7f6c400907a0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.086+0000 7f6c34ff9640 1 -- 172.21.15.8:0/1724413578 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c38004390 con 0x55f1adbd3b70 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.086+0000 7f6c34ff9640 1 -- 172.21.15.8:0/1724413578 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380197e0 con 0x55f1adbd3b70 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 -- 172.21.15.8:0/1724413578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3b70 msgr2=0x55f1adbdc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3b70 0x55f1adbdc210 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009db0 tx=0x7f6c38011810 comp rx=0 tx=0).stop 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 -- 172.21.15.8:0/1724413578 shutdown_connections 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbdc750 0x55f1adbdeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3b70 0x55f1adbdc210 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1724413578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3250 0x55f1adbd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 -- 172.21.15.8:0/1724413578 >> 172.21.15.8:0/1724413578 conn(0x55f1adbcd980 msgr2=0x55f1adbcdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 -- 172.21.15.8:0/1724413578 shutdown_connections 2024-02-19T16:23:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 -- 172.21.15.8:0/1724413578 wait complete. 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 -- start start 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 Processor -- start 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adbd97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.087+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adbd9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbda230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4f653740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adb3b020 con 0x55f1adbd3250 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4f653740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ad80e430 con 0x55f1adbd3b70 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4f653740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1ad7cae00 con 0x55f1adbdc750 2024-02-19T16:23:48.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4f653740 10 client.0 _ll_register_callbacks cb 0x55f1adae2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adbd97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adbd97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49660/0 (socket says 172.21.15.8:49660) 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 -- 172.21.15.8:0/4011714694 learned_addr learned my addr 172.21.15.8:0/4011714694 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4659c640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adbd9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbda230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 -- 172.21.15.8:0/4011714694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 msgr2=0x55f1adbda230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbda230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 -- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 msgr2=0x55f1adbd9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adbd9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 -- 172.21.15.8:0/4011714694 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380099e0 con 0x55f1adbd3250 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c4659c640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adbd9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adbd97b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006010 tx=0x7f6c3c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2658482247070 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.088+0000 7f6c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c2dffb640 1 -- 172.21.15.8:0/4011714694 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004250 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c2dffb640 1 -- 172.21.15.8:0/4011714694 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6c3c0043b0 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1ad83a7e0 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c2dffb640 1 -- 172.21.15.8:0/4011714694 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c3c004df0 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 10 client.15891 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f1adb396a0 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c2dffb640 1 -- 172.21.15.8:0/4011714694 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f1ad83a7e0 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c2dffb640 1 -- 172.21.15.8:0/4011714694 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c3c0039c0 con 0x55f1adbd3250 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 10 client.15891 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 20 client.15891 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adbeefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 4 client.15891 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.089+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1ad877720 con 0x55f1adbecba0 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.090+0000 7f6c4659c640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adbeefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.090+0000 7f6c2dffb640 10 client.15891 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.090+0000 7f6c4659c640 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adbeefc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f93d711c7cd957 server_cookie=3d8a5ace1676ad9e in_seq=0 out_seq=0 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c2dffb640 1 -- 172.21.15.8:0/4011714694 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f1ad877720 con 0x55f1adbecba0 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c2dffb640 10 client.15891 handle_command_reply: tid=0 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 client.15891 shutdown 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 msgr2=0x55f1adbd97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adbd97b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006010 tx=0x7f6c3c009260 comp rx=0 tx=0).stop 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 shutdown_connections 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adbeefc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c2dffb640 0 client.15891 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbda230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adbd9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 --2- 172.21.15.8:0/4011714694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adbd97b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 >> 172.21.15.8:0/4011714694 conn(0x55f1adbcd980 msgr2=0x55f1adbcf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 shutdown_connections 2024-02-19T16:23:48.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.091+0000 7f6c4f653740 1 -- 172.21.15.8:0/4011714694 wait complete. 2024-02-19T16:23:48.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.092+0000 7f6c4f653740 20 client.15891 trim_cache size 0 max 16384 2024-02-19T16:23:48.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 Processor -- start 2024-02-19T16:23:48.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 -- start start 2024-02-19T16:23:48.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 0x55f1adbdbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 0x55f1adbe7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbdc750 0x55f1adbe8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adac0130 con 0x55f1adbdc750 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adbe96c0 con 0x55f1adbd3250 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4f653740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adbd45f0 con 0x55f1adbd3b70 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 0x55f1adbe7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 0x55f1adbdbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 0x55f1adbe7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39462/0 (socket says 172.21.15.8:39462) 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 0x55f1adbdbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52502/0 (socket says 172.21.15.8:52502) 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c4659c640 1 -- 172.21.15.8:0/1390717215 learned_addr learned my addr 172.21.15.8:0/1390717215 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.104+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbdc750 0x55f1adbe8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c45d9b640 1 -- 172.21.15.8:0/1390717215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 msgr2=0x55f1adbe7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 0x55f1adbe7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c45d9b640 1 -- 172.21.15.8:0/1390717215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbdc750 msgr2=0x55f1adbe8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbdc750 0x55f1adbe8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c45d9b640 1 -- 172.21.15.8:0/1390717215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1adad5c20 con 0x55f1adbd3250 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c4659c640 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 0x55f1adbe7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 0x55f1adbdbee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c003dd0 tx=0x7f6c3c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2a01ce2d150703 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c2ffff640 1 -- 172.21.15.8:0/1390717215 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004250 con 0x55f1adbd3250 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c2ffff640 1 -- 172.21.15.8:0/1390717215 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c049070 con 0x55f1adbd3250 2024-02-19T16:23:48.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.105+0000 7f6c2ffff640 1 -- 172.21.15.8:0/1390717215 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0052f0 con 0x55f1adbd3250 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 -- 172.21.15.8:0/1390717215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 msgr2=0x55f1adbdbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 0x55f1adbdbee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c003dd0 tx=0x7f6c3c005e40 comp rx=0 tx=0).stop 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 -- 172.21.15.8:0/1390717215 shutdown_connections 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbdc750 0x55f1adbe8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbd3b70 0x55f1adbe7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 --2- 172.21.15.8:0/1390717215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3250 0x55f1adbdbee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 -- 172.21.15.8:0/1390717215 >> 172.21.15.8:0/1390717215 conn(0x55f1adbcd980 msgr2=0x55f1adbccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 -- 172.21.15.8:0/1390717215 shutdown_connections 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 -- 172.21.15.8:0/1390717215 wait complete. 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 -- start start 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.106+0000 7f6c4f653740 1 Processor -- start 2024-02-19T16:23:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adc40710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adc40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbe0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adbe0920 con 0x55f1adbd3250 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adbe0a90 con 0x55f1adbd3b70 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1adbe0c50 con 0x55f1adbdc750 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adc40710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 10 client.0 _ll_register_callbacks cb 0x7f6c3c004790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c45d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adc40710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49674/0 (socket says 172.21.15.8:49674) 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c45d9b640 1 -- 172.21.15.8:0/3999023981 learned_addr learned my addr 172.21.15.8:0/3999023981 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adc40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbe0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 -- 172.21.15.8:0/3999023981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 msgr2=0x55f1adbe0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbe0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 -- 172.21.15.8:0/3999023981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 msgr2=0x55f1adc40710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adc40710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 -- 172.21.15.8:0/3999023981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1adafd060 con 0x55f1adbd3b70 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4659c640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adc40c50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c38014040 tx=0x7f6c380193e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b96a05e4d9c7dd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c2dffb640 1 -- 172.21.15.8:0/3999023981 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38005040 con 0x55f1adbd3b70 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c38004390 con 0x55f1adbd3b70 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.107+0000 7f6c2dffb640 1 -- 172.21.15.8:0/3999023981 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6c380051a0 con 0x55f1adbd3b70 2024-02-19T16:23:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c2dffb640 1 -- 172.21.15.8:0/3999023981 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c38005360 con 0x55f1adbd3b70 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 10 client.25667 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f1adb396a0 con 0x55f1adbd3b70 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c2dffb640 1 -- 172.21.15.8:0/3999023981 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c38004390 con 0x55f1adbd3b70 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c2dffb640 1 -- 172.21.15.8:0/3999023981 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c38003660 con 0x55f1adbd3b70 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 10 client.25667 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 10 client.25667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 20 client.25667 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adc4b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 4 client.25667 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1ad8c6890 con 0x55f1adbecba0 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.108+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adc4b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c45d9b640 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adc4b780 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8899aea821035e3 server_cookie=c81ef6e1314bddcd in_seq=0 out_seq=0 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c2dffb640 10 client.25667 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c2dffb640 1 -- 172.21.15.8:0/3999023981 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1ad8c6890 con 0x55f1adbecba0 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c2dffb640 10 client.25667 handle_command_reply: tid=0 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 client.25667 shutdown 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 msgr2=0x55f1adc40c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adc40c50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c38014040 tx=0x7f6c380193e0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 shutdown_connections 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55f1adbecba0 0x55f1adc4b780 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c2dffb640 0 client.25667 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:23:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1adbdc750 0x55f1adbe0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1adbd3b70 0x55f1adc40c50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 --2- 172.21.15.8:0/3999023981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1adbd3250 0x55f1adc40710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 >> 172.21.15.8:0/3999023981 conn(0x55f1adbcd980 msgr2=0x55f1adbd96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 shutdown_connections 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 1 -- 172.21.15.8:0/3999023981 wait complete. 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.109+0000 7f6c4f653740 20 client.25667 trim_cache size 0 max 16384 2024-02-19T16:23:48.118 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:23:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c300a2350 msgr2=0x7f6c300a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c300a2350 0x7f6c300a4810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cbf60 tx=0x7f6c400907a0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 msgr2=0x7f6c4819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c4819e640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000efd0 tx=0x7f6c4000c6a0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 shutdown_connections 2024-02-19T16:23:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c300a2350 0x7f6c300a4810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c481064a0 0x7f6c4819e640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c48100e10 0x7f6c4819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 --2- 172.21.15.8:0/2959630991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c48100450 0x7f6c48199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 >> 172.21.15.8:0/2959630991 conn(0x7f6c48077550 msgr2=0x7f6c4810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 shutdown_connections 2024-02-19T16:23:48.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.110+0000 7f6c4ca76640 1 -- 172.21.15.8:0/2959630991 wait complete. 2024-02-19T16:23:48.121 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:48 smithi008 ceph-mon[32573]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:48.121 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:48 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3334557168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:23:48.122 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:23:48.122 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:23:48.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 -- 172.21.15.8:0/1128364225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1056a0 msgr2=0x7fa91c10d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 --2- 172.21.15.8:0/1128364225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1056a0 0x7fa91c10d270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa914009cd0 tx=0x7fa91402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 -- 172.21.15.8:0/1128364225 shutdown_connections 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 --2- 172.21.15.8:0/1128364225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1056a0 0x7fa91c10d270 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 --2- 172.21.15.8:0/1128364225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c100f80 0x7fa91c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 --2- 172.21.15.8:0/1128364225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 0x7fa91c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.229+0000 7fa923607640 1 -- 172.21.15.8:0/1128364225 >> 172.21.15.8:0/1128364225 conn(0x7fa91c0774f0 msgr2=0x7fa91c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 -- 172.21.15.8:0/1128364225 shutdown_connections 2024-02-19T16:23:48.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 -- 172.21.15.8:0/1128364225 wait complete. 2024-02-19T16:23:48.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 Processor -- start 2024-02-19T16:23:48.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 -- start start 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 0x7fa91c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c1056a0 0x7fa91c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 0x7fa91c197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1982d0 con 0x7fa91c1978e0 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c198440 con 0x7fa91c1005d0 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.230+0000 7fa923607640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c198600 con 0x7fa91c1056a0 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 0x7fa91c197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 0x7fa91c197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49686/0 (socket says 172.21.15.8:49686) 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 -- 172.21.15.8:0/538679179 learned_addr learned my addr 172.21.15.8:0/538679179 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa920ba2640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c1056a0 0x7fa91c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa9213a3640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 0x7fa91c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 -- 172.21.15.8:0/538679179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c1056a0 msgr2=0x7fa91c19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c1056a0 0x7fa91c19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 -- 172.21.15.8:0/538679179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 msgr2=0x7fa91c19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 0x7fa91c19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 -- 172.21.15.8:0/538679179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914009900 con 0x7fa91c1978e0 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa9213a3640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 0x7fa91c19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa920ba2640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c1056a0 0x7fa91c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa921ba4640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 0x7fa91c197d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa914004af0 tx=0x7fa914034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa9127fc640 1 -- 172.21.15.8:0/538679179 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140328c0 con 0x7fa91c1978e0 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.231+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c1a45f0 con 0x7fa91c1978e0 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.232+0000 7fa9127fc640 1 -- 172.21.15.8:0/538679179 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa914003e50 con 0x7fa91c1978e0 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.232+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1a4aa0 con 0x7fa91c1978e0 2024-02-19T16:23:48.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.232+0000 7fa9127fc640 1 -- 172.21.15.8:0/538679179 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914004860 con 0x7fa91c1978e0 2024-02-19T16:23:48.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.234+0000 7fa9261e9740 1 Processor -- start 2024-02-19T16:23:48.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.235+0000 7fa9127fc640 1 -- 172.21.15.8:0/538679179 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fa91400a040 con 0x7fa91c1978e0 2024-02-19T16:23:48.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.235+0000 7fa9261e9740 1 -- start start 2024-02-19T16:23:48.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.235+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c70630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.235+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c79750 0x560292c7bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9261e9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292c2b960 con 0x560292c70250 2024-02-19T16:23:48.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9261e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292bdf310 con 0x560292c70b70 2024-02-19T16:23:48.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9261e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292bdc840 con 0x560292c79750 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa920ba2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c70630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa920ba2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c70630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49700/0 (socket says 172.21.15.8:49700) 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa920ba2640 1 -- 172.21.15.8:0/3098521308 learned_addr learned my addr 172.21.15.8:0/3098521308 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa920ba2640 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c79750 0x560292c7bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 -- 172.21.15.8:0/3098521308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c79750 msgr2=0x560292c7bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c79750 0x560292c7bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 -- 172.21.15.8:0/3098521308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 msgr2=0x560292c70630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c70630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 -- 172.21.15.8:0/3098521308 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560292b9a060 con 0x560292c70b70 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa920ba2640 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c70630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9127fc640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa9080a2350 0x7fa9080a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9213a3640 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c79210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa918005d00 tx=0x7fa91800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97010c4d2fc214a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa9127fc640 1 -- 172.21.15.8:0/538679179 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa9140f1210 con 0x7fa91c1978e0 2024-02-19T16:23:48.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.236+0000 7fa920ba2640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa9080a2350 0x7fa9080a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.237+0000 7fa9077fe640 1 -- 172.21.15.8:0/3098521308 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918004220 con 0x560292c70b70 2024-02-19T16:23:48.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.237+0000 7fa9077fe640 1 -- 172.21.15.8:0/3098521308 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa918015070 con 0x560292c70b70 2024-02-19T16:23:48.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.237+0000 7fa9077fe640 1 -- 172.21.15.8:0/3098521308 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918011900 con 0x560292c70b70 2024-02-19T16:23:48.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.237+0000 7fa9261e9740 1 -- 172.21.15.8:0/3098521308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 msgr2=0x560292c79210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.237+0000 7fa920ba2640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa9080a2350 0x7fa9080a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x560292c6bb10 tx=0x7fa90c015040 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.237+0000 7fa9261e9740 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c79210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa918005d00 tx=0x7fa91800bf90 comp rx=0 tx=0).stop 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 -- 172.21.15.8:0/3098521308 shutdown_connections 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c79750 0x560292c7bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c79210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 --2- 172.21.15.8:0/3098521308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c70630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 -- 172.21.15.8:0/3098521308 >> 172.21.15.8:0/3098521308 conn(0x560292c6a980 msgr2=0x560292c6ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 -- 172.21.15.8:0/3098521308 shutdown_connections 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 -- 172.21.15.8:0/3098521308 wait complete. 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 -- start start 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 Processor -- start 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.238+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c76730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70b70 0x560292c76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9261e9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292bd8020 con 0x560292c70250 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c76730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9261e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602928ab430 con 0x560292c79750 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9261e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292867e00 con 0x560292c70b70 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9261e9740 10 client.0 _ll_register_callbacks cb 0x560292b7fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c76730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49704/0 (socket says 172.21.15.8:49704) 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 -- 172.21.15.8:0/4231140941 learned_addr learned my addr 172.21.15.8:0/4231140941 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa920ba2640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70b70 0x560292c76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa921ba4640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 -- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70b70 msgr2=0x560292c76c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70b70 0x560292c76c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 -- 172.21.15.8:0/4231140941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 msgr2=0x560292c771b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c771b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 -- 172.21.15.8:0/4231140941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91800bc70 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa921ba4640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c771b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa920ba2640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70b70 0x560292c76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa9213a3640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c76730 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa918004240 tx=0x7fa918004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df28ae67d3051775 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa904ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.239+0000 7fa904ff9640 1 -- 172.21.15.8:0/4231140941 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91801f070 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602928d77e0 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa904ff9640 1 -- 172.21.15.8:0/4231140941 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa918017070 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa904ff9640 1 -- 172.21.15.8:0/4231140941 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa918004490 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 10 client.15909 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560292bd66a0 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa904ff9640 1 -- 172.21.15.8:0/4231140941 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5602928d77e0 con 0x560292c70250 2024-02-19T16:23:48.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa904ff9640 1 -- 172.21.15.8:0/4231140941 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa918004b40 con 0x560292c70250 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 10 client.15909 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 20 client.15909 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292c8be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 4 client.15909 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560292914720 con 0x560292c899f0 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.240+0000 7fa920ba2640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292c8be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.241+0000 7fa904ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:48.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.241+0000 7fa920ba2640 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292c8be30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67002066d69f8306 server_cookie=ac56d8a6c1fa9198 in_seq=0 out_seq=0 2024-02-19T16:23:48.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa904ff9640 1 -- 172.21.15.8:0/4231140941 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560292914720 con 0x560292c899f0 2024-02-19T16:23:48.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa904ff9640 10 client.15909 handle_command_reply: tid=0 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 client.15909 shutdown 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 msgr2=0x560292c76730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c76730 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa918004240 tx=0x7fa918004270 comp rx=0 tx=0).stop 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 shutdown_connections 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c771b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292c8be30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70b70 0x560292c76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa904ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4231140941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70250 0x560292c76730 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 >> 172.21.15.8:0/4231140941 conn(0x560292c6a980 msgr2=0x560292c6c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 shutdown_connections 2024-02-19T16:23:48.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 1 -- 172.21.15.8:0/4231140941 wait complete. 2024-02-19T16:23:48.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.242+0000 7fa9261e9740 20 client.15909 trim_cache size 0 max 16384 2024-02-19T16:23:48.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 Processor -- start 2024-02-19T16:23:48.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 -- start start 2024-02-19T16:23:48.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c78df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70b70 0x560292c70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c69a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292b5d130 con 0x560292c70b70 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292c714c0 con 0x560292c79750 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9261e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292c66610 con 0x560292c70250 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c78df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa920ba2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70b70 0x560292c70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c78df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39522/0 (socket says 172.21.15.8:39522) 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa921ba4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c69a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.255+0000 7fa9213a3640 1 -- 172.21.15.8:0/4052765341 learned_addr learned my addr 172.21.15.8:0/4052765341 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa9213a3640 1 -- 172.21.15.8:0/4052765341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 msgr2=0x560292c69a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa9213a3640 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c69a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa9213a3640 1 -- 172.21.15.8:0/4052765341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70b70 msgr2=0x560292c70f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa9213a3640 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70b70 0x560292c70f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa9213a3640 1 -- 172.21.15.8:0/4052765341 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560292b72c20 con 0x560292c70250 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa921ba4640 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c69a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa9213a3640 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c78df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9180118c0 tx=0x7fa9180118f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=957e8a877779c68e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa906ffd640 1 -- 172.21.15.8:0/4052765341 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91801f070 con 0x560292c70250 2024-02-19T16:23:48.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa906ffd640 1 -- 172.21.15.8:0/4052765341 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa918020040 con 0x560292c70250 2024-02-19T16:23:48.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.256+0000 7fa906ffd640 1 -- 172.21.15.8:0/4052765341 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918024eb0 con 0x560292c70250 2024-02-19T16:23:48.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 -- 172.21.15.8:0/4052765341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 msgr2=0x560292c78df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c78df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9180118c0 tx=0x7fa9180118f0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 -- 172.21.15.8:0/4052765341 shutdown_connections 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c79750 0x560292c69a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c70b70 0x560292c70f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 --2- 172.21.15.8:0/4052765341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c78df0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.257+0000 7fa9261e9740 1 -- 172.21.15.8:0/4052765341 >> 172.21.15.8:0/4052765341 conn(0x560292c6a980 msgr2=0x560292c692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 -- 172.21.15.8:0/4052765341 shutdown_connections 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 -- 172.21.15.8:0/4052765341 wait complete. 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 -- start start 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 Processor -- start 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c79750 0x560292ce4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292c740a0 con 0x560292c79750 2024-02-19T16:23:48.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292ce4840 con 0x560292c70b70 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560292ce49b0 con 0x560292c70250 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9261e9740 10 client.0 _ll_register_callbacks cb 0x7fa90c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.258+0000 7fa920ba2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa9213a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c73460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39526/0 (socket says 172.21.15.8:39526) 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa9213a3640 1 -- 172.21.15.8:0/2528626672 learned_addr learned my addr 172.21.15.8:0/2528626672 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa921ba4640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c79750 0x560292ce4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa920ba2640 1 -- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 msgr2=0x560292c73460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa920ba2640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c73460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa920ba2640 1 -- 172.21.15.8:0/2528626672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c79750 msgr2=0x560292ce4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa920ba2640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c79750 0x560292ce4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa920ba2640 1 -- 172.21.15.8:0/2528626672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1a4aa0 con 0x560292c70b70 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa9213a3640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c73460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:48.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa920ba2640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c739a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x560292c8c6b0 tx=0x7fa90c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d7d18a569f8a4a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa904ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa904ff9640 1 -- 172.21.15.8:0/2528626672 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c03a040 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560292c660f0 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa904ff9640 1 -- 172.21.15.8:0/2528626672 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa90c0047a0 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa904ff9640 1 -- 172.21.15.8:0/2528626672 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa90c010e80 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa9261e9740 10 client.25685 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.259+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560292bd66a0 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa904ff9640 1 -- 172.21.15.8:0/2528626672 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560292c660f0 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa904ff9640 1 -- 172.21.15.8:0/2528626672 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa90c02f040 con 0x560292c70b70 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9261e9740 10 client.25685 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9261e9740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9261e9740 20 client.25685 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9261e9740 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292ce8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9261e9740 4 client.25685 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:23:48.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa918015070 con 0x560292c899f0 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa9213a3640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292ce8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.260+0000 7fa904ff9640 10 client.25685 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9213a3640 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292ce8270 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3fde0d39182b91 server_cookie=34a25e9e4d5b705 in_seq=0 out_seq=0 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa904ff9640 1 -- 172.21.15.8:0/2528626672 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa918015070 con 0x560292c899f0 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa904ff9640 10 client.25685 handle_command_reply: tid=0 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 client.25685 shutdown 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 msgr2=0x560292c739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c739a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x560292c8c6b0 tx=0x7fa90c009c20 comp rx=0 tx=0).stop 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 shutdown_connections 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560292c79750 0x560292ce4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560292c70b70 0x560292c739a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560292c899f0 0x560292ce8270 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 --2- 172.21.15.8:0/2528626672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560292c70250 0x560292c73460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa904ff9640 0 client.25685 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:48.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 >> 172.21.15.8:0/2528626672 conn(0x560292c6a980 msgr2=0x560292c69b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 shutdown_connections 2024-02-19T16:23:48.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 1 -- 172.21.15.8:0/2528626672 wait complete. 2024-02-19T16:23:48.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.261+0000 7fa9261e9740 20 client.25685 trim_cache size 0 max 16384 2024-02-19T16:23:48.268 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa9080a2350 msgr2=0x7fa9080a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa9080a2350 0x7fa9080a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x560292c6bb10 tx=0x7fa90c015040 comp rx=0 tx=0).stop 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 msgr2=0x7fa91c197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 0x7fa91c197d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa914004af0 tx=0x7fa914034cc0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 shutdown_connections 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa91c1978e0 0x7fa91c197d90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa9080a2350 0x7fa9080a4810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91c1056a0 0x7fa91c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 --2- 172.21.15.8:0/538679179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa91c1005d0 0x7fa91c19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:48.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 >> 172.21.15.8:0/538679179 conn(0x7fa91c0774f0 msgr2=0x7fa91c068930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:48.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 shutdown_connections 2024-02-19T16:23:48.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:48.262+0000 7fa923607640 1 -- 172.21.15.8:0/538679179 wait complete. 2024-02-19T16:23:48.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:48 smithi191 ceph-mon[37203]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:48.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:48 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3334557168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:23:48.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:48 smithi156 ceph-mon[36502]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:48.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:48 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3334557168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:23:49.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:23:49.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:23:49.275 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:23:49.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 -- 172.21.15.8:0/1483556181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 msgr2=0x7fb730068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 --2- 172.21.15.8:0/1483556181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb730068b70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009930 tx=0x7fb71c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:23:49.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 -- 172.21.15.8:0/1483556181 shutdown_connections 2024-02-19T16:23:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 --2- 172.21.15.8:0/1483556181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 0x7fb73010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 --2- 172.21.15.8:0/1483556181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb730068b70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 --2- 172.21.15.8:0/1483556181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7301074f0 0x7fb7301078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 -- 172.21.15.8:0/1483556181 >> 172.21.15.8:0/1483556181 conn(0x7fb7300774f0 msgr2=0x7fb730077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 -- 172.21.15.8:0/1483556181 shutdown_connections 2024-02-19T16:23:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.393+0000 7fb736f3e640 1 -- 172.21.15.8:0/1483556181 wait complete. 2024-02-19T16:23:49.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 Processor -- start 2024-02-19T16:23:49.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 -- start start 2024-02-19T16:23:49.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb7301003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 0x7fb7301008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7301074f0 0x7fb730100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301a5c40 con 0x7fb7300686f0 2024-02-19T16:23:49.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301a5db0 con 0x7fb7301074f0 2024-02-19T16:23:49.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb736f3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301a5f70 con 0x7fb7300690b0 2024-02-19T16:23:49.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb734cda640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb7301003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb734cda640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb7301003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49732/0 (socket says 172.21.15.8:49732) 2024-02-19T16:23:49.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb734cda640 1 -- 172.21.15.8:0/3972202181 learned_addr learned my addr 172.21.15.8:0/3972202181 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 0x7fb7301008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb7354db640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7301074f0 0x7fb730100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb72ffff640 1 -- 172.21.15.8:0/3972202181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7301074f0 msgr2=0x7fb730100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7301074f0 0x7fb730100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb72ffff640 1 -- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 msgr2=0x7fb7301003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb7301003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb72ffff640 1 -- 172.21.15.8:0/3972202181 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c009560 con 0x7fb7300690b0 2024-02-19T16:23:49.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.394+0000 7fb734cda640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb7301003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:49.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.395+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 0x7fb7301008f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb71c02c170 tx=0x7fb71c0042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.395+0000 7fb72dffb640 1 -- 172.21.15.8:0/3972202181 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0044c0 con 0x7fb7300690b0 2024-02-19T16:23:49.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.395+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7301a61f0 con 0x7fb7300690b0 2024-02-19T16:23:49.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.395+0000 7fb72dffb640 1 -- 172.21.15.8:0/3972202181 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb71c02c5d0 con 0x7fb7300690b0 2024-02-19T16:23:49.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.395+0000 7fb72dffb640 1 -- 172.21.15.8:0/3972202181 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c003df0 con 0x7fb7300690b0 2024-02-19T16:23:49.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.395+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7301a6720 con 0x7fb7300690b0 2024-02-19T16:23:49.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.397+0000 7fb739bf9740 1 Processor -- start 2024-02-19T16:23:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb72dffb640 1 -- 172.21.15.8:0/3972202181 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fb71c03e450 con 0x7fb7300690b0 2024-02-19T16:23:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 -- start start 2024-02-19T16:23:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a4e8b0 0x55ee18a50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee189b3f90 con 0x55ee18a45d30 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee189b14c0 con 0x55ee18a4e8b0 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.398+0000 7fb739bf9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee189acca0 con 0x55ee18a45380 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a4e8b0 0x55ee18a50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a45760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39554/0 (socket says 172.21.15.8:39554) 2024-02-19T16:23:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 -- 172.21.15.8:0/1155741006 learned_addr learned my addr 172.21.15.8:0/1155741006 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb734cda640 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 -- 172.21.15.8:0/1155741006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a4e8b0 msgr2=0x55ee18a50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a4e8b0 0x55ee18a50d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 -- 172.21.15.8:0/1155741006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 msgr2=0x55ee18a4e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb72dffb640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb7200a2400 0x7fb7200a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 -- 172.21.15.8:0/1155741006 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee1896f0b0 con 0x55ee18a45380 2024-02-19T16:23:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb72dffb640 1 -- 172.21.15.8:0/3972202181 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb71c0f0bb0 con 0x7fb7300690b0 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb734cda640 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb7200a2400 0x7fb7200a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb7354db640 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a45760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728009fd0 tx=0x7fb7280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff939aee69301e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.399+0000 7fb71affd640 1 -- 172.21.15.8:0/1155741006 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728020030 con 0x55ee18a45380 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb71affd640 1 -- 172.21.15.8:0/1155741006 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb728004760 con 0x55ee18a45380 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb71affd640 1 -- 172.21.15.8:0/1155741006 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72801bb40 con 0x55ee18a45380 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 -- 172.21.15.8:0/1155741006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 msgr2=0x55ee18a45760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb7354db640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb7200a2400 0x7fb7200a48c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb728020000 tx=0x7fb728021000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a45760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728009fd0 tx=0x7fb7280118d0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 -- 172.21.15.8:0/1155741006 shutdown_connections 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a4e8b0 0x55ee18a50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 --2- 172.21.15.8:0/1155741006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a45760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 -- 172.21.15.8:0/1155741006 >> 172.21.15.8:0/1155741006 conn(0x55ee18a3f840 msgr2=0x55ee18a3fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 -- 172.21.15.8:0/1155741006 shutdown_connections 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.400+0000 7fb739bf9740 1 -- 172.21.15.8:0/1155741006 wait complete. 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb739bf9740 1 -- start start 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb739bf9740 1 Processor -- start 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45380 0x55ee18a4b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a4e8b0 0x55ee18a4c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb739bf9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18951f40 con 0x55ee18a45d30 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49746/0 (socket says 172.21.15.8:49746) 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.401+0000 7fb72ffff640 1 -- 172.21.15.8:0/2931196781 learned_addr learned my addr 172.21.15.8:0/2931196781 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee186804b0 con 0x55ee18a45380 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18932670 con 0x55ee18a4e8b0 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45380 0x55ee18a4b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb7354db640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a4e8b0 0x55ee18a4c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb739bf9740 10 client.0 _ll_register_callbacks cb 0x55ee1899ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 -- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a4e8b0 msgr2=0x55ee18a4c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a4e8b0 0x55ee18a4c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 -- 172.21.15.8:0/2931196781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 msgr2=0x55ee18a4bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 -- 172.21.15.8:0/2931196781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb728009c80 con 0x55ee18a45380 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb72ffff640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee18986f40 con 0x55ee18a45380 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb734cda640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45380 0x55ee18a4b880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55ee18a44e50 tx=0x7fb724012d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf1419fb254f950 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb70ffff640 10 client.25697 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb70ffff640 1 -- 172.21.15.8:0/2931196781 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724017070 con 0x55ee18a45380 2024-02-19T16:23:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb70ffff640 1 -- 172.21.15.8:0/2931196781 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb724004020 con 0x55ee18a45380 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb70ffff640 1 -- 172.21.15.8:0/2931196781 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb724004200 con 0x55ee18a45380 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb739bf9740 10 client.25697 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.402+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x55ee189ab320 con 0x55ee18a45380 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb70ffff640 1 -- 172.21.15.8:0/2931196781 <== mon.2 v2:172.21.15.156:3300/0 4 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb724004360 con 0x55ee18a45380 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb739bf9740 10 client.25697 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb739bf9740 10 client.25697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb739bf9740 20 client.25697 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb739bf9740 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18a5ecc0 0x55ee18a610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb739bf9740 4 client.25697 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee189a3310 con 0x55ee18a5ecc0 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb72ffff640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18a5ecc0 0x55ee18a610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb70ffff640 10 client.25697 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.403+0000 7fb72ffff640 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18a5ecc0 0x55ee18a610d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4763843e8fa7a14 server_cookie=584c320f83a5c826 in_seq=0 out_seq=0 2024-02-19T16:23:49.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb70ffff640 1 -- 172.21.15.8:0/2931196781 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee189a3310 con 0x55ee18a5ecc0 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb70ffff640 10 client.25697 handle_command_reply: tid=0 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 client.25697 shutdown 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45380 msgr2=0x55ee18a4b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45380 0x55ee18a4b880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55ee18a44e50 tx=0x7fb724012d90 comp rx=0 tx=0).stop 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 shutdown_connections 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18a5ecc0 0x55ee18a610d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a4e8b0 0x55ee18a4c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a45d30 0x55ee18a4bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb70ffff640 0 client.25697 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 --2- 172.21.15.8:0/2931196781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45380 0x55ee18a4b880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 >> 172.21.15.8:0/2931196781 conn(0x55ee18a3f840 msgr2=0x55ee18a417a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 shutdown_connections 2024-02-19T16:23:49.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.404+0000 7fb739bf9740 1 -- 172.21.15.8:0/2931196781 wait complete. 2024-02-19T16:23:49.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.405+0000 7fb739bf9740 20 client.25697 trim_cache size 0 max 16384 2024-02-19T16:23:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.434+0000 7fb739bf9740 1 Processor -- start 2024-02-19T16:23:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.434+0000 7fb739bf9740 1 -- start start 2024-02-19T16:23:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a4c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a59de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb739bf9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18a471a0 con 0x55ee18a4e8b0 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb739bf9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18a3f3a0 con 0x55ee18a45d30 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb739bf9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18a446f0 con 0x55ee18a45380 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a4c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb734cda640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb7354db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a59de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a4c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52578/0 (socket says 172.21.15.8:52578) 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb734cda640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a4dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39578/0 (socket says 172.21.15.8:39578) 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 -- 172.21.15.8:0/3426186741 learned_addr learned my addr 172.21.15.8:0/3426186741 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 -- 172.21.15.8:0/3426186741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 msgr2=0x55ee18a4dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a4dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 -- 172.21.15.8:0/3426186741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 msgr2=0x55ee18a59de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a59de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.435+0000 7fb72ffff640 1 -- 172.21.15.8:0/3426186741 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee18947c00 con 0x55ee18a45d30 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.436+0000 7fb72ffff640 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a4c120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0b9a80 tx=0x7fb71c003840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248560074eb1ffc8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.436+0000 7fb71a7fc640 1 -- 172.21.15.8:0/3426186741 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0bce80 con 0x55ee18a45d30 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.436+0000 7fb71a7fc640 1 -- 172.21.15.8:0/3426186741 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb71c0bcbf0 con 0x55ee18a45d30 2024-02-19T16:23:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.436+0000 7fb71a7fc640 1 -- 172.21.15.8:0/3426186741 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0bb3b0 con 0x55ee18a45d30 2024-02-19T16:23:49.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 -- 172.21.15.8:0/3426186741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 msgr2=0x55ee18a4c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a4c120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0b9a80 tx=0x7fb71c003840 comp rx=0 tx=0).stop 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 -- 172.21.15.8:0/3426186741 shutdown_connections 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a59de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a4c120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 --2- 172.21.15.8:0/3426186741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 -- 172.21.15.8:0/3426186741 >> 172.21.15.8:0/3426186741 conn(0x55ee18a3f840 msgr2=0x55ee18a41730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 -- 172.21.15.8:0/3426186741 shutdown_connections 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 -- 172.21.15.8:0/3426186741 wait complete. 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 -- start start 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.437+0000 7fb739bf9740 1 Processor -- start 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a56480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a52800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb734cda640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a56480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb734cda640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a56480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:39580/0 (socket says 172.21.15.8:39580) 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb7354db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a52800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a52800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52580/0 (socket says 172.21.15.8:52580) 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb7354db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a52d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49770/0 (socket says 172.21.15.8:49770) 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18a53280 con 0x55ee18a4e8b0 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18a533f0 con 0x55ee18a45d30 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee18a535b0 con 0x55ee18a45380 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb734cda640 1 -- 172.21.15.8:0/143537590 learned_addr learned my addr 172.21.15.8:0/143537590 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.438+0000 7fb739bf9740 10 client.0 _ll_register_callbacks cb 0x7fb724009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb734cda640 1 -- 172.21.15.8:0/143537590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 msgr2=0x55ee18a52800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb734cda640 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a52800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb734cda640 1 -- 172.21.15.8:0/143537590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 msgr2=0x55ee18a52d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb734cda640 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a52d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb734cda640 1 -- 172.21.15.8:0/143537590 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7301a6720 con 0x55ee18a45380 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb72ffff640 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a52800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb734cda640 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a56480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb72400fee0 tx=0x7fb7240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd742e8ff76c9945 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb70ffff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb70ffff640 1 -- 172.21.15.8:0/143537590 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724004450 con 0x55ee18a45380 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee189d2900 con 0x55ee18a45380 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb70ffff640 1 -- 172.21.15.8:0/143537590 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb724000e40 con 0x55ee18a45380 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb70ffff640 1 -- 172.21.15.8:0/143537590 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb724036710 con 0x55ee18a45380 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb739bf9740 10 client.25798 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ee1896d910 con 0x55ee18a45380 2024-02-19T16:23:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.439+0000 7fb70ffff640 1 -- 172.21.15.8:0/143537590 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ee189d2900 con 0x55ee18a45380 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb70ffff640 1 -- 172.21.15.8:0/143537590 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb7240035f0 con 0x55ee18a45380 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb739bf9740 10 client.25798 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb739bf9740 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb739bf9740 20 client.25798 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb739bf9740 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18abb9b0 0x55ee18abdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb739bf9740 4 client.25798 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ee186ac860 con 0x55ee18abb9b0 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb72ffff640 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18abb9b0 0x55ee18abdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb70ffff640 10 client.25798 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.440+0000 7fb72ffff640 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18abb9b0 0x55ee18abdda0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca885e8853f1af1 server_cookie=a7748c898b9db534 in_seq=0 out_seq=0 2024-02-19T16:23:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.441+0000 7fb70ffff640 1 -- 172.21.15.8:0/143537590 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee186ac860 con 0x55ee18abb9b0 2024-02-19T16:23:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.441+0000 7fb70ffff640 10 client.25798 handle_command_reply: tid=0 2024-02-19T16:23:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.441+0000 7fb739bf9740 1 client.25798 shutdown 2024-02-19T16:23:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 msgr2=0x55ee18a56480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a56480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb72400fee0 tx=0x7fb7240046c0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 shutdown_connections 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ee18a4e8b0 0x55ee18a52d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee18a45d30 0x55ee18a52800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ee18abb9b0 0x55ee18abdda0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 --2- 172.21.15.8:0/143537590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee18a45380 0x55ee18a56480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb70ffff640 0 client.25798 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 >> 172.21.15.8:0/143537590 conn(0x55ee18a3f840 msgr2=0x55ee18a40ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 shutdown_connections 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 1 -- 172.21.15.8:0/143537590 wait complete. 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.442+0000 7fb739bf9740 20 client.25798 trim_cache size 0 max 16384 2024-02-19T16:23:49.447 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:23:49.448 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:23:49.448 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "2ec8450d-ef3e-4a1e-8b55-aeefd72ac5f3", 2024-02-19T16:23:49.448 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:23:49.448 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:23:49.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb7200a2400 msgr2=0x7fb7200a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb7200a2400 0x7fb7200a48c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb728020000 tx=0x7fb728021000 comp rx=0 tx=0).stop 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 msgr2=0x7fb7301008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 0x7fb7301008f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb71c02c170 tx=0x7fb71c0042d0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 shutdown_connections 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7301074f0 0x7fb730100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb7200a2400 0x7fb7200a48c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7300690b0 0x7fb7301008f0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb71c02c170 tx=0x7fb71c0042d0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 --2- 172.21.15.8:0/3972202181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7300686f0 0x7fb7301003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 >> 172.21.15.8:0/3972202181 conn(0x7fb7300774f0 msgr2=0x7fb7300fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 shutdown_connections 2024-02-19T16:23:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.443+0000 7fb736f3e640 1 -- 172.21.15.8:0/3972202181 wait complete. 2024-02-19T16:23:49.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:23:49.459 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.565+0000 7f4ef9623640 1 -- 172.21.15.8:0/774014869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef4106830 msgr2=0x7f4ef410e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.565+0000 7f4ef9623640 1 --2- 172.21.15.8:0/774014869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef4106830 0x7f4ef410e3b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009ca0 tx=0x7f4eec02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.565+0000 7f4ef9623640 1 -- 172.21.15.8:0/774014869 shutdown_connections 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 --2- 172.21.15.8:0/774014869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef4106830 0x7f4ef410e3b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 --2- 172.21.15.8:0/774014869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ef41005b0 0x7f4ef4106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 --2- 172.21.15.8:0/774014869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ef40ffc00 0x7f4ef40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 -- 172.21.15.8:0/774014869 >> 172.21.15.8:0/774014869 conn(0x7f4ef4076920 msgr2=0x7f4ef40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 -- 172.21.15.8:0/774014869 shutdown_connections 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 -- 172.21.15.8:0/774014869 wait complete. 2024-02-19T16:23:49.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.566+0000 7f4ef9623640 1 Processor -- start 2024-02-19T16:23:49.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 -- start start 2024-02-19T16:23:49.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 0x7f4ef419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ef4106830 0x7f4ef419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ef41978a0 0x7f4ef4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 0x7f4ef419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 0x7f4ef419d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52586/0 (socket says 172.21.15.8:52586) 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 -- 172.21.15.8:0/4163798958 learned_addr learned my addr 172.21.15.8:0/4163798958 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef37fe640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ef41978a0 0x7f4ef4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ef4106830 0x7f4ef419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4198350 con 0x7f4ef41978a0 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41984c0 con 0x7f4ef40ffc00 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4198630 con 0x7f4ef4106830 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 -- 172.21.15.8:0/4163798958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ef4106830 msgr2=0x7f4ef419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ef4106830 0x7f4ef419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 -- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ef41978a0 msgr2=0x7f4ef4197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ef41978a0 0x7f4ef4197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.567+0000 7f4ef2ffd640 1 -- 172.21.15.8:0/4163798958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec009900 con 0x7f4ef40ffc00 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.568+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 0x7f4ef419d6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e9f0 tx=0x7f4ee400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.568+0000 7f4edbfff640 1 -- 172.21.15.8:0/4163798958 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4009720 con 0x7f4ef40ffc00 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.568+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef41a4590 con 0x7f4ef40ffc00 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.568+0000 7f4edbfff640 1 -- 172.21.15.8:0/4163798958 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee400cd80 con 0x7f4ef40ffc00 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.568+0000 7f4edbfff640 1 -- 172.21.15.8:0/4163798958 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4010690 con 0x7f4ef40ffc00 2024-02-19T16:23:49.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.568+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef41a4a70 con 0x7f4ef40ffc00 2024-02-19T16:23:49.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.570+0000 7f4edbfff640 1 -- 172.21.15.8:0/4163798958 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f4ee40046c0 con 0x7f4ef40ffc00 2024-02-19T16:23:49.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.571+0000 7f4edbfff640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4edc0a23d0 0x7f4edc0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.571+0000 7f4edbfff640 1 -- 172.21.15.8:0/4163798958 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4ee4014070 con 0x7f4ef40ffc00 2024-02-19T16:23:49.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.571+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4edc0a23d0 0x7f4edc0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.571+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4edc0a23d0 0x7f4edc0a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ff160 tx=0x7f4ee8005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 Processor -- start 2024-02-19T16:23:49.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 -- start start 2024-02-19T16:23:49.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bee2b0 0x557541bee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 0x557541bf7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bf77b0 0x557541bf9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541b5d280 con 0x557541beebd0 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541b5a7b0 con 0x557541bee2b0 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4efc201740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541b55f90 con 0x557541bf77b0 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 0x557541bf7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 0x557541bf7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49808/0 (socket says 172.21.15.8:49808) 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 -- 172.21.15.8:0/773203077 learned_addr learned my addr 172.21.15.8:0/773203077 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bee2b0 0x557541bee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef37fe640 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bf77b0 0x557541bf9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 -- 172.21.15.8:0/773203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bf77b0 msgr2=0x557541bf9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bf77b0 0x557541bf9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 -- 172.21.15.8:0/773203077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bee2b0 msgr2=0x557541bee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bee2b0 0x557541bee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.572+0000 7f4ef27fc640 1 -- 172.21.15.8:0/773203077 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557541b14570 con 0x557541beebd0 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.573+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 0x557541bf7270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e600 tx=0x7f4ee8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6384bd75b427c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.573+0000 7f4ed97fa640 1 -- 172.21.15.8:0/773203077 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8004020 con 0x557541beebd0 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.573+0000 7f4ed97fa640 1 -- 172.21.15.8:0/773203077 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8021070 con 0x557541beebd0 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.573+0000 7f4ed97fa640 1 -- 172.21.15.8:0/773203077 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee801c670 con 0x557541beebd0 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 -- 172.21.15.8:0/773203077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 msgr2=0x557541bf7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 0x557541bf7270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e600 tx=0x7f4ee8014000 comp rx=0 tx=0).stop 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 -- 172.21.15.8:0/773203077 shutdown_connections 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bf77b0 0x557541bf9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541beebd0 0x557541bf7270 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 --2- 172.21.15.8:0/773203077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bee2b0 0x557541bee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 -- 172.21.15.8:0/773203077 >> 172.21.15.8:0/773203077 conn(0x557541be8990 msgr2=0x557541be8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 -- 172.21.15.8:0/773203077 shutdown_connections 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 -- 172.21.15.8:0/773203077 wait complete. 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 -- start start 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.574+0000 7f4efc201740 1 Processor -- start 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541bf02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541829430 con 0x557541bf77b0 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575417e5e00 con 0x557541bf07f0 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541adafd0 con 0x557541bee2b0 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 10 client.0 _ll_register_callbacks cb 0x557541b30fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52596/0 (socket says 172.21.15.8:52596) 2024-02-19T16:23:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 -- 172.21.15.8:0/1256587509 learned_addr learned my addr 172.21.15.8:0/1256587509 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541bf02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 -- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 msgr2=0x557541bf7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 -- 172.21.15.8:0/1256587509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 msgr2=0x557541bf02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541bf02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 -- 172.21.15.8:0/1256587509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee8017070 con 0x557541bf07f0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541bf02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ef37fe640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf0c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4eec02c700 tx=0x7f4eec034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c18cd6050441dde server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ed6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/1256587509 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0328b0 con 0x557541bf07f0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/1256587509 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4eec046070 con 0x557541bf07f0 2024-02-19T16:23:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.575+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575418557e0 con 0x557541bf07f0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/1256587509 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eec032eb0 con 0x557541bf07f0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 10 client.25715 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557541b54610 con 0x557541bf07f0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/1256587509 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575418557e0 con 0x557541bf07f0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/1256587509 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4eec0044b0 con 0x557541bf07f0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 10 client.25715 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 10 client.25715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 20 client.25715 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c107b0 0x557541c12bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 4 client.25715 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557541892720 con 0x557541c107b0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.576+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c107b0 0x557541c12bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4ed6ff5640 10 client.25715 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c107b0 0x557541c12bd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac64bfa7635f2af server_cookie=eff4fbb0399f9925 in_seq=0 out_seq=0 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/1256587509 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557541892720 con 0x557541c107b0 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4ed6ff5640 10 client.25715 handle_command_reply: tid=0 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4efc201740 1 client.25715 shutdown 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 msgr2=0x557541bf0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.577+0000 7f4efc201740 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf0c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4eec02c700 tx=0x7f4eec034670 comp rx=0 tx=0).stop 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 shutdown_connections 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf0c70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541bf02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c107b0 0x557541c12bd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 --2- 172.21.15.8:0/1256587509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 >> 172.21.15.8:0/1256587509 conn(0x557541be8990 msgr2=0x557541bea730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4ed6ff5640 0 client.25715 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 shutdown_connections 2024-02-19T16:23:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 1 -- 172.21.15.8:0/1256587509 wait complete. 2024-02-19T16:23:49.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.578+0000 7f4efc201740 20 client.25715 trim_cache size 0 max 16384 2024-02-19T16:23:49.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.610+0000 7f4efc201740 1 Processor -- start 2024-02-19T16:23:49.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.610+0000 7f4efc201740 1 -- start start 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.610+0000 7f4efc201740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.610+0000 7f4efc201740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4efc201740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541befb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4efc201740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541bf9fa0 con 0x557541bf77b0 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4efc201740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541be9c20 con 0x557541bf07f0 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4efc201740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541bed660 con 0x557541bee2b0 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541befb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52604/0 (socket says 172.21.15.8:52604) 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541befb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49812/0 (socket says 172.21.15.8:49812) 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 -- 172.21.15.8:0/628226320 learned_addr learned my addr 172.21.15.8:0/628226320 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 -- 172.21.15.8:0/628226320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 msgr2=0x557541bf6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 -- 172.21.15.8:0/628226320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 msgr2=0x557541befb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541befb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 -- 172.21.15.8:0/628226320 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557541af0ac0 con 0x557541bf07f0 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf6f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e5d0 tx=0x7f4ee8011970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d05754a4e99b884 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.611+0000 7f4ed8ff9640 1 -- 172.21.15.8:0/628226320 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee802c070 con 0x557541bf07f0 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4ed8ff9640 1 -- 172.21.15.8:0/628226320 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee800edb0 con 0x557541bf07f0 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4ed8ff9640 1 -- 172.21.15.8:0/628226320 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80048e0 con 0x557541bf07f0 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 -- 172.21.15.8:0/628226320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 msgr2=0x557541bf6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf6f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e5d0 tx=0x7f4ee8011970 comp rx=0 tx=0).stop 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 -- 172.21.15.8:0/628226320 shutdown_connections 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541befb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541bf6f90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 --2- 172.21.15.8:0/628226320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541bf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 -- 172.21.15.8:0/628226320 >> 172.21.15.8:0/628226320 conn(0x557541be8990 msgr2=0x557541be8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 -- 172.21.15.8:0/628226320 shutdown_connections 2024-02-19T16:23:49.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.612+0000 7f4efc201740 1 -- 172.21.15.8:0/628226320 wait complete. 2024-02-19T16:23:49.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 -- start start 2024-02-19T16:23:49.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 Processor -- start 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541c0c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541c0cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541c092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541c099b0 con 0x557541bf77b0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541c09b20 con 0x557541bf07f0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557541c09ce0 con 0x557541bee2b0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541c0cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541c0cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:52612/0 (socket says 172.21.15.8:52612) 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4ef27fc640 1 -- 172.21.15.8:0/240799197 learned_addr learned my addr 172.21.15.8:0/240799197 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4efc201740 10 client.0 _ll_register_callbacks cb 0x7f4ee4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.613+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541c0c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef37fe640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541c092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef27fc640 1 -- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 msgr2=0x557541c0c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541c0c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef27fc640 1 -- 172.21.15.8:0/240799197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 msgr2=0x557541c092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541c092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef27fc640 1 -- 172.21.15.8:0/240799197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef41a4a70 con 0x557541bf07f0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541c0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ef27fc640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541c0cc40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8011940 tx=0x7f4ee800ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc688d4b36af9f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ed6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/240799197 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee802c070 con 0x557541bf07f0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/240799197 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4ee8005240 con 0x557541bf07f0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee8021070 con 0x557541bf07f0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/240799197 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee8003800 con 0x557541bf07f0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4efc201740 10 client.25721 fetch_fsmap learned FSMap version 24 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.614+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557541b14570 con 0x557541bf07f0 2024-02-19T16:23:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/240799197 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8003e40 con 0x557541bf07f0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/240799197 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4ee8026b70 con 0x557541bf07f0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4efc201740 10 client.25721 fetch_fsmap finished waiting for FSMap version 24 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4efc201740 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4efc201740 20 client.25721 populate_metadata read hostname 'smithi008' 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4efc201740 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c6ceb0 0x557541c6f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4efc201740 4 client.25721 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f4ee8021070 con 0x557541c6ceb0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.615+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c6ceb0 0x557541c6f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4ed6ff5640 10 client.25721 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4ef2ffd640 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c6ceb0 0x557541c6f2a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4d31953ab042c2 server_cookie=33d5dfe96d67ee77 in_seq=0 out_seq=0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4ed6ff5640 1 -- 172.21.15.8:0/240799197 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f4ee8021070 con 0x557541c6ceb0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4ed6ff5640 10 client.25721 handle_command_reply: tid=0 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 client.25721 shutdown 2024-02-19T16:23:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 msgr2=0x557541c0cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541c0cc40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8011940 tx=0x7f4ee800ec00 comp rx=0 tx=0).stop 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 shutdown_connections 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557541bf77b0 0x557541c092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557541bf07f0 0x557541c0cc40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557541c6ceb0 0x557541c6f2a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4ed6ff5640 0 client.25721 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 --2- 172.21.15.8:0/240799197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557541bee2b0 0x557541c0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 >> 172.21.15.8:0/240799197 conn(0x557541be8990 msgr2=0x557541befb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 shutdown_connections 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 1 -- 172.21.15.8:0/240799197 wait complete. 2024-02-19T16:23:49.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.616+0000 7f4efc201740 20 client.25721 trim_cache size 0 max 16384 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout: "2ec8450d-ef3e-4a1e-8b55-aeefd72ac5f3": { 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "2ec8450d-ef3e-4a1e-8b55-aeefd72ac5f3", 2024-02-19T16:23:49.627 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:23:49.628 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:23:49.628 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:23:49.628 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4edc0a23d0 msgr2=0x7f4edc0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4edc0a23d0 0x7f4edc0a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ff160 tx=0x7f4ee8005f90 comp rx=0 tx=0).stop 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 msgr2=0x7f4ef419d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 0x7f4ef419d6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e9f0 tx=0x7f4ee400edf0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 shutdown_connections 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ef41978a0 0x7f4ef4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4edc0a23d0 0x7f4edc0a4890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ef4106830 0x7f4ef419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 --2- 172.21.15.8:0/4163798958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ef40ffc00 0x7f4ef419d6e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:23:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 >> 172.21.15.8:0/4163798958 conn(0x7f4ef4076920 msgr2=0x7f4ef4104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:23:49.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 shutdown_connections 2024-02-19T16:23:49.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:23:49.617+0000 7f4ef9623640 1 -- 172.21.15.8:0/4163798958 wait complete. 2024-02-19T16:23:49.631 INFO:tasks.cephfs.filesystem:scrub status for tag:2ec8450d-ef3e-4a1e-8b55-aeefd72ac5f3 - {'path': '/', 'tag': '2ec8450d-ef3e-4a1e-8b55-aeefd72ac5f3', 'options': 'recursive,force'} 2024-02-19T16:23:50.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:50 smithi191 ceph-mon[37203]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:23:50.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:50 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:23:50.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:50 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:50.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:50 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:23:50.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:50 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:50 smithi156 ceph-mon[36502]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:23:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:50 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:23:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:50 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:50 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:23:50.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:50 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:50.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:50 smithi008 ceph-mon[32573]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:23:50.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:50 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:23:50.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:50 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:50.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:50 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:23:50.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:50 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:23:52.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:52 smithi191 ceph-mon[37203]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:52.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:52 smithi156 ceph-mon[36502]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:52.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:52 smithi008 ceph-mon[32573]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:54.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:54 smithi191 ceph-mon[37203]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:54.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:54 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:23:54.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:54 smithi156 ceph-mon[36502]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:54.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:54 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:23:54.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:54 smithi008 ceph-mon[32573]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:23:54.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:54 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:23:56.276 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:56 smithi191 ceph-mon[37203]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:23:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:56 smithi156 ceph-mon[36502]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:23:56.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:56 smithi008 ceph-mon[32573]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:23:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:23:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:23:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:58 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:58 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:23:58.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:23:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:58.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:58 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:59.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:59 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:23:59.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:59 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:23:59.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:23:59 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:59.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:59 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:23:59.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:59 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:23:59.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:23:59 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:23:59.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:59 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:23:59.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:59 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:23:59.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:23:59 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:24:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:00 smithi156 ceph-mon[36502]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:24:00.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:00 smithi008 ceph-mon[32573]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:24:00.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:00 smithi191 ceph-mon[37203]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:24:02.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:02 smithi156 ceph-mon[36502]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:24:02.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:02 smithi008 ceph-mon[32573]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:24:02.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:02 smithi191 ceph-mon[37203]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:24:04.329 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:04 smithi191 ceph-mon[37203]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:24:04.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:04 smithi156 ceph-mon[36502]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:24:04.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:04 smithi008 ceph-mon[32573]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:24:06.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:06 smithi156 ceph-mon[36502]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:24:06.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:06 smithi008 ceph-mon[32573]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:24:06.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:06 smithi191 ceph-mon[37203]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:24:08.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:08 smithi156 ceph-mon[36502]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:08.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:08 smithi008 ceph-mon[32573]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:08.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:08 smithi191 ceph-mon[37203]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:08.777 INFO:tasks.workunit.client.0.smithi008.stderr:Updating files: 93% (11421/12253) Updating files: 94% (11518/12253) Updating files: 95% (11641/12253) Updating files: 96% (11763/12253) Updating files: 97% (11886/12253) Updating files: 98% (12008/12253) Updating files: 99% (12131/12253) Updating files: 100% (12253/12253) Updating files: 100% (12253/12253), done. 2024-02-19T16:24:09.385 INFO:tasks.workunit.client.0.smithi008.stderr:Note: switching to '9f8908eed2b2d7e8c316bd5be7ed6e49024931ee'. 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr:state without impacting any branches by switching back to a branch. 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr: git switch -c 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.386 INFO:tasks.workunit.client.0.smithi008.stderr:Or undo this operation with: 2024-02-19T16:24:09.387 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.387 INFO:tasks.workunit.client.0.smithi008.stderr: git switch - 2024-02-19T16:24:09.387 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.387 INFO:tasks.workunit.client.0.smithi008.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-19T16:24:09.387 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:09.387 INFO:tasks.workunit.client.0.smithi008.stderr:HEAD is now at 9f8908eed2b mds/quiesce: only use ACTIVE daemons for the quiesce cluster 2024-02-19T16:24:09.388 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-19T16:24:09.416 INFO:tasks.workunit.client.0.smithi008.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-19T16:24:09.418 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-19T16:24:09.418 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-19T16:24:10.122 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-19T16:24:10.185 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-19T16:24:10.224 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-19T16:24:10.226 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-19T16:24:10.226 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-19T16:24:10.264 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-19T16:24:10.267 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:24:10.267 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-19T16:24:10.323 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-02-19T16:24:10.392 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-02-19T16:24:10.393 DEBUG:teuthology.orchestra.run.smithi008:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-02-19T16:24:10.393 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:10 smithi156 ceph-mon[36502]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:24:10.413 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2024-02-19T16:24:10.413 INFO:tasks.workunit.client.0.smithi008.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-02-19T16:24:10.415 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into 'xfstests-dev'... 2024-02-19T16:24:10.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:10 smithi008 ceph-mon[32573]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:24:10.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:10 smithi191 ceph-mon[37203]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:24:12.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:12 smithi156 ceph-mon[36502]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:12.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:12 smithi008 ceph-mon[32573]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:12 smithi191 ceph-mon[37203]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:14.321 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:14 smithi008 ceph-mon[32573]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:14.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:14 smithi156 ceph-mon[36502]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:14.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:14 smithi191 ceph-mon[37203]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:24:16.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:16 smithi156 ceph-mon[36502]: pgmap v44: 129 pgs: 129 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-02-19T16:24:16.523 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:16 smithi008 ceph-mon[32573]: pgmap v44: 129 pgs: 129 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-02-19T16:24:16.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:16 smithi191 ceph-mon[37203]: pgmap v44: 129 pgs: 129 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-02-19T16:24:18.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:18 smithi156 ceph-mon[36502]: pgmap v45: 129 pgs: 129 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-02-19T16:24:18.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:18 smithi008 ceph-mon[32573]: pgmap v45: 129 pgs: 129 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-02-19T16:24:18.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:18 smithi191 ceph-mon[37203]: pgmap v45: 129 pgs: 129 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-02-19T16:24:19.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:24:19.632 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:24:19.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 -- 172.21.15.8:0/89367654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 msgr2=0x7ff7e810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 --2- 172.21.15.8:0/89367654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e810c9d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009cd0 tx=0x7ff7e002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 -- 172.21.15.8:0/89367654 shutdown_connections 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 --2- 172.21.15.8:0/89367654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e810c9d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 --2- 172.21.15.8:0/89367654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e80ffb00 0x7ff7e80fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 --2- 172.21.15.8:0/89367654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8105ba0 0x7ff7e8105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 -- 172.21.15.8:0/89367654 >> 172.21.15.8:0/89367654 conn(0x7ff7e8076920 msgr2=0x7ff7e80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 -- 172.21.15.8:0/89367654 shutdown_connections 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.741+0000 7ff7ef719640 1 -- 172.21.15.8:0/89367654 wait complete. 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 Processor -- start 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 -- start start 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e80ffb00 0x7ff7e819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8105ba0 0x7ff7e81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8197f40 con 0x7ff7e81004c0 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e81980b0 con 0x7ff7e8105ba0 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ef719640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8198270 con 0x7ff7e80ffb00 2024-02-19T16:24:19.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e819dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45244/0 (socket says 172.21.15.8:45244) 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 -- 172.21.15.8:0/941523469 learned_addr learned my addr 172.21.15.8:0/941523469 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7ed4b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e80ffb00 0x7ff7e819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8105ba0 0x7ff7e81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 -- 172.21.15.8:0/941523469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e80ffb00 msgr2=0x7ff7e819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e80ffb00 0x7ff7e819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 -- 172.21.15.8:0/941523469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8105ba0 msgr2=0x7ff7e81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8105ba0 0x7ff7e81978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.742+0000 7ff7eccb4640 1 -- 172.21.15.8:0/941523469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0009900 con 0x7ff7e81004c0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.743+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e819dc80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00ea40 tx=0x7ff7dc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.743+0000 7ff7e67fc640 1 -- 172.21.15.8:0/941523469 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00cd40 con 0x7ff7e81004c0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.743+0000 7ff7e67fc640 1 -- 172.21.15.8:0/941523469 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7dc005500 con 0x7ff7e81004c0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.743+0000 7ff7e67fc640 1 -- 172.21.15.8:0/941523469 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc016670 con 0x7ff7e81004c0 2024-02-19T16:24:19.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.743+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e8198550 con 0x7ff7e81004c0 2024-02-19T16:24:19.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.743+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e81a47e0 con 0x7ff7e81004c0 2024-02-19T16:24:19.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.745+0000 7ff7e67fc640 1 -- 172.21.15.8:0/941523469 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7ff7dc004ec0 con 0x7ff7e81004c0 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 Processor -- start 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 -- start start 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x560043955690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x56004395e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56004395e7b0 0x560043960ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600438c4280 con 0x5600439552b0 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600438c17b0 con 0x56004395e7b0 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600438bcf90 con 0x560043955bd0 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7e67fc640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff7d00a24a0 0x7ff7d00a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x560043955690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7ed4b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x56004395e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x560043955690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45248/0 (socket says 172.21.15.8:45248) 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3518745918 learned_addr learned my addr 172.21.15.8:0/3518745918 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56004395e7b0 0x560043960ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff7d00a24a0 0x7ff7d00a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7e67fc640 1 -- 172.21.15.8:0/941523469 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff7dc0ce3f0 con 0x7ff7e81004c0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3518745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 msgr2=0x56004395e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x56004395e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3518745918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56004395e7b0 msgr2=0x560043960ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56004395e7b0 0x560043960ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3518745918 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004387b570 con 0x5600439552b0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.746+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56004395e7b0 0x560043960ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x560043955690 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0004af0 tx=0x7ff7e0034810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ceaa8dbd076e22 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff7d00a24a0 0x7ff7d00a4960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002c700 tx=0x7ff7e003c000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7cf7fe640 1 -- 172.21.15.8:0/3518745918 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0048070 con 0x5600439552b0 2024-02-19T16:24:19.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7cf7fe640 1 -- 172.21.15.8:0/3518745918 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7e0032df0 con 0x5600439552b0 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7cf7fe640 1 -- 172.21.15.8:0/3518745918 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00042a0 con 0x5600439552b0 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3518745918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 msgr2=0x560043955690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.747+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x560043955690 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0004af0 tx=0x7ff7e0034810 comp rx=0 tx=0).stop 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3518745918 shutdown_connections 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56004395e7b0 0x560043960ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x56004395e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3518745918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x560043955690 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3518745918 >> 172.21.15.8:0/3518745918 conn(0x56004394f990 msgr2=0x56004394fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3518745918 shutdown_connections 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3518745918 wait complete. 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- start start 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 Processor -- start 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x56004395e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560043955bd0 0x56004395b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x56004395bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043590430 con 0x5600439552b0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004354ce00 con 0x560043955bd0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043841fd0 con 0x56004395e7b0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7f23cc740 10 client.0 _ll_register_callbacks cb 0x560043897fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7ed4b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x56004395e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7ed4b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x56004395e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45252/0 (socket says 172.21.15.8:45252) 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.748+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/1773890221 learned_addr learned my addr 172.21.15.8:0/1773890221 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x56004395bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560043955bd0 0x56004395b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 msgr2=0x56004395bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x56004395bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/1773890221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560043955bd0 msgr2=0x56004395b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560043955bd0 0x56004395b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/1773890221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e810d2d0 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x56004395e260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8012700 tx=0x7ff7d8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ae1ad6f2132162 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ccff9640 1 -- 172.21.15.8:0/1773890221 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8017070 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ccff9640 1 -- 172.21.15.8:0/1773890221 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff7d800f8c0 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600435f9720 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ccff9640 1 -- 172.21.15.8:0/1773890221 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7d800fa80 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7f23cc740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5600438bb610 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.749+0000 7ff7ccff9640 1 -- 172.21.15.8:0/1773890221 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5600435f9720 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7ccff9640 1 -- 172.21.15.8:0/1773890221 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff7d8010430 con 0x5600439552b0 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7f23cc740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7f23cc740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7f23cc740 20 client.15960 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:19.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56004396ea80 0x560043970ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7f23cc740 4 client.15960 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560043648890 con 0x56004396ea80 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56004396ea80 0x560043970ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56004396ea80 0x560043970ea0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a3b421ee813cd8 server_cookie=8a88a577782e54b0 in_seq=0 out_seq=0 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.750+0000 7ff7ccff9640 10 client.15960 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7ccff9640 1 -- 172.21.15.8:0/1773890221 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560043648890 con 0x56004396ea80 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7ccff9640 10 client.15960 handle_command_reply: tid=0 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 client.15960 shutdown 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 msgr2=0x56004395e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x56004395e260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8012700 tx=0x7ff7d8012bc0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 shutdown_connections 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56004396ea80 0x560043970ea0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x56004395bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560043955bd0 0x56004395b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7ccff9640 0 client.15960 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:19.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/1773890221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600439552b0 0x56004395e260 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 >> 172.21.15.8:0/1773890221 conn(0x56004394f990 msgr2=0x560043951730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 shutdown_connections 2024-02-19T16:24:19.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 1 -- 172.21.15.8:0/1773890221 wait complete. 2024-02-19T16:24:19.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.751+0000 7ff7f23cc740 20 client.15960 trim_cache size 0 max 16384 2024-02-19T16:24:19.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.768+0000 7ff7f23cc740 1 Processor -- start 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 -- start start 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x56004395dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 0x56004395e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x560043969ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043950a10 con 0x560043955bd0 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043950790 con 0x5600439552b0 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043951cf0 con 0x56004395e7b0 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7eccb4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 0x56004395e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7ed4b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x56004395dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7eccb4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 0x56004395e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45256/0 (socket says 172.21.15.8:45256) 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7eccb4640 1 -- 172.21.15.8:0/3100304464 learned_addr learned my addr 172.21.15.8:0/3100304464 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x560043969ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/3100304464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 msgr2=0x560043969ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x560043969ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/3100304464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 msgr2=0x56004395e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 0x56004395e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7ed4b5640 1 -- 172.21.15.8:0/3100304464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560043857ac0 con 0x5600439552b0 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.769+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 0x56004395e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x56004395dd20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8010810 tx=0x7ff7d8009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471cf2135e3f0208 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7ceffd640 1 -- 172.21.15.8:0/3100304464 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8017040 con 0x5600439552b0 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7ceffd640 1 -- 172.21.15.8:0/3100304464 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7d8010a00 con 0x5600439552b0 2024-02-19T16:24:19.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7ceffd640 1 -- 172.21.15.8:0/3100304464 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80053b0 con 0x5600439552b0 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3100304464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 msgr2=0x56004395dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x56004395dd20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8010810 tx=0x7ff7d8009dd0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3100304464 shutdown_connections 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56004395e7b0 0x560043969ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560043955bd0 0x56004395e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3100304464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x56004395dd20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.770+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3100304464 >> 172.21.15.8:0/3100304464 conn(0x56004394f990 msgr2=0x56004394fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3100304464 shutdown_connections 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3100304464 wait complete. 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 -- start start 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 Processor -- start 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x560043966070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x5600439623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 0x560043962930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043966940 con 0x56004395e7b0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043962e70 con 0x5600439552b0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560043963030 con 0x560043955bd0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7f23cc740 10 client.0 _ll_register_callbacks cb 0x7ff7d8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.771+0000 7ff7edcb6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 0x560043962930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7eccb4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x5600439623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 0x560043962930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45270/0 (socket says 172.21.15.8:45270) 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3353202162 learned_addr learned my addr 172.21.15.8:0/3353202162 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x560043966070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 msgr2=0x5600439623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x5600439623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3353202162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 msgr2=0x560043966070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:19.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x560043966070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 -- 172.21.15.8:0/3353202162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e8198550 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7eccb4640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x5600439623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x560043966070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7edcb6640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 0x560043962930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002c630 tx=0x7ff7e0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e91d8ec582b110e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7ccff9640 1 -- 172.21.15.8:0/3353202162 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0048040 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7ccff9640 1 -- 172.21.15.8:0/3353202162 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff7e0004450 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600435bc7e0 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7ccff9640 1 -- 172.21.15.8:0/3353202162 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e0032b40 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7f23cc740 10 client.15966 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.772+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5600438bb610 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7ccff9640 1 -- 172.21.15.8:0/3353202162 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5600435bc7e0 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7ccff9640 1 -- 172.21.15.8:0/3353202162 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff7e004cba0 con 0x56004395e7b0 2024-02-19T16:24:19.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7f23cc740 10 client.15966 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7f23cc740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7f23cc740 20 client.15966 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600439cb4d0 0x5600439cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7f23cc740 4 client.15966 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600435ab4e0 con 0x5600439cb4d0 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600439cb4d0 0x5600439cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7ccff9640 10 client.15966 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.773+0000 7ff7ed4b5640 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600439cb4d0 0x5600439cd8c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4a9fd1f15b096d server_cookie=1301457924eb551f in_seq=0 out_seq=0 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7ccff9640 1 -- 172.21.15.8:0/3353202162 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5600435ab4e0 con 0x5600439cb4d0 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7ccff9640 10 client.15966 handle_command_reply: tid=0 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 client.15966 shutdown 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 msgr2=0x560043962930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 0x560043962930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002c630 tx=0x7ff7e0009800 comp rx=0 tx=0).stop 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 shutdown_connections 2024-02-19T16:24:19.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56004395e7b0 0x560043962930 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600439cb4d0 0x5600439cd8c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560043955bd0 0x5600439623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 --2- 172.21.15.8:0/3353202162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600439552b0 0x560043966070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7ccff9640 0 client.15966 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.774+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 >> 172.21.15.8:0/3353202162 conn(0x56004394f990 msgr2=0x56004395ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 shutdown_connections 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7f23cc740 1 -- 172.21.15.8:0/3353202162 wait complete. 2024-02-19T16:24:19.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7f23cc740 20 client.15966 trim_cache size 0 max 16384 2024-02-19T16:24:19.782 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:24:19.782 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:24:19.782 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:24:19.782 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:24:19.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff7d00a24a0 msgr2=0x7ff7d00a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff7d00a24a0 0x7ff7d00a4960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002c700 tx=0x7ff7e003c000 comp rx=0 tx=0).stop 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 msgr2=0x7ff7e819dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e819dc80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00ea40 tx=0x7ff7dc00ef00 comp rx=0 tx=0).stop 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 shutdown_connections 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8105ba0 0x7ff7e81978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7e81004c0 0x7ff7e819dc80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff7d00a24a0 0x7ff7d00a4960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 --2- 172.21.15.8:0/941523469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e80ffb00 0x7ff7e819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 >> 172.21.15.8:0/941523469 conn(0x7ff7e8076920 msgr2=0x7ff7e81029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 shutdown_connections 2024-02-19T16:24:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.775+0000 7ff7ef719640 1 -- 172.21.15.8:0/941523469 wait complete. 2024-02-19T16:24:19.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:24:19.787 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:24:19.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 -- 172.21.15.8:0/3183272975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1400ff9f0 msgr2=0x7fa1400ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 --2- 172.21.15.8:0/3183272975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1400ff9f0 0x7fa1400ffe70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa130009890 tx=0x7fa13002bb70 comp rx=0 tx=0).stop 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 -- 172.21.15.8:0/3183272975 shutdown_connections 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 --2- 172.21.15.8:0/3183272975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1401003b0 0x7fa14010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 --2- 172.21.15.8:0/3183272975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1400ff9f0 0x7fa1400ffe70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 --2- 172.21.15.8:0/3183272975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa140107580 0x7fa140107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 -- 172.21.15.8:0/3183272975 >> 172.21.15.8:0/3183272975 conn(0x7fa140076980 msgr2=0x7fa1400fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 -- 172.21.15.8:0/3183272975 shutdown_connections 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.906+0000 7fa147572640 1 -- 172.21.15.8:0/3183272975 wait complete. 2024-02-19T16:24:19.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 Processor -- start 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 -- start start 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 0x7fa14019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1401003b0 0x7fa14019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa140107580 0x7fa140197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140197f50 con 0x7fa1400ff9f0 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401980c0 con 0x7fa1401003b0 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa147572640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140198280 con 0x7fa140107580 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 0x7fa14019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 0x7fa14019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45292/0 (socket says 172.21.15.8:45292) 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 -- 172.21.15.8:0/3003054683 learned_addr learned my addr 172.21.15.8:0/3003054683 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa145b0f640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa140107580 0x7fa140197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa144b0d640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1401003b0 0x7fa14019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 -- 172.21.15.8:0/3003054683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa140107580 msgr2=0x7fa140197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa140107580 0x7fa140197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 -- 172.21.15.8:0/3003054683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1401003b0 msgr2=0x7fa14019dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1401003b0 0x7fa14019dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.907+0000 7fa14530e640 1 -- 172.21.15.8:0/3003054683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138009900 con 0x7fa1400ff9f0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.908+0000 7fa14530e640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 0x7fa14019d750 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9f0 tx=0x7fa13c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.908+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c009720 con 0x7fa1400ff9f0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.908+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa13c00cd80 con 0x7fa1400ff9f0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.908+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa130009490 con 0x7fa1400ff9f0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.908+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c010690 con 0x7fa1400ff9f0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.908+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a4610 con 0x7fa1400ff9f0 2024-02-19T16:24:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.909+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa140102d10 con 0x7fa1400ff9f0 2024-02-19T16:24:19.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.910+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fa13c0046c0 con 0x7fa1400ff9f0 2024-02-19T16:24:19.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.910+0000 7fa1367fc640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2350 0x7fa12c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:19.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.911+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa13c014070 con 0x7fa1400ff9f0 2024-02-19T16:24:19.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.911+0000 7fa144b0d640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2350 0x7fa12c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:19.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.911+0000 7fa144b0d640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2350 0x7fa12c0a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa130037040 tx=0x7fa1300056d0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:19.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:19.915+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fa13c097120 con 0x7fa1400ff9f0 2024-02-19T16:24:20.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.136+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa140068730 con 0x7fa1400ff9f0 2024-02-19T16:24:20.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.137+0000 7fa1367fc640 1 -- 172.21.15.8:0/3003054683 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fa140068730 con 0x7fa1400ff9f0 2024-02-19T16:24:20.138 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:24:20.138 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:24:18.989266+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:24:20.138 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2350 msgr2=0x7fa12c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2350 0x7fa12c0a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa130037040 tx=0x7fa1300056d0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 msgr2=0x7fa14019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 0x7fa14019d750 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9f0 tx=0x7fa13c00edf0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 shutdown_connections 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2350 0x7fa12c0a4810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa140107580 0x7fa140197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1401003b0 0x7fa14019dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 --2- 172.21.15.8:0/3003054683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1400ff9f0 0x7fa14019d750 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.139+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 >> 172.21.15.8:0/3003054683 conn(0x7fa140076980 msgr2=0x7fa1400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.140+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 shutdown_connections 2024-02-19T16:24:20.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.140+0000 7fa147572640 1 -- 172.21.15.8:0/3003054683 wait complete. 2024-02-19T16:24:20.151 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:24:20.151 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:24:20.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 -- 172.21.15.8:0/1117540831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c107560 msgr2=0x7fab9c107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 --2- 172.21.15.8:0/1117540831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c107560 0x7fab9c107940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009900 tx=0x7fab8c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 -- 172.21.15.8:0/1117540831 shutdown_connections 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 --2- 172.21.15.8:0/1117540831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c101e60 0x7fab9c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 --2- 172.21.15.8:0/1117540831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c1014a0 0x7fab9c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 --2- 172.21.15.8:0/1117540831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c107560 0x7fab9c107940 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 -- 172.21.15.8:0/1117540831 >> 172.21.15.8:0/1117540831 conn(0x7fab9c0788f0 msgr2=0x7fab9c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 -- 172.21.15.8:0/1117540831 shutdown_connections 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.256+0000 7faba149a640 1 -- 172.21.15.8:0/1117540831 wait complete. 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 Processor -- start 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 -- start start 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 0x7fab9c0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101e60 0x7fab9c073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c107560 0x7fab9c074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1a5ed0 con 0x7fab9c1014a0 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1a6040 con 0x7fab9c107560 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7faba149a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1a6200 con 0x7fab9c101e60 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7fab9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 0x7fab9c0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7fab9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 0x7fab9c0737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45316/0 (socket says 172.21.15.8:45316) 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7fab9affd640 1 -- 172.21.15.8:0/349287221 learned_addr learned my addr 172.21.15.8:0/349287221 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7fab93fff640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101e60 0x7fab9c073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.257+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c107560 0x7fab9c074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab9affd640 1 -- 172.21.15.8:0/349287221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101e60 msgr2=0x7fab9c073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab9affd640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101e60 0x7fab9c073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab9affd640 1 -- 172.21.15.8:0/349287221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c107560 msgr2=0x7fab9c074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab9affd640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c107560 0x7fab9c074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab9affd640 1 -- 172.21.15.8:0/349287221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c009560 con 0x7fab9c1014a0 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab9affd640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 0x7fab9c0737f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005940 tx=0x7fab8c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab98ff9640 1 -- 172.21.15.8:0/349287221 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0042d0 con 0x7fab9c1014a0 2024-02-19T16:24:20.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c1a6480 con 0x7fab9c1014a0 2024-02-19T16:24:20.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab98ff9640 1 -- 172.21.15.8:0/349287221 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab8c004430 con 0x7fab9c1014a0 2024-02-19T16:24:20.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7fab98ff9640 1 -- 172.21.15.8:0/349287221 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c003c50 con 0x7fab9c1014a0 2024-02-19T16:24:20.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.258+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c1a6990 con 0x7fab9c1014a0 2024-02-19T16:24:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7fab98ff9640 1 -- 172.21.15.8:0/349287221 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fab8c0054f0 con 0x7fab9c1014a0 2024-02-19T16:24:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 Processor -- start 2024-02-19T16:24:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 -- start start 2024-02-19T16:24:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeecd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeed6750 0x560feeed8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feee88960 con 0x560feeecd250 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feee3c310 con 0x560feeed6750 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7faba4073740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feee39840 con 0x560feeecdb70 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeecd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.261+0000 7fab98ff9640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fab880a2350 0x7fab880a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeecd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45322/0 (socket says 172.21.15.8:45322) 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1821059340 learned_addr learned my addr 172.21.15.8:0/1821059340 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab93fff640 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeed6750 0x560feeed8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fab880a2350 0x7fab880a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab98ff9640 1 -- 172.21.15.8:0/349287221 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fab8c0f0730 con 0x7fab9c1014a0 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1821059340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 msgr2=0x560feeed6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1821059340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeed6750 msgr2=0x560feeed8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeed6750 0x560feeed8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1821059340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560feedf7060 con 0x560feeecd250 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeecd630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fab94005c40 tx=0x7fab94013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88db8575e4fe0e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab917fa640 1 -- 172.21.15.8:0/1821059340 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab940134e0 con 0x560feeecd250 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fab880a2350 0x7fab880a4810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab94023000 tx=0x7fab94021000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab917fa640 1 -- 172.21.15.8:0/1821059340 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab94012d00 con 0x560feeecd250 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.262+0000 7fab917fa640 1 -- 172.21.15.8:0/1821059340 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94010d10 con 0x560feeecd250 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 -- 172.21.15.8:0/1821059340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 msgr2=0x560feeecd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeecd630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fab94005c40 tx=0x7fab94013f50 comp rx=0 tx=0).stop 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 -- 172.21.15.8:0/1821059340 shutdown_connections 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeed6750 0x560feeed8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 --2- 172.21.15.8:0/1821059340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeecd630 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 -- 172.21.15.8:0/1821059340 >> 172.21.15.8:0/1821059340 conn(0x560feeec7980 msgr2=0x560feeec7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 -- 172.21.15.8:0/1821059340 shutdown_connections 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 -- 172.21.15.8:0/1821059340 wait complete. 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 -- start start 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.263+0000 7faba4073740 1 Processor -- start 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeed3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeed3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeed41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feee35020 con 0x560feeecd250 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeb08430 con 0x560feeecdb70 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeac4e00 con 0x560feeed6750 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 10 client.0 _ll_register_callbacks cb 0x560feeddccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeed3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeed41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeed3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45330/0 (socket says 172.21.15.8:45330) 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 -- 172.21.15.8:0/1773314512 learned_addr learned my addr 172.21.15.8:0/1773314512 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab93fff640 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeed3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 -- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 msgr2=0x560feeed41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeed41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 -- 172.21.15.8:0/1773314512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 msgr2=0x560feeed3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeed3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 -- 172.21.15.8:0/1773314512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab940136a0 con 0x560feeecd250 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab9affd640 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeed3730 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b9600 tx=0x7fab8c0bc830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e7c4efd0c674e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab827fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab827fc640 1 -- 172.21.15.8:0/1773314512 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0db150 con 0x560feeecd250 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7fab827fc640 1 -- 172.21.15.8:0/1773314512 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fab8c0bcc30 con 0x560feeecd250 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.264+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560feeb347e0 con 0x560feeecd250 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7fab827fc640 1 -- 172.21.15.8:0/1773314512 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab8c0c0d80 con 0x560feeecd250 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 10 client.15984 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560feee336a0 con 0x560feeecd250 2024-02-19T16:24:20.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7fab827fc640 1 -- 172.21.15.8:0/1773314512 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560feeb347e0 con 0x560feeecd250 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7fab827fc640 1 -- 172.21.15.8:0/1773314512 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fab8c03c070 con 0x560feeecd250 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 10 client.15984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 20 client.15984 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feeee6900 0x560feeee8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 4 client.15984 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560feeb71720 con 0x560feeee6900 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.265+0000 7fab93fff640 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feeee6900 0x560feeee8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7fab827fc640 10 client.15984 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7fab93fff640 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feeee6900 0x560feeee8cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a486e02913b47cce server_cookie=aa7a813cf78e4eb1 in_seq=0 out_seq=0 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7fab827fc640 1 -- 172.21.15.8:0/1773314512 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560feeb71720 con 0x560feeee6900 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7fab827fc640 10 client.15984 handle_command_reply: tid=0 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 client.15984 shutdown 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 msgr2=0x560feeed3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeed3730 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b9600 tx=0x7fab8c0bc830 comp rx=0 tx=0).stop 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 shutdown_connections 2024-02-19T16:24:20.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feeee6900 0x560feeee8cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeed41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7fab827fc640 0 client.15984 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeed3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 --2- 172.21.15.8:0/1773314512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeed3730 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 >> 172.21.15.8:0/1773314512 conn(0x560feeec7980 msgr2=0x560feeec96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 shutdown_connections 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.266+0000 7faba4073740 1 -- 172.21.15.8:0/1773314512 wait complete. 2024-02-19T16:24:20.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.267+0000 7faba4073740 20 client.15984 trim_cache size 0 max 16384 2024-02-19T16:24:20.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 Processor -- start 2024-02-19T16:24:20.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 -- start start 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 0x560feeec6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 0x560feeec9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feedba130 con 0x560feeed6750 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeec8780 con 0x560feeecd250 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7faba4073740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeee9370 con 0x560feeecdb70 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 0x560feeec9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7fab9affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 0x560feeec6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7fab9b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 0x560feeec9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45336/0 (socket says 172.21.15.8:45336) 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7fab9affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 0x560feeec6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:54022/0 (socket says 172.21.15.8:54022) 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1940575460 learned_addr learned my addr 172.21.15.8:0/1940575460 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.279+0000 7fab93fff640 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1940575460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 msgr2=0x560feeed3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1940575460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 msgr2=0x560feeec6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 0x560feeec6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9b7fe640 1 -- 172.21.15.8:0/1940575460 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560feedcfc20 con 0x560feeed6750 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9affd640 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 0x560feeec6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 0x560feeec9390 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fab94014fd0 tx=0x7fab94005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90ec4018c953e8a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab90ff9640 1 -- 172.21.15.8:0/1940575460 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9402c040 con 0x560feeed6750 2024-02-19T16:24:20.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab90ff9640 1 -- 172.21.15.8:0/1940575460 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab94012e70 con 0x560feeed6750 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.280+0000 7fab90ff9640 1 -- 172.21.15.8:0/1940575460 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab940122d0 con 0x560feeed6750 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- 172.21.15.8:0/1940575460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 msgr2=0x560feeec9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 0x560feeec9390 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fab94014fd0 tx=0x7fab94005f90 comp rx=0 tx=0).stop 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- 172.21.15.8:0/1940575460 shutdown_connections 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeed6750 0x560feeec9390 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeecdb70 0x560feeed3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- 172.21.15.8:0/1940575460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecd250 0x560feeec6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- 172.21.15.8:0/1940575460 >> 172.21.15.8:0/1940575460 conn(0x560feeec7980 msgr2=0x560feeec8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- 172.21.15.8:0/1940575460 shutdown_connections 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- 172.21.15.8:0/1940575460 wait complete. 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- start start 2024-02-19T16:24:20.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 Processor -- start 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeede170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeeda4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeedaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeedaf70 con 0x560feeecd250 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeedb0e0 con 0x560feeecdb70 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560feeedb2a0 con 0x560feeed6750 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7faba4073740 10 client.0 _ll_register_callbacks cb 0x7fab8c0bcab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7fab9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeede170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7fab9affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeede170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45352/0 (socket says 172.21.15.8:45352) 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7fab9affd640 1 -- 172.21.15.8:0/2402175862 learned_addr learned my addr 172.21.15.8:0/2402175862 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7fab9b7fe640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeedaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.281+0000 7fab93fff640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeeda4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab9affd640 1 -- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 msgr2=0x560feeedaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab9affd640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeedaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab9affd640 1 -- 172.21.15.8:0/2402175862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 msgr2=0x560feeeda4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab9affd640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeeda4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab9affd640 1 -- 172.21.15.8:0/2402175862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560feee336a0 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab9affd640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeede170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92839297ae483cb server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab827fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab827fc640 1 -- 172.21.15.8:0/2402175862 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0ed9a0 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab827fc640 1 -- 172.21.15.8:0/2402175862 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fab8c0db150 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab94012d00 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab827fc640 1 -- 172.21.15.8:0/2402175862 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab8c0b8ed0 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 10 client.15990 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560feedf7060 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab827fc640 1 -- 172.21.15.8:0/2402175862 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab8c0cb080 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7fab827fc640 1 -- 172.21.15.8:0/2402175862 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fab8c03c070 con 0x560feeecd250 2024-02-19T16:24:20.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 10 client.15990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 20 client.15990 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feef43160 0x560feef45550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 4 client.15990 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.282+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560feebc0890 con 0x560feef43160 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7fab93fff640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feef43160 0x560feef45550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7fab93fff640 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feef43160 0x560feef45550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1096161798b0bbf5 server_cookie=d6f6be5e66c0cfce in_seq=0 out_seq=0 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7fab827fc640 10 client.15990 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7fab827fc640 1 -- 172.21.15.8:0/2402175862 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560feebc0890 con 0x560feef43160 2024-02-19T16:24:20.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7fab827fc640 10 client.15990 handle_command_reply: tid=0 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 client.15990 shutdown 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 msgr2=0x560feeede170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeede170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c003660 comp rx=0 tx=0).stop 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 shutdown_connections 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560feef43160 0x560feef45550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560feeed6750 0x560feeedaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7fab827fc640 0 client.15990 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:20.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560feeecdb70 0x560feeeda4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 --2- 172.21.15.8:0/2402175862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560feeecd250 0x560feeede170 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 >> 172.21.15.8:0/2402175862 conn(0x560feeec7980 msgr2=0x560feeec6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 shutdown_connections 2024-02-19T16:24:20.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 1 -- 172.21.15.8:0/2402175862 wait complete. 2024-02-19T16:24:20.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.283+0000 7faba4073740 20 client.15990 trim_cache size 0 max 16384 2024-02-19T16:24:20.299 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:24:20.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fab880a2350 msgr2=0x7fab880a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fab880a2350 0x7fab880a4810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab94023000 tx=0x7fab94021000 comp rx=0 tx=0).stop 2024-02-19T16:24:20.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 msgr2=0x7fab9c0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 0x7fab9c0737f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005940 tx=0x7fab8c02c670 comp rx=0 tx=0).stop 2024-02-19T16:24:20.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 shutdown_connections 2024-02-19T16:24:20.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab9c107560 0x7fab9c074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fab880a2350 0x7fab880a4810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab9c101e60 0x7fab9c073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 --2- 172.21.15.8:0/349287221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab9c1014a0 0x7fab9c0737f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 >> 172.21.15.8:0/349287221 conn(0x7fab9c0788f0 msgr2=0x7fab9c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 shutdown_connections 2024-02-19T16:24:20.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.284+0000 7faba149a640 1 -- 172.21.15.8:0/349287221 wait complete. 2024-02-19T16:24:20.304 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:24:20.304 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:24:20.328 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:20 smithi008 ceph-mon[32573]: pgmap v46: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-02-19T16:24:20.328 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:20 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:20.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:20 smithi156 ceph-mon[36502]: pgmap v46: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-02-19T16:24:20.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:20 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 -- 172.21.15.8:0/2582868345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec06a560 msgr2=0x7f57ec10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 --2- 172.21.15.8:0/2582868345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec06a560 0x7f57ec10e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009ca0 tx=0x7f57dc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 -- 172.21.15.8:0/2582868345 shutdown_connections 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 --2- 172.21.15.8:0/2582868345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec06a560 0x7f57ec10e320 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 --2- 172.21.15.8:0/2582868345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57ec069ba0 0x7f57ec06a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 --2- 172.21.15.8:0/2582868345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ec107560 0x7f57ec107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 -- 172.21.15.8:0/2582868345 >> 172.21.15.8:0/2582868345 conn(0x7f57ec0774c0 msgr2=0x7f57ec0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 -- 172.21.15.8:0/2582868345 shutdown_connections 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.408+0000 7f57f0807640 1 -- 172.21.15.8:0/2582868345 wait complete. 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 Processor -- start 2024-02-19T16:24:20.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 -- start start 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 0x7f57ec19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ec06a560 0x7f57ec19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57ec107560 0x7f57ec197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec197ed0 con 0x7f57ec069ba0 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec198040 con 0x7f57ec06a560 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57f0807640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec198200 con 0x7f57ec107560 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 0x7f57ec19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 0x7f57ec19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45380/0 (socket says 172.21.15.8:45380) 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 -- 172.21.15.8:0/4270606147 learned_addr learned my addr 172.21.15.8:0/4270606147 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57e9d9b640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ec06a560 0x7f57ec19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ead9d640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57ec107560 0x7f57ec197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 -- 172.21.15.8:0/4270606147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57ec107560 msgr2=0x7f57ec197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57ec107560 0x7f57ec197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 -- 172.21.15.8:0/4270606147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ec06a560 msgr2=0x7f57ec19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ec06a560 0x7f57ec19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.409+0000 7f57ea59c640 1 -- 172.21.15.8:0/4270606147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc009900 con 0x7f57ec069ba0 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.410+0000 7f57ea59c640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 0x7f57ec19d6d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e9f0 tx=0x7f57e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.410+0000 7f57e37fe640 1 -- 172.21.15.8:0/4270606147 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4009700 con 0x7f57ec069ba0 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.410+0000 7f57e37fe640 1 -- 172.21.15.8:0/4270606147 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e400cd80 con 0x7f57ec069ba0 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.410+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec1984e0 con 0x7f57ec069ba0 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.410+0000 7f57e37fe640 1 -- 172.21.15.8:0/4270606147 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4018920 con 0x7f57ec069ba0 2024-02-19T16:24:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.410+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec1a4770 con 0x7f57ec069ba0 2024-02-19T16:24:20.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.412+0000 7f57e37fe640 1 -- 172.21.15.8:0/4270606147 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f57e4018a80 con 0x7f57ec069ba0 2024-02-19T16:24:20.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.412+0000 7f57e37fe640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57d80a23d0 0x7f57d80a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.412+0000 7f57f33e7740 1 Processor -- start 2024-02-19T16:24:20.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.412+0000 7f57e9d9b640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57d80a23d0 0x7f57d80a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.412+0000 7f57e37fe640 1 -- 172.21.15.8:0/4270606147 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f57e4014070 con 0x7f57ec069ba0 2024-02-19T16:24:20.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.412+0000 7f57f33e7740 1 -- start start 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 0x55d50e144630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e14d750 0x55d50e14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57f33e7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e0ff960 con 0x55d50e144250 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57f33e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e0b3310 con 0x55d50e14d750 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57f33e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e0b0840 con 0x55d50e144b70 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 0x55d50e144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 0x55d50e144630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45392/0 (socket says 172.21.15.8:45392) 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 -- 172.21.15.8:0/1417676106 learned_addr learned my addr 172.21.15.8:0/1417676106 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ead9d640 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e14d750 0x55d50e14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57e9d9b640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57d80a23d0 0x7f57d80a4890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f57ec106c50 tx=0x7f57d4005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57e9d9b640 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 -- 172.21.15.8:0/1417676106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 msgr2=0x55d50e14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 -- 172.21.15.8:0/1417676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e14d750 msgr2=0x55d50e14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e14d750 0x55d50e14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 -- 172.21.15.8:0/1417676106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50e06e060 con 0x55d50e144250 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ead9d640 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e14d750 0x55d50e14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57ea59c640 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 0x55d50e144630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e9c0 tx=0x7f57e409ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa19f5e195d042 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.413+0000 7f57e0ff9640 1 -- 172.21.15.8:0/1417676106 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4099160 con 0x55d50e144250 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57e0ff9640 1 -- 172.21.15.8:0/1417676106 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e409e070 con 0x55d50e144250 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57e0ff9640 1 -- 172.21.15.8:0/1417676106 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40cbec0 con 0x55d50e144250 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 -- 172.21.15.8:0/1417676106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 msgr2=0x55d50e144630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 0x55d50e144630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e9c0 tx=0x7f57e409ae60 comp rx=0 tx=0).stop 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 -- 172.21.15.8:0/1417676106 shutdown_connections 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e14d750 0x55d50e14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 --2- 172.21.15.8:0/1417676106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e144250 0x55d50e144630 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 -- 172.21.15.8:0/1417676106 >> 172.21.15.8:0/1417676106 conn(0x55d50e13e980 msgr2=0x55d50e13ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 -- 172.21.15.8:0/1417676106 shutdown_connections 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.414+0000 7f57f33e7740 1 -- 172.21.15.8:0/1417676106 wait complete. 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 -- start start 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 Processor -- start 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e14a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e14b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e0ac020 con 0x55d50e14d750 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50dd7f430 con 0x55d50e144250 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50dd3be00 con 0x55d50e144b70 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57f33e7740 10 client.0 _ll_register_callbacks cb 0x55d50e053cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e14b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45404/0 (socket says 172.21.15.8:45404) 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57e9d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 -- 172.21.15.8:0/3163945875 learned_addr learned my addr 172.21.15.8:0/3163945875 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ea59c640 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e14a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 -- 172.21.15.8:0/3163945875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 msgr2=0x55d50e14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 -- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 msgr2=0x55d50e14a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e14a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.415+0000 7f57ead9d640 1 -- 172.21.15.8:0/3163945875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec1a4770 con 0x55d50e14d750 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57ead9d640 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e14b210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f57dc036000 tx=0x7f57dc034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c674864f004268 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57d1ffb640 1 -- 172.21.15.8:0/3163945875 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0328a0 con 0x55d50e14d750 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57d1ffb640 1 -- 172.21.15.8:0/3163945875 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f57dc032a00 con 0x55d50e14d750 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d50ddab7e0 con 0x55d50e14d750 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57d1ffb640 1 -- 172.21.15.8:0/3163945875 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57dc003970 con 0x55d50e14d750 2024-02-19T16:24:20.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 10 client.16002 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d50e0aa6a0 con 0x55d50e14d750 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57d1ffb640 1 -- 172.21.15.8:0/3163945875 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d50ddab7e0 con 0x55d50e14d750 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57d1ffb640 1 -- 172.21.15.8:0/3163945875 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f57dc048480 con 0x55d50e14d750 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 10 client.16002 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 10 client.16002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 20 client.16002 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e15dc40 0x55d50e160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 4 client.16002 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.416+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d50dde8720 con 0x55d50e15dc40 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.417+0000 7f57ea59c640 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e15dc40 0x55d50e160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.417+0000 7f57ea59c640 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e15dc40 0x55d50e160060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7358a04bb1ceb81 server_cookie=9a9202e965b4be17 in_seq=0 out_seq=0 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.417+0000 7f57d1ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.417+0000 7f57d1ffb640 1 -- 172.21.15.8:0/3163945875 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d50dde8720 con 0x55d50e15dc40 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.417+0000 7f57d1ffb640 10 client.16002 handle_command_reply: tid=0 2024-02-19T16:24:20.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.417+0000 7f57f33e7740 1 client.16002 shutdown 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 msgr2=0x55d50e14b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e14b210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f57dc036000 tx=0x7f57dc034670 comp rx=0 tx=0).stop 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 shutdown_connections 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e15dc40 0x55d50e160060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e14b210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e14acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57d1ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 --2- 172.21.15.8:0/3163945875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 >> 172.21.15.8:0/3163945875 conn(0x55d50e13e980 msgr2=0x55d50e1406d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 shutdown_connections 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 1 -- 172.21.15.8:0/3163945875 wait complete. 2024-02-19T16:24:20.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.418+0000 7f57f33e7740 20 client.16002 trim_cache size 0 max 16384 2024-02-19T16:24:20.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 Processor -- start 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 -- start start 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144250 0x55d50e14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144b70 0x55d50e14b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e13da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e031130 con 0x55d50e14d750 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e146410 con 0x55d50e144b70 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57f33e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e13f980 con 0x55d50e144250 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57ead9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e13da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57ead9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e13da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45412/0 (socket says 172.21.15.8:45412) 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57ead9d640 1 -- 172.21.15.8:0/2046426285 learned_addr learned my addr 172.21.15.8:0/2046426285 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57ea59c640 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144250 0x55d50e14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.430+0000 7f57e9d9b640 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144b70 0x55d50e14b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57ead9d640 1 -- 172.21.15.8:0/2046426285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144250 msgr2=0x55d50e14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57ead9d640 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144250 0x55d50e14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57ead9d640 1 -- 172.21.15.8:0/2046426285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144b70 msgr2=0x55d50e14b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57ead9d640 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144b70 0x55d50e14b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57ead9d640 1 -- 172.21.15.8:0/2046426285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50e046c20 con 0x55d50e14d750 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57ead9d640 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e13da00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009ca0 tx=0x7f57dc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0f5f679045e49a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57d3fff640 1 -- 172.21.15.8:0/2046426285 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc048930 con 0x55d50e14d750 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57d3fff640 1 -- 172.21.15.8:0/2046426285 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57dc048a90 con 0x55d50e14d750 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57d3fff640 1 -- 172.21.15.8:0/2046426285 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc004000 con 0x55d50e14d750 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 -- 172.21.15.8:0/2046426285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 msgr2=0x55d50e13da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e13da00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009ca0 tx=0x7f57dc009800 comp rx=0 tx=0).stop 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 -- 172.21.15.8:0/2046426285 shutdown_connections 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e13da00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144b70 0x55d50e14b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 --2- 172.21.15.8:0/2046426285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144250 0x55d50e14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 -- 172.21.15.8:0/2046426285 >> 172.21.15.8:0/2046426285 conn(0x55d50e13e980 msgr2=0x55d50e13d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.431+0000 7f57f33e7740 1 -- 172.21.15.8:0/2046426285 shutdown_connections 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 -- 172.21.15.8:0/2046426285 wait complete. 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 -- start start 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 Processor -- start 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e147c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e1b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e1b2a00 con 0x55d50e14d750 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e1b2b40 con 0x55d50e144250 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e1b2d00 con 0x55d50e144b70 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57f33e7740 10 client.0 _ll_register_callbacks cb 0x7f57d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e1b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e1b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45428/0 (socket says 172.21.15.8:45428) 2024-02-19T16:24:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 -- 172.21.15.8:0/196988817 learned_addr learned my addr 172.21.15.8:0/196988817 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ea59c640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57e9d9b640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 -- 172.21.15.8:0/196988817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 msgr2=0x55d50e147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 -- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 msgr2=0x55d50e147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.432+0000 7f57ead9d640 1 -- 172.21.15.8:0/196988817 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d50e0aa6a0 con 0x55d50e14d750 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57ead9d640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e1b24c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f57dc058040 tx=0x7f57dc048c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3878dcda2d599d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57d1ffb640 1 -- 172.21.15.8:0/196988817 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc048930 con 0x55d50e14d750 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57d1ffb640 1 -- 172.21.15.8:0/196988817 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f57dc003df0 con 0x55d50e14d750 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57e409e070 con 0x55d50e14d750 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57d1ffb640 1 -- 172.21.15.8:0/196988817 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57dc048690 con 0x55d50e14d750 2024-02-19T16:24:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 10 client.16008 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d50e06c8c0 con 0x55d50e14d750 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57d1ffb640 1 -- 172.21.15.8:0/196988817 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e409e070 con 0x55d50e14d750 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57d1ffb640 1 -- 172.21.15.8:0/196988817 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f57dc004670 con 0x55d50e14d750 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 10 client.16008 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 20 client.16008 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e1ba090 0x55d50e1bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 4 client.16008 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d50de37890 con 0x55d50e1ba090 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.433+0000 7f57ea59c640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e1ba090 0x55d50e1bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57ea59c640 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e1ba090 0x55d50e1bc480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69d2822f4f796360 server_cookie=cbf2068bcd52a882 in_seq=0 out_seq=0 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57d1ffb640 10 client.16008 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57d1ffb640 1 -- 172.21.15.8:0/196988817 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d50de37890 con 0x55d50e1ba090 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57d1ffb640 10 client.16008 handle_command_reply: tid=0 2024-02-19T16:24:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 client.16008 shutdown 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 msgr2=0x55d50e1b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e1b24c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f57dc058040 tx=0x7f57dc048c00 comp rx=0 tx=0).stop 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 shutdown_connections 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d50e1ba090 0x55d50e1bc480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d50e14d750 0x55d50e1b24c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57d1ffb640 0 client.16008 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d50e144b70 0x55d50e147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 --2- 172.21.15.8:0/196988817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d50e144250 0x55d50e147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 >> 172.21.15.8:0/196988817 conn(0x55d50e13e980 msgr2=0x55d50e1404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 shutdown_connections 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 1 -- 172.21.15.8:0/196988817 wait complete. 2024-02-19T16:24:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.434+0000 7f57f33e7740 20 client.16008 trim_cache size 0 max 16384 2024-02-19T16:24:20.443 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:24:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57d80a23d0 msgr2=0x7f57d80a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57d80a23d0 0x7f57d80a4890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f57ec106c50 tx=0x7f57d4005f90 comp rx=0 tx=0).stop 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 msgr2=0x7f57ec19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 0x7f57ec19d6d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e9f0 tx=0x7f57e400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 shutdown_connections 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57d80a23d0 0x7f57d80a4890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57ec107560 0x7f57ec197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ec06a560 0x7f57ec19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 --2- 172.21.15.8:0/4270606147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec069ba0 0x7f57ec19d6d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 >> 172.21.15.8:0/4270606147 conn(0x7f57ec0774c0 msgr2=0x7f57ec0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 shutdown_connections 2024-02-19T16:24:20.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.435+0000 7f57f0807640 1 -- 172.21.15.8:0/4270606147 wait complete. 2024-02-19T16:24:20.446 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:24:20.446 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:24:20.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:20 smithi191 ceph-mon[37203]: pgmap v46: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-02-19T16:24:20.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:20 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- 172.21.15.8:0/4247754977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801056a0 msgr2=0x7fce8010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- 172.21.15.8:0/4247754977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801056a0 0x7fce8010d270 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fce78009cd0 tx=0x7fce7802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- 172.21.15.8:0/4247754977 shutdown_connections 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- 172.21.15.8:0/4247754977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801056a0 0x7fce8010d270 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- 172.21.15.8:0/4247754977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce80100f80 0x7fce80104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- 172.21.15.8:0/4247754977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce801005d0 0x7fce801009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- 172.21.15.8:0/4247754977 >> 172.21.15.8:0/4247754977 conn(0x7fce800774f0 msgr2=0x7fce80077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- 172.21.15.8:0/4247754977 shutdown_connections 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- 172.21.15.8:0/4247754977 wait complete. 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 Processor -- start 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- start start 2024-02-19T16:24:20.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce801005d0 0x7fce8019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce801056a0 0x7fce8019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 0x7fce80197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce801982d0 con 0x7fce801978e0 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce80198440 con 0x7fce801005d0 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.591+0000 7fce86b2c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce80198600 con 0x7fce801056a0 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 0x7fce80197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 0x7fce80197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45434/0 (socket says 172.21.15.8:45434) 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 -- 172.21.15.8:0/3581060249 learned_addr learned my addr 172.21.15.8:0/3581060249 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce7ffff640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce801056a0 0x7fce8019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce848c8640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce801005d0 0x7fce8019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 -- 172.21.15.8:0/3581060249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce801056a0 msgr2=0x7fce8019dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce801056a0 0x7fce8019dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 -- 172.21.15.8:0/3581060249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce801005d0 msgr2=0x7fce8019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce801005d0 0x7fce8019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 -- 172.21.15.8:0/3581060249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c009490 con 0x7fce801978e0 2024-02-19T16:24:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce850c9640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 0x7fce80197d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fce78004af0 tx=0x7fce78032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce7dffb640 1 -- 172.21.15.8:0/3581060249 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780328b0 con 0x7fce801978e0 2024-02-19T16:24:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce7dffb640 1 -- 172.21.15.8:0/3581060249 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce78003e50 con 0x7fce801978e0 2024-02-19T16:24:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78009900 con 0x7fce801978e0 2024-02-19T16:24:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce7dffb640 1 -- 172.21.15.8:0/3581060249 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78004740 con 0x7fce801978e0 2024-02-19T16:24:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.592+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce801a48e0 con 0x7fce801978e0 2024-02-19T16:24:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce7dffb640 1 -- 172.21.15.8:0/3581060249 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7fce7800a040 con 0x7fce801978e0 2024-02-19T16:24:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 Processor -- start 2024-02-19T16:24:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 -- start start 2024-02-19T16:24:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 0x5600f6db4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dbd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6dbd750 0x5600f6dbfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6d6f960 con 0x5600f6db4b70 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6d23310 con 0x5600f6dbd750 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce897e1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6d20840 con 0x5600f6db4250 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce7dffb640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fce700a2350 0x7fce700a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce848c8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dbd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 0x5600f6db4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce848c8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dbd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45444/0 (socket says 172.21.15.8:45444) 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 0x5600f6db4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:59582/0 (socket says 172.21.15.8:59582) 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce848c8640 1 -- 172.21.15.8:0/1327345077 learned_addr learned my addr 172.21.15.8:0/1327345077 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6dbd750 0x5600f6dbfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.595+0000 7fce850c9640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fce700a2350 0x7fce700a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7dffb640 1 -- 172.21.15.8:0/3581060249 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fce780f1170 con 0x7fce801978e0 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7ffff640 1 -- 172.21.15.8:0/1327345077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6dbd750 msgr2=0x5600f6dbfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6dbd750 0x5600f6dbfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7ffff640 1 -- 172.21.15.8:0/1327345077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 msgr2=0x5600f6dbd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dbd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7ffff640 1 -- 172.21.15.8:0/1327345077 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600f6cde060 con 0x5600f6db4250 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 0x5600f6db4630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce801990d0 tx=0x7fce6c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af5408d4aee754d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce850c9640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fce700a2350 0x7fce700a4810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce780ee500 tx=0x7fce780a8040 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce6affd640 1 -- 172.21.15.8:0/1327345077 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c005020 con 0x5600f6db4250 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce6affd640 1 -- 172.21.15.8:0/1327345077 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce6c005660 con 0x5600f6db4250 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.596+0000 7fce6affd640 1 -- 172.21.15.8:0/1327345077 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00f9f0 con 0x5600f6db4250 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 -- 172.21.15.8:0/1327345077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 msgr2=0x5600f6db4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 0x5600f6db4630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce801990d0 tx=0x7fce6c004730 comp rx=0 tx=0).stop 2024-02-19T16:24:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 -- 172.21.15.8:0/1327345077 shutdown_connections 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6dbd750 0x5600f6dbfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dbd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 --2- 172.21.15.8:0/1327345077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6db4250 0x5600f6db4630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 -- 172.21.15.8:0/1327345077 >> 172.21.15.8:0/1327345077 conn(0x5600f6dae980 msgr2=0x5600f6daed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 -- 172.21.15.8:0/1327345077 shutdown_connections 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 -- 172.21.15.8:0/1327345077 wait complete. 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 -- start start 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.597+0000 7fce897e1740 1 Processor -- start 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dbad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dbb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6d1c020 con 0x5600f6db4250 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f69ef430 con 0x5600f6db4b70 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f69abe00 con 0x5600f6dbd750 2024-02-19T16:24:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 10 client.0 _ll_register_callbacks cb 0x5600f6cc3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45446/0 (socket says 172.21.15.8:45446) 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 -- 172.21.15.8:0/507872069 learned_addr learned my addr 172.21.15.8:0/507872069 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce848c8640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dbad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dbb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 -- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 msgr2=0x5600f6dbb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dbb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 -- 172.21.15.8:0/507872069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 msgr2=0x5600f6dbad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dbad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 -- 172.21.15.8:0/507872069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c019040 con 0x5600f6db4250 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce848c8640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dbad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce7ffff640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dba810 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fce6c006010 tx=0x7fce6c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dc282b123567a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce5ffff640 1 -- 172.21.15.8:0/507872069 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c026070 con 0x5600f6db4250 2024-02-19T16:24:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce5ffff640 1 -- 172.21.15.8:0/507872069 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fce6c005350 con 0x5600f6db4250 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.598+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600f6a1b7e0 con 0x5600f6db4250 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce5ffff640 1 -- 172.21.15.8:0/507872069 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c0047a0 con 0x5600f6db4250 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5600f6d1a6a0 con 0x5600f6db4250 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce5ffff640 1 -- 172.21.15.8:0/507872069 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5600f6a1b7e0 con 0x5600f6db4250 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce5ffff640 1 -- 172.21.15.8:0/507872069 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fce6c00fd30 con 0x5600f6db4250 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 20 client.16020 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6dcdaa0 0x5600f6dcfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 4 client.16020 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600f6a58720 con 0x5600f6dcdaa0 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.599+0000 7fce848c8640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6dcdaa0 0x5600f6dcfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.600+0000 7fce5ffff640 10 client.16020 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.600+0000 7fce848c8640 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6dcdaa0 0x5600f6dcfec0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce84b8379cced79 server_cookie=bef83f8e3c4b1bc5 in_seq=0 out_seq=0 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce5ffff640 1 -- 172.21.15.8:0/507872069 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5600f6a58720 con 0x5600f6dcdaa0 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce5ffff640 10 client.16020 handle_command_reply: tid=0 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 client.16020 shutdown 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 msgr2=0x5600f6dba810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dba810 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fce6c006010 tx=0x7fce6c0093f0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 shutdown_connections 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6dcdaa0 0x5600f6dcfec0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dbb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dbad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce5ffff640 0 client.16020 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 --2- 172.21.15.8:0/507872069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dba810 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 >> 172.21.15.8:0/507872069 conn(0x5600f6dae980 msgr2=0x5600f6db06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 shutdown_connections 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 1 -- 172.21.15.8:0/507872069 wait complete. 2024-02-19T16:24:20.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.601+0000 7fce897e1740 20 client.16020 trim_cache size 0 max 16384 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.613+0000 7fce897e1740 1 Processor -- start 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.613+0000 7fce897e1740 1 -- start start 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce897e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4250 0x5600f6dbcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce897e1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce897e1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dc8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce897e1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6ca1130 con 0x5600f6db4b70 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce897e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6db0cb0 con 0x5600f6db4250 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce897e1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6daa5c0 con 0x5600f6dbd750 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45460/0 (socket says 172.21.15.8:45460) 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 -- 172.21.15.8:0/3762667608 learned_addr learned my addr 172.21.15.8:0/3762667608 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce7ffff640 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4250 0x5600f6dbcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce7ffff640 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dc8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 -- 172.21.15.8:0/3762667608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 msgr2=0x5600f6dc8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dc8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 -- 172.21.15.8:0/3762667608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4250 msgr2=0x5600f6dbcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4250 0x5600f6dbcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 -- 172.21.15.8:0/3762667608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600f6cb6c20 con 0x5600f6db4b70 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce7ffff640 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4250 0x5600f6dbcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce848c8640 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dada00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5600f6dbbde0 tx=0x7fce74005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665653254181a8e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce6a7fc640 1 -- 172.21.15.8:0/3762667608 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74017040 con 0x5600f6db4b70 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce6a7fc640 1 -- 172.21.15.8:0/3762667608 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce74018040 con 0x5600f6db4b70 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.614+0000 7fce6a7fc640 1 -- 172.21.15.8:0/3762667608 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740035f0 con 0x5600f6db4b70 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 -- 172.21.15.8:0/3762667608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 msgr2=0x5600f6dada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dada00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5600f6dbbde0 tx=0x7fce74005f90 comp rx=0 tx=0).stop 2024-02-19T16:24:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 -- 172.21.15.8:0/3762667608 shutdown_connections 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6dc8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4b70 0x5600f6dada00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 --2- 172.21.15.8:0/3762667608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4250 0x5600f6dbcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 -- 172.21.15.8:0/3762667608 >> 172.21.15.8:0/3762667608 conn(0x5600f6dae980 msgr2=0x5600f6dad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 -- 172.21.15.8:0/3762667608 shutdown_connections 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 -- 172.21.15.8:0/3762667608 wait complete. 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 -- start start 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.615+0000 7fce897e1740 1 Processor -- start 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dc1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dc2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6e29320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6dc2c00 con 0x5600f6db4250 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6e29860 con 0x5600f6db4b70 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6e299a0 con 0x5600f6dbd750 2024-02-19T16:24:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 10 client.0 _ll_register_callbacks cb 0x7fce74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dc1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dc1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45466/0 (socket says 172.21.15.8:45466) 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 -- 172.21.15.8:0/1979067565 learned_addr learned my addr 172.21.15.8:0/1979067565 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce848c8640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dc2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6e29320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 -- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 msgr2=0x5600f6e29320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6e29320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 -- 172.21.15.8:0/1979067565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 msgr2=0x5600f6dc2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dc2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 -- 172.21.15.8:0/1979067565 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce74012040 con 0x5600f6db4250 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce848c8640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dc2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce7ffff640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dc1e10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fce6c003640 tx=0x7fce6c02f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2250cf64c57d070c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce5ffff640 1 -- 172.21.15.8:0/1979067565 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c026040 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce5ffff640 1 -- 172.21.15.8:0/1979067565 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fce6c000e40 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce6c005660 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce5ffff640 1 -- 172.21.15.8:0/1979067565 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c003c40 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 10 client.16026 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.616+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5600f6cdc8c0 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce5ffff640 1 -- 172.21.15.8:0/1979067565 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce6c005660 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce5ffff640 1 -- 172.21.15.8:0/1979067565 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fce6c011140 con 0x5600f6db4250 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce897e1740 10 client.16026 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce897e1740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce897e1740 20 client.16026 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce897e1740 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6e2ad00 0x5600f6e2d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce897e1740 4 client.16026 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5600f6aa7890 con 0x5600f6e2ad00 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce848c8640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6e2ad00 0x5600f6e2d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce848c8640 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6e2ad00 0x5600f6e2d0f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9b1e6f4f6d3e1f server_cookie=dd2a09a2f118a577 in_seq=0 out_seq=0 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.617+0000 7fce5ffff640 10 client.16026 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce5ffff640 1 -- 172.21.15.8:0/1979067565 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5600f6aa7890 con 0x5600f6e2ad00 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce5ffff640 10 client.16026 handle_command_reply: tid=0 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 client.16026 shutdown 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 msgr2=0x5600f6dc1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dc1e10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fce6c003640 tx=0x7fce6c02f040 comp rx=0 tx=0).stop 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 shutdown_connections 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5600f6e2ad00 0x5600f6e2d0f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5600f6dbd750 0x5600f6e29320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600f6db4b70 0x5600f6dc2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce5ffff640 0 client.16026 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 --2- 172.21.15.8:0/1979067565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600f6db4250 0x5600f6dc1e10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 >> 172.21.15.8:0/1979067565 conn(0x5600f6dae980 msgr2=0x5600f6dadb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 shutdown_connections 2024-02-19T16:24:20.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 1 -- 172.21.15.8:0/1979067565 wait complete. 2024-02-19T16:24:20.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce897e1740 20 client.16026 trim_cache size 0 max 16384 2024-02-19T16:24:20.627 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fce700a2350 msgr2=0x7fce700a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fce700a2350 0x7fce700a4810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce780ee500 tx=0x7fce780a8040 comp rx=0 tx=0).stop 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 msgr2=0x7fce80197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 0x7fce80197d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fce78004af0 tx=0x7fce78032570 comp rx=0 tx=0).stop 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 shutdown_connections 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce801978e0 0x7fce80197d90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fce700a2350 0x7fce700a4810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce801056a0 0x7fce8019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.618+0000 7fce86b2c640 1 --2- 172.21.15.8:0/3581060249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce801005d0 0x7fce8019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:20.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.619+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 >> 172.21.15.8:0/3581060249 conn(0x7fce800774f0 msgr2=0x7fce800688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:20.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.619+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 shutdown_connections 2024-02-19T16:24:20.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:20.619+0000 7fce86b2c640 1 -- 172.21.15.8:0/3581060249 wait complete. 2024-02-19T16:24:21.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:21 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3003054683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:24:21.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:21 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3003054683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:24:21.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:21 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3003054683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:24:21.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:24:21.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:24:21.630 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:24:21.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/3927968265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 msgr2=0x7f3f9c107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/3927968265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c107960 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009900 tx=0x7f3f9002bc30 comp rx=0 tx=0).stop 2024-02-19T16:24:21.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/3927968265 shutdown_connections 2024-02-19T16:24:21.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/3927968265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c1003b0 0x7f3f9c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/3927968265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f9c0ff9f0 0x7f3f9c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/3927968265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c107960 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/3927968265 >> 172.21.15.8:0/3927968265 conn(0x7f3f9c0769b0 msgr2=0x7f3f9c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/3927968265 shutdown_connections 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.751+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/3927968265 wait complete. 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 Processor -- start 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 -- start start 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c0ff9f0 0x7f3f9c100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f9c1003b0 0x7f3f9c101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c1a5e90 con 0x7f3f9c107580 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c1a6000 con 0x7f3f9c0ff9f0 2024-02-19T16:24:21.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa2f8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c1a61c0 con 0x7f3f9c1003b0 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45476/0 (socket says 172.21.15.8:45476) 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 -- 172.21.15.8:0/2534835348 learned_addr learned my addr 172.21.15.8:0/2534835348 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c0ff9f0 0x7f3f9c100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f9c1003b0 0x7f3f9c101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 -- 172.21.15.8:0/2534835348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f9c1003b0 msgr2=0x7f3f9c101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f9c1003b0 0x7f3f9c101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 -- 172.21.15.8:0/2534835348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c0ff9f0 msgr2=0x7f3f9c100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c0ff9f0 0x7f3f9c100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 -- 172.21.15.8:0/2534835348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f90009560 con 0x7f3f9c107580 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.752+0000 7f3fa1529640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c101650 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f9400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.753+0000 7f3f99ffb640 1 -- 172.21.15.8:0/2534835348 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400bea0 con 0x7f3f9c107580 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.753+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9c1a64a0 con 0x7f3f9c107580 2024-02-19T16:24:21.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.753+0000 7f3f99ffb640 1 -- 172.21.15.8:0/2534835348 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f94009940 con 0x7f3f9c107580 2024-02-19T16:24:21.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.753+0000 7f3f99ffb640 1 -- 172.21.15.8:0/2534835348 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940106c0 con 0x7f3f9c107580 2024-02-19T16:24:21.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.753+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c1a6960 con 0x7f3f9c107580 2024-02-19T16:24:21.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.755+0000 7f3f99ffb640 1 -- 172.21.15.8:0/2534835348 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7f3f94004240 con 0x7f3f9c107580 2024-02-19T16:24:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.755+0000 7f3fa5b6e740 1 Processor -- start 2024-02-19T16:24:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 -- start start 2024-02-19T16:24:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3f99ffb640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3f8c0a2350 0x7f3f8c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4384f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b438582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b4385ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b437bdf90 con 0x558b438588b0 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b437bb4c0 con 0x558b4384fd30 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b437b6ca0 con 0x558b4384f380 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3f8c0a2350 0x7f3f8c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3f99ffb640 1 -- 172.21.15.8:0/2534835348 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3f940ce390 con 0x7f3f9c107580 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4384f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4384f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:59644/0 (socket says 172.21.15.8:59644) 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 -- 172.21.15.8:0/1548656284 learned_addr learned my addr 172.21.15.8:0/1548656284 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b438582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa1529640 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b4385ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 -- 172.21.15.8:0/1548656284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 msgr2=0x558b438582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b438582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 -- 172.21.15.8:0/1548656284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 msgr2=0x558b4385ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b4385ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 -- 172.21.15.8:0/1548656284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b437790b0 con 0x558b4384f380 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3f8c0a2350 0x7f3f8c0a4810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009710 tx=0x7f3f88009af0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.756+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4384f760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009900 tx=0x7f3f9002c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e57184096970909 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3f86ffd640 1 -- 172.21.15.8:0/1548656284 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9002c990 con 0x558b4384f380 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3f86ffd640 1 -- 172.21.15.8:0/1548656284 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f9002caf0 con 0x558b4384f380 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3f86ffd640 1 -- 172.21.15.8:0/1548656284 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90005120 con 0x558b4384f380 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/1548656284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 msgr2=0x558b4384f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4384f760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009900 tx=0x7f3f9002c750 comp rx=0 tx=0).stop 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/1548656284 shutdown_connections 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b4385ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b438582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/1548656284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4384f760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.757+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/1548656284 >> 172.21.15.8:0/1548656284 conn(0x558b43849840 msgr2=0x558b43849c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/1548656284 shutdown_connections 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/1548656284 wait complete. 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 -- start start 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 Processor -- start 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b438557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b43855ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b43856220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4375bf40 con 0x558b438588b0 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4348a4b0 con 0x558b4384fd30 2024-02-19T16:24:21.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4373c670 con 0x558b4384f380 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa5b6e740 10 client.0 _ll_register_callbacks cb 0x558b437a9f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa1529640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b43856220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.758+0000 7f3fa0d28640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b438557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b438557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:59650/0 (socket says 172.21.15.8:59650) 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 -- 172.21.15.8:0/278151405 learned_addr learned my addr 172.21.15.8:0/278151405 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b43855ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 -- 172.21.15.8:0/278151405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 msgr2=0x558b43855ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b43855ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 -- 172.21.15.8:0/278151405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 msgr2=0x558b43856220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b43856220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 -- 172.21.15.8:0/278151405 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c10ec40 con 0x558b4384f380 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa1529640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b43856220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b438557a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f9002c170 tx=0x7f3f90004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a0747087fb060e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3f7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3f7bfff640 1 -- 172.21.15.8:0/278151405 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90004520 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b43790f40 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3f7bfff640 1 -- 172.21.15.8:0/278151405 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3f90005070 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3f7bfff640 1 -- 172.21.15.8:0/278151405 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f90005230 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa5b6e740 10 client.25906 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.759+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558b437b5320 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3f7bfff640 1 -- 172.21.15.8:0/278151405 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b43790f40 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3f7bfff640 1 -- 172.21.15.8:0/278151405 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3f9003c050 con 0x558b4384f380 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3fa5b6e740 10 client.25906 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:21.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3fa5b6e740 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3fa5b6e740 20 client.25906 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438689b0 0x558b4386add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3fa5b6e740 4 client.25906 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b437ad310 con 0x558b438689b0 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438689b0 0x558b4386add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438689b0 0x558b4386add0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a08346b793250961 server_cookie=c553f361ffd2db3a in_seq=0 out_seq=0 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.760+0000 7f3f7bfff640 10 client.25906 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3f7bfff640 1 -- 172.21.15.8:0/278151405 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558b437ad310 con 0x558b438689b0 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3f7bfff640 10 client.25906 handle_command_reply: tid=0 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 client.25906 shutdown 2024-02-19T16:24:21.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 msgr2=0x558b438557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b438557a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f9002c170 tx=0x7f3f90004970 comp rx=0 tx=0).stop 2024-02-19T16:24:21.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 shutdown_connections 2024-02-19T16:24:21.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b438588b0 0x558b43856220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b4384fd30 0x558b43855ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438689b0 0x558b4386add0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/278151405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b438557a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 >> 172.21.15.8:0/278151405 conn(0x558b43849840 msgr2=0x558b4384b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3f7bfff640 0 client.25906 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 shutdown_connections 2024-02-19T16:24:21.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/278151405 wait complete. 2024-02-19T16:24:21.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.761+0000 7f3fa5b6e740 20 client.25906 trim_cache size 0 max 16384 2024-02-19T16:24:21.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 Processor -- start 2024-02-19T16:24:21.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 -- start start 2024-02-19T16:24:21.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b43857ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b43856040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b43848760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b43845180 con 0x558b4384fd30 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4385b0e0 con 0x558b438588b0 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4385aca0 con 0x558b4384f380 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b43856040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b43856040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45498/0 (socket says 172.21.15.8:45498) 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 -- 172.21.15.8:0/940539766 learned_addr learned my addr 172.21.15.8:0/940539766 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b43857ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa1529640 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b43848760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 -- 172.21.15.8:0/940539766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 msgr2=0x558b43857ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b43857ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 -- 172.21.15.8:0/940539766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 msgr2=0x558b43848760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b43848760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3f9bfff640 1 -- 172.21.15.8:0/940539766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b43751c00 con 0x558b4384fd30 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.778+0000 7f3fa1529640 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b43848760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b43856040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x558b43850130 tx=0x7f3f88005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e553d752317a7b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3f867fc640 1 -- 172.21.15.8:0/940539766 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88026070 con 0x558b4384fd30 2024-02-19T16:24:21.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3f867fc640 1 -- 172.21.15.8:0/940539766 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f880044c0 con 0x558b4384fd30 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3f867fc640 1 -- 172.21.15.8:0/940539766 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88003e20 con 0x558b4384fd30 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/940539766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 msgr2=0x558b43856040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b43856040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x558b43850130 tx=0x7f3f88005f90 comp rx=0 tx=0).stop 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/940539766 shutdown_connections 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b43848760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b43856040 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/940539766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b43857ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/940539766 >> 172.21.15.8:0/940539766 conn(0x558b43849840 msgr2=0x558b43848050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/940539766 shutdown_connections 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.779+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/940539766 wait complete. 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 -- start start 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 Processor -- start 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4385c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b4385c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b438c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4385ccf0 con 0x558b4384fd30 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4385ce60 con 0x558b438588b0 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b438c37e0 con 0x558b4384f380 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa5b6e740 10 client.0 _ll_register_callbacks cb 0x7f3f940963b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3f9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b4385c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3f9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b4385c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45510/0 (socket says 172.21.15.8:45510) 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3f9bfff640 1 -- 172.21.15.8:0/268914904 learned_addr learned my addr 172.21.15.8:0/268914904 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4385c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3fa1529640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b438c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.780+0000 7f3f9bfff640 1 -- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 msgr2=0x558b4385c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4385c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f9bfff640 1 -- 172.21.15.8:0/268914904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 msgr2=0x558b438c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b438c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f9bfff640 1 -- 172.21.15.8:0/268914904 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8801b040 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f9bfff640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b4385c550 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3f88013280 tx=0x7f3f88009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0d74b7dea04064 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f7bfff640 1 -- 172.21.15.8:0/268914904 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88026070 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b437dc900 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f7bfff640 1 -- 172.21.15.8:0/268914904 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3f8800fde0 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f7bfff640 1 -- 172.21.15.8:0/268914904 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f88003cc0 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3fa5b6e740 10 client.16044 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558b43777910 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f7bfff640 1 -- 172.21.15.8:0/268914904 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b437dc900 con 0x558b4384fd30 2024-02-19T16:24:21.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3f7bfff640 1 -- 172.21.15.8:0/268914904 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3f880046c0 con 0x558b4384fd30 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3fa5b6e740 10 client.16044 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3fa5b6e740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.781+0000 7f3fa5b6e740 20 client.16044 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.782+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438c50b0 0x558b438c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.782+0000 7f3fa5b6e740 4 client.16044 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.782+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558b434b6860 con 0x558b438c50b0 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.782+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438c50b0 0x558b438c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.782+0000 7f3fa0d28640 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438c50b0 0x558b438c74d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de0cfb18d45344a server_cookie=1f6da559b14f3be8 in_seq=0 out_seq=0 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.782+0000 7f3f7bfff640 10 client.16044 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3f7bfff640 1 -- 172.21.15.8:0/268914904 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558b434b6860 con 0x558b438c50b0 2024-02-19T16:24:21.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3f7bfff640 10 client.16044 handle_command_reply: tid=0 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 client.16044 shutdown 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 msgr2=0x558b4385c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b4385c550 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3f88013280 tx=0x7f3f88009f10 comp rx=0 tx=0).stop 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 shutdown_connections 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b438588b0 0x558b438c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4384fd30 0x558b4385c550 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x558b438c50b0 0x558b438c74d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 --2- 172.21.15.8:0/268914904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b4384f380 0x558b4385c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3f7bfff640 0 client.16044 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 >> 172.21.15.8:0/268914904 conn(0x558b43849840 msgr2=0x558b4386f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 shutdown_connections 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 1 -- 172.21.15.8:0/268914904 wait complete. 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.783+0000 7f3fa5b6e740 20 client.16044 trim_cache size 0 max 16384 2024-02-19T16:24:21.796 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:24:21.797 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:24:21.797 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "4ca98691-9369-43ff-a3db-628fdb9f3011", 2024-02-19T16:24:21.797 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:24:21.797 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:24:21.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3f8c0a2350 msgr2=0x7f3f8c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3f8c0a2350 0x7f3f8c0a4810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009710 tx=0x7f3f88009af0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 msgr2=0x7f3f9c101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c101650 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f9400b260 comp rx=0 tx=0).stop 2024-02-19T16:24:21.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 shutdown_connections 2024-02-19T16:24:21.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3f9c107580 0x7f3f9c101650 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3f8c0a2350 0x7f3f8c0a4810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f9c1003b0 0x7f3f9c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 --2- 172.21.15.8:0/2534835348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c0ff9f0 0x7f3f9c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 >> 172.21.15.8:0/2534835348 conn(0x7f3f9c0769b0 msgr2=0x7f3f9c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 shutdown_connections 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.784+0000 7f3fa2f8c640 1 -- 172.21.15.8:0/2534835348 wait complete. 2024-02-19T16:24:21.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:24:21.799 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 -- 172.21.15.8:0/2104463817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb74100760 msgr2=0x7feb741047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 --2- 172.21.15.8:0/2104463817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb74100760 0x7feb741047a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feb6400aa40 tx=0x7feb6402cf50 comp rx=0 tx=0).stop 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 -- 172.21.15.8:0/2104463817 shutdown_connections 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 --2- 172.21.15.8:0/2104463817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb74104e80 0x7feb7410ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 --2- 172.21.15.8:0/2104463817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb74100760 0x7feb741047a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 --2- 172.21.15.8:0/2104463817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb740ffdb0 0x7feb74100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 -- 172.21.15.8:0/2104463817 >> 172.21.15.8:0/2104463817 conn(0x7feb74076910 msgr2=0x7feb740fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 -- 172.21.15.8:0/2104463817 shutdown_connections 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 -- 172.21.15.8:0/2104463817 wait complete. 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 Processor -- start 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.907+0000 7feb7a308640 1 -- start start 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb7a308640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb740ffdb0 0x7feb7419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb7a308640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb74104e80 0x7feb7419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb7a308640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 0x7feb74197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb7a308640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb741983a0 con 0x7feb741979b0 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb7a308640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74198510 con 0x7feb740ffdb0 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb7a308640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb741986d0 con 0x7feb74104e80 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 0x7feb74197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 0x7feb74197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45530/0 (socket says 172.21.15.8:45530) 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 -- 172.21.15.8:0/387440196 learned_addr learned my addr 172.21.15.8:0/387440196 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb737fe640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb74104e80 0x7feb7419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb73fff640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb740ffdb0 0x7feb7419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 -- 172.21.15.8:0/387440196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb74104e80 msgr2=0x7feb7419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb74104e80 0x7feb7419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 -- 172.21.15.8:0/387440196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb740ffdb0 msgr2=0x7feb7419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb740ffdb0 0x7feb7419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 -- 172.21.15.8:0/387440196 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6400a670 con 0x7feb741979b0 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.908+0000 7feb788a5640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 0x7feb74197e60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00eb10 tx=0x7feb6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.909+0000 7feb717fa640 1 -- 172.21.15.8:0/387440196 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004b20 con 0x7feb741979b0 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.909+0000 7feb717fa640 1 -- 172.21.15.8:0/387440196 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb6c004c80 con 0x7feb741979b0 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.909+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb741a4670 con 0x7feb741979b0 2024-02-19T16:24:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.909+0000 7feb717fa640 1 -- 172.21.15.8:0/387440196 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c010650 con 0x7feb741979b0 2024-02-19T16:24:21.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.909+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb741a4b00 con 0x7feb741979b0 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb717fa640 1 -- 172.21.15.8:0/387440196 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 137000+0+0 (secure 0 0 0) 0x7feb6c005020 con 0x7feb741979b0 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 Processor -- start 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb717fa640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7feb5c0a2350 0x7feb5c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 -- start start 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb737fe640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7feb5c0a2350 0x7feb5c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb717fa640 1 -- 172.21.15.8:0/387440196 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7feb6c014070 con 0x7feb741979b0 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d45524280 con 0x563d455b5bd0 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d455217b0 con 0x563d455b52b0 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.911+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4551cf90 con 0x563d455be7b0 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455be270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45536/0 (socket says 172.21.15.8:45536) 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb737fe640 1 -- 172.21.15.8:0/906797818 learned_addr learned my addr 172.21.15.8:0/906797818 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb788a5640 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb737fe640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7feb5c0a2350 0x7feb5c0a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb64005ab0 tx=0x7feb640356b0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 -- 172.21.15.8:0/906797818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 msgr2=0x563d455c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 -- 172.21.15.8:0/906797818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 msgr2=0x563d455be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 -- 172.21.15.8:0/906797818 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d454db570 con 0x563d455b52b0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb737fe640 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb73fff640 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b5690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feb68009bf0 tx=0x7feb68011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c231d83fa4268da server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb5a7fc640 1 -- 172.21.15.8:0/906797818 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004310 con 0x563d455b52b0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb5a7fc640 1 -- 172.21.15.8:0/906797818 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb68004470 con 0x563d455b52b0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.912+0000 7feb5a7fc640 1 -- 172.21.15.8:0/906797818 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb680035f0 con 0x563d455b52b0 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 -- 172.21.15.8:0/906797818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 msgr2=0x563d455b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b5690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feb68009bf0 tx=0x7feb68011440 comp rx=0 tx=0).stop 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 -- 172.21.15.8:0/906797818 shutdown_connections 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/906797818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b5690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 -- 172.21.15.8:0/906797818 >> 172.21.15.8:0/906797818 conn(0x563d455af990 msgr2=0x563d455afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 -- 172.21.15.8:0/906797818 shutdown_connections 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 -- 172.21.15.8:0/906797818 wait complete. 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.913+0000 7feb7cfb6740 1 -- start start 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 Processor -- start 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 0x563d455be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b5bd0 0x563d455bb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d451f0430 con 0x563d455b52b0 2024-02-19T16:24:21.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d451ace00 con 0x563d455be7b0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d454a1fd0 con 0x563d455b5bd0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb7cfb6740 10 client.0 _ll_register_callbacks cb 0x563d454f7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 0x563d455be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 0x563d455be1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45546/0 (socket says 172.21.15.8:45546) 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 -- 172.21.15.8:0/2413738466 learned_addr learned my addr 172.21.15.8:0/2413738466 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb737fe640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b5bd0 0x563d455bb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb788a5640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 -- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b5bd0 msgr2=0x563d455bb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b5bd0 0x563d455bb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 -- 172.21.15.8:0/2413738466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 msgr2=0x563d455bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455bbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 -- 172.21.15.8:0/2413738466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb68009850 con 0x563d455b52b0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb73fff640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 0x563d455be1f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feb68004cd0 tx=0x7feb68004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae082e487e0f3a29 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.914+0000 7feb537fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb537fe640 1 -- 172.21.15.8:0/2413738466 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb680037f0 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d4521c7e0 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb537fe640 1 -- 172.21.15.8:0/2413738466 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7feb68003950 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb537fe640 1 -- 172.21.15.8:0/2413738466 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb68003b10 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 10 client.16056 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d4551b610 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb537fe640 1 -- 172.21.15.8:0/2413738466 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563d4521c7e0 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb537fe640 1 -- 172.21.15.8:0/2413738466 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feb68024ca0 con 0x563d455b52b0 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 10 client.16056 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 20 client.16056 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d455ce8e0 0x563d455d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 4 client.16056 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d45259720 con 0x563d455ce8e0 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.915+0000 7feb737fe640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d455ce8e0 0x563d455d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb537fe640 10 client.16056 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb737fe640 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d455ce8e0 0x563d455d0d00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216f963c4422c6f0 server_cookie=4a7acceade5ccfed in_seq=0 out_seq=0 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb537fe640 1 -- 172.21.15.8:0/2413738466 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563d45259720 con 0x563d455ce8e0 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb537fe640 10 client.16056 handle_command_reply: tid=0 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb7cfb6740 1 client.16056 shutdown 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 msgr2=0x563d455be1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.916+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 0x563d455be1f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feb68004cd0 tx=0x7feb68004740 comp rx=0 tx=0).stop 2024-02-19T16:24:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 shutdown_connections 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455bbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d455ce8e0 0x563d455d0d00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b5bd0 0x563d455bb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/2413738466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b52b0 0x563d455be1f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 >> 172.21.15.8:0/2413738466 conn(0x563d455af990 msgr2=0x563d455b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb537fe640 0 client.16056 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 shutdown_connections 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 1 -- 172.21.15.8:0/2413738466 wait complete. 2024-02-19T16:24:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.917+0000 7feb7cfb6740 20 client.16056 trim_cache size 0 max 16384 2024-02-19T16:24:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 Processor -- start 2024-02-19T16:24:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 -- start start 2024-02-19T16:24:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d455c1170 con 0x563d455b5bd0 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d455ce6e0 con 0x563d455b52b0 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d455af200 con 0x563d455be7b0 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455bdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45556/0 (socket says 172.21.15.8:45556) 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 -- 172.21.15.8:0/4262361435 learned_addr learned my addr 172.21.15.8:0/4262361435 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb788a5640 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb73fff640 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 -- 172.21.15.8:0/4262361435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 msgr2=0x563d455be1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455be1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 -- 172.21.15.8:0/4262361435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 msgr2=0x563d455b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 -- 172.21.15.8:0/4262361435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d454b7ac0 con 0x563d455b5bd0 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb73fff640 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb737fe640 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455bdcb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb6402d490 tx=0x7feb6402f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddd84e5fd0a7cc5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.933+0000 7feb59ffb640 1 -- 172.21.15.8:0/4262361435 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64004ac0 con 0x563d455b5bd0 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb59ffb640 1 -- 172.21.15.8:0/4262361435 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb64035e90 con 0x563d455b5bd0 2024-02-19T16:24:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb59ffb640 1 -- 172.21.15.8:0/4262361435 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640595a0 con 0x563d455b5bd0 2024-02-19T16:24:21.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 -- 172.21.15.8:0/4262361435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 msgr2=0x563d455bdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455bdcb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb6402d490 tx=0x7feb6402f050 comp rx=0 tx=0).stop 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 -- 172.21.15.8:0/4262361435 shutdown_connections 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455be7b0 0x563d455be1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455bdcb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/4262361435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455b52b0 0x563d455b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 -- 172.21.15.8:0/4262361435 >> 172.21.15.8:0/4262361435 conn(0x563d455af990 msgr2=0x563d455b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 -- 172.21.15.8:0/4262361435 shutdown_connections 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 -- 172.21.15.8:0/4262361435 wait complete. 2024-02-19T16:24:21.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.934+0000 7feb7cfb6740 1 -- start start 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 Processor -- start 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b52b0 0x563d455c1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d455b9540 con 0x563d455b5bd0 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d455b96b0 con 0x563d455be7b0 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4562e660 con 0x563d455b52b0 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 10 client.0 _ll_register_callbacks cb 0x7feb6c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455c1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45560/0 (socket says 172.21.15.8:45560) 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 -- 172.21.15.8:0/1820526320 learned_addr learned my addr 172.21.15.8:0/1820526320 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb73fff640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b52b0 0x563d455c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb788a5640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 -- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b52b0 msgr2=0x563d455c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b52b0 0x563d455c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 -- 172.21.15.8:0/1820526320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 msgr2=0x563d455c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 -- 172.21.15.8:0/1820526320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb64031070 con 0x563d455b5bd0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb737fe640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455c1b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feb6402de30 tx=0x7feb640041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ecbd20093764db server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb537fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb537fe640 1 -- 172.21.15.8:0/1820526320 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640037f0 con 0x563d455b5bd0 2024-02-19T16:24:21.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb537fe640 1 -- 172.21.15.8:0/1820526320 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7feb64054070 con 0x563d455b5bd0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.935+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb68004470 con 0x563d455b5bd0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb537fe640 1 -- 172.21.15.8:0/1820526320 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb640359b0 con 0x563d455b5bd0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 10 client.16062 fetch_fsmap learned FSMap version 25 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d454dd6a0 con 0x563d455b5bd0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb537fe640 1 -- 172.21.15.8:0/1820526320 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb68004470 con 0x563d455b5bd0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb537fe640 1 -- 172.21.15.8:0/1820526320 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feb64004d30 con 0x563d455b5bd0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 10 client.16062 fetch_fsmap finished waiting for FSMap version 25 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 20 client.16062 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d4562fb70 0x563d45631f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 4 client.16062 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563d452a8890 con 0x563d4562fb70 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.936+0000 7feb73fff640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d4562fb70 0x563d45631f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:21.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb73fff640 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d4562fb70 0x563d45631f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b76cbc97aa2d8e server_cookie=7978eab7fd11c608 in_seq=0 out_seq=0 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb537fe640 10 client.16062 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb537fe640 1 -- 172.21.15.8:0/1820526320 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563d452a8890 con 0x563d4562fb70 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb537fe640 10 client.16062 handle_command_reply: tid=0 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 client.16062 shutdown 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 msgr2=0x563d455c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455c1b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feb6402de30 tx=0x7feb640041e0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 shutdown_connections 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d455be7b0 0x563d455c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563d455b5bd0 0x563d455c1b70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563d4562fb70 0x563d45631f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 --2- 172.21.15.8:0/1820526320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d455b52b0 0x563d455c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb537fe640 0 client.16062 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 >> 172.21.15.8:0/1820526320 conn(0x563d455af990 msgr2=0x563d455aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 shutdown_connections 2024-02-19T16:24:21.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 1 -- 172.21.15.8:0/1820526320 wait complete. 2024-02-19T16:24:21.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7cfb6740 20 client.16062 trim_cache size 0 max 16384 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "4ca98691-9369-43ff-a3db-628fdb9f3011": { 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "4ca98691-9369-43ff-a3db-628fdb9f3011", 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:24:21.950 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7feb5c0a2350 msgr2=0x7feb5c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.937+0000 7feb7a308640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7feb5c0a2350 0x7feb5c0a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb64005ab0 tx=0x7feb640356b0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 msgr2=0x7feb74197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 0x7feb74197e60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00eb10 tx=0x7feb6c00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 shutdown_connections 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb741979b0 0x7feb74197e60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7feb5c0a2350 0x7feb5c0a4810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb74104e80 0x7feb7419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 --2- 172.21.15.8:0/387440196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb740ffdb0 0x7feb7419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:21.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 >> 172.21.15.8:0/387440196 conn(0x7feb74076910 msgr2=0x7feb74103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:21.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 shutdown_connections 2024-02-19T16:24:21.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:21.938+0000 7feb7a308640 1 -- 172.21.15.8:0/387440196 wait complete. 2024-02-19T16:24:21.953 INFO:tasks.cephfs.filesystem:scrub status for tag:4ca98691-9369-43ff-a3db-628fdb9f3011 - {'path': '/', 'tag': '4ca98691-9369-43ff-a3db-628fdb9f3011', 'options': 'recursive,force'} 2024-02-19T16:24:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:22 smithi008 ceph-mon[32573]: pgmap v47: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-02-19T16:24:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:22 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:24:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:22 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:22 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:22.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:22 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:22 smithi156 ceph-mon[36502]: pgmap v47: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-02-19T16:24:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:22 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:24:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:22 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:22 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:22.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:22 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:22.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:22 smithi191 ceph-mon[37203]: pgmap v47: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-02-19T16:24:22.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:22 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:24:22.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:22 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:22.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:22 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:22.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:22 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:23.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:23 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:23.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:23 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:23.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:23 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:23.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:23 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:23.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:23 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:23.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:23 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:24.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:24 smithi008 ceph-mon[32573]: pgmap v48: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-02-19T16:24:24.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:24 smithi191 ceph-mon[37203]: pgmap v48: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-02-19T16:24:24.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:24 smithi156 ceph-mon[36502]: pgmap v48: 129 pgs: 129 active+clean; 15 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-02-19T16:24:25.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:25 smithi008 ceph-mon[32573]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-02-19T16:24:25.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:25 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:25.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:25 smithi191 ceph-mon[37203]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-02-19T16:24:25.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:25 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:25.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:25 smithi156 ceph-mon[36502]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-02-19T16:24:25.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:25 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:26.519 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:26 smithi008 ceph-mon[32573]: pgmap v49: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 20 op/s 2024-02-19T16:24:26.526 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:26 smithi191 ceph-mon[37203]: pgmap v49: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 20 op/s 2024-02-19T16:24:26.579 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:26 smithi156 ceph-mon[36502]: pgmap v49: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 20 op/s 2024-02-19T16:24:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:28 smithi008 ceph-mon[32573]: pgmap v50: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 345 KiB/s wr, 14 op/s 2024-02-19T16:24:28.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:28 smithi191 ceph-mon[37203]: pgmap v50: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 345 KiB/s wr, 14 op/s 2024-02-19T16:24:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:28 smithi156 ceph-mon[36502]: pgmap v50: 129 pgs: 129 active+clean; 18 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 345 KiB/s wr, 14 op/s 2024-02-19T16:24:29.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:29 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:24:29.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:29 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:24:29.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:29 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:24:30.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:30 smithi008 ceph-mon[32573]: pgmap v51: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-02-19T16:24:30.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:30 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:24:30.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:30 smithi191 ceph-mon[37203]: pgmap v51: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-02-19T16:24:30.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:30 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:24:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:30 smithi156 ceph-mon[36502]: pgmap v51: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-02-19T16:24:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:30 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:24:32.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:32 smithi008 ceph-mon[32573]: pgmap v52: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 513 KiB/s wr, 28 op/s 2024-02-19T16:24:32.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:32 smithi191 ceph-mon[37203]: pgmap v52: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 513 KiB/s wr, 28 op/s 2024-02-19T16:24:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:32 smithi156 ceph-mon[36502]: pgmap v52: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 513 KiB/s wr, 28 op/s 2024-02-19T16:24:34.632 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:34 smithi156 ceph-mon[36502]: pgmap v53: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 513 KiB/s wr, 28 op/s 2024-02-19T16:24:34.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:34 smithi008 ceph-mon[32573]: pgmap v53: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 513 KiB/s wr, 28 op/s 2024-02-19T16:24:34.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:34 smithi191 ceph-mon[37203]: pgmap v53: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 513 KiB/s wr, 28 op/s 2024-02-19T16:24:35.703 INFO:tasks.workunit.client.0.smithi008.stderr:Updating files: 7% (324/4260) Updating files: 8% (341/4260) Updating files: 9% (384/4260) Updating files: 10% (426/4260) Updating files: 11% (469/4260) Updating files: 11% (492/4260) Updating files: 12% (512/4260) Updating files: 13% (554/4260) Updating files: 14% (597/4260) Updating files: 15% (639/4260) Updating files: 16% (682/4260) Updating files: 17% (725/4260) Updating files: 18% (767/4260) Updating files: 19% (810/4260) Updating files: 20% (852/4260) Updating files: 20% (853/4260) Updating files: 21% (895/4260) Updating files: 22% (938/4260) Updating files: 23% (980/4260) Updating files: 24% (1023/4260) Updating files: 25% (1065/4260) Updating files: 26% (1108/4260) Updating files: 27% (1151/4260) Updating files: 28% (1193/4260) Updating files: 29% (1236/4260) Updating files: 29% (1270/4260) Updating files: 30% (1278/4260) Updating files: 31% (1321/4260) Updating files: 32% (1364/4260) Updating files: 33% (1406/4260) Updating files: 34% (1449/4260) Updating files: 35% (1491/4260) Updating files: 36% (1534/4260) Updating files: 37% (1577/4260) Updating files: 38% (1619/4260) Updating files: 39% (1662/4260) Updating files: 39% (1692/4260) Updating files: 40% (1704/4260) Updating files: 41% (1747/4260) Updating files: 42% (1790/4260) Updating files: 43% (1832/4260) Updating files: 44% (1875/4260) Updating files: 45% (1917/4260) Updating files: 46% (1960/4260) Updating files: 47% (2003/4260) Updating files: 48% (2045/4260) Updating files: 49% (2088/4260) Updating files: 50% (2130/4260) Updating files: 51% (2173/4260) Updating files: 52% (2216/4260) Updating files: 53% (2258/4260) Updating files: 54% (2301/4260) Updating files: 55% (2343/4260) Updating files: 55% (2372/4260) Updating files: 56% (2386/4260) Updating files: 57% (2429/4260) Updating files: 58% (2471/4260) Updating files: 59% (2514/4260) Updating files: 60% (2556/4260) Updating files: 61% (2599/4260) Updating files: 62% (2642/4260) Updating files: 63% (2684/4260) Updating files: 64% (2727/4260) Updating files: 64% (2760/4260) Updating files: 65% (2769/4260) Updating files: 66% (2812/4260) Updating files: 67% (2855/4260) Updating files: 68% (2897/4260) Updating files: 69% (2940/4260) Updating files: 70% (2982/4260) Updating files: 70% (2998/4260) Updating files: 71% (3025/4260) Updating files: 72% (3068/4260) Updating files: 73% (3110/4260) Updating files: 74% (3153/4260) Updating files: 75% (3195/4260) Updating files: 76% (3238/4260) Updating files: 77% (3281/4260) Updating files: 78% (3323/4260) Updating files: 78% (3349/4260) Updating files: 79% (3366/4260) Updating files: 80% (3408/4260) Updating files: 81% (3451/4260) Updating files: 82% (3494/4260) Updating files: 83% (3536/4260) Updating files: 84% (3579/4260) Updating files: 85% (3621/4260) Updating files: 86% (3664/4260) Updating files: 87% (3707/4260) Updating files: 88% (3749/4260) Updating files: 89% (3792/4260) Updating files: 90% (3834/4260) Updating files: 91% (3877/4260) Updating files: 92% (3920/4260) Updating files: 93% (3962/4260) Updating files: 94% (4005/4260) Updating files: 95% (4047/4260) Updating files: 96% (4090/4260) Updating files: 97% (4133/4260) Updating files: 98% (4175/4260) Updating files: 99% (4218/4260) Updating files: 100% (4260/4260) Updating files: 100% (4260/4260), done. 2024-02-19T16:24:35.710 INFO:tasks.workunit.client.0.smithi008.stderr:+ cd xfstests-dev 2024-02-19T16:24:35.710 INFO:tasks.workunit.client.0.smithi008.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-02-19T16:24:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:36 smithi008 ceph-mon[32573]: pgmap v54: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 564 KiB/s wr, 34 op/s 2024-02-19T16:24:36.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:36 smithi191 ceph-mon[37203]: pgmap v54: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 564 KiB/s wr, 34 op/s 2024-02-19T16:24:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:36 smithi156 ceph-mon[36502]: pgmap v54: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 564 KiB/s wr, 34 op/s 2024-02-19T16:24:38.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:38 smithi008 ceph-mon[32573]: pgmap v55: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 291 KiB/s wr, 24 op/s 2024-02-19T16:24:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:38 smithi191 ceph-mon[37203]: pgmap v55: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 291 KiB/s wr, 24 op/s 2024-02-19T16:24:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:38 smithi156 ceph-mon[36502]: pgmap v55: 129 pgs: 129 active+clean; 20 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 291 KiB/s wr, 24 op/s 2024-02-19T16:24:40.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:40 smithi008 ceph-mon[32573]: pgmap v56: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-02-19T16:24:40.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:40 smithi191 ceph-mon[37203]: pgmap v56: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-02-19T16:24:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:40 smithi156 ceph-mon[36502]: pgmap v56: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-02-19T16:24:42.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:42 smithi008 ceph-mon[32573]: pgmap v57: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-02-19T16:24:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:42 smithi191 ceph-mon[37203]: pgmap v57: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-02-19T16:24:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:42 smithi156 ceph-mon[36502]: pgmap v57: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-02-19T16:24:44.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:44 smithi008 ceph-mon[32573]: pgmap v58: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-02-19T16:24:44.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:44 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:44.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:44 smithi191 ceph-mon[37203]: pgmap v58: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-02-19T16:24:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:44 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:44.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:44 smithi156 ceph-mon[36502]: pgmap v58: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-02-19T16:24:44.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:44 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:24:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:46 smithi008 ceph-mon[32573]: pgmap v59: 129 pgs: 129 active+clean; 47 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 487 op/s 2024-02-19T16:24:46.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:46 smithi191 ceph-mon[37203]: pgmap v59: 129 pgs: 129 active+clean; 47 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 487 op/s 2024-02-19T16:24:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:46 smithi156 ceph-mon[36502]: pgmap v59: 129 pgs: 129 active+clean; 47 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 487 op/s 2024-02-19T16:24:48.232 INFO:tasks.workunit.client.0.smithi008.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-02-19T16:24:48.232 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.232 INFO:tasks.workunit.client.0.smithi008.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:state without impacting any branches by switching back to a branch. 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: git switch -c 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:Or undo this operation with: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: git switch - 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-02-19T16:24:48.233 INFO:tasks.workunit.client.0.smithi008.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-02-19T16:24:48.235 INFO:tasks.workunit.client.0.smithi008.stderr:+ make -j4 2024-02-19T16:24:48.244 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize -cfi 2024-02-19T16:24:48.644 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: putting auxiliary files in '.'. 2024-02-19T16:24:48.644 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file './config.guess' 2024-02-19T16:24:48.649 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file './config.sub' 2024-02-19T16:24:48.653 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file './install-sh' 2024-02-19T16:24:48.658 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file './ltmain.sh' 2024-02-19T16:24:48.680 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-02-19T16:24:48.680 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-02-19T16:24:48.686 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-02-19T16:24:48.691 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-02-19T16:24:48.697 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-02-19T16:24:48.702 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-02-19T16:24:48.708 INFO:tasks.workunit.client.0.smithi008.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-02-19T16:24:48.709 INFO:tasks.workunit.client.0.smithi008.stdout:cp include/install-sh . 2024-02-19T16:24:48.718 INFO:tasks.workunit.client.0.smithi008.stdout:aclocal -I m4 2024-02-19T16:24:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:48 smithi008 ceph-mon[32573]: pgmap v60: 129 pgs: 129 active+clean; 47 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 482 op/s 2024-02-19T16:24:48.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:48 smithi191 ceph-mon[37203]: pgmap v60: 129 pgs: 129 active+clean; 47 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 482 op/s 2024-02-19T16:24:48.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:48 smithi156 ceph-mon[36502]: pgmap v60: 129 pgs: 129 active+clean; 47 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 482 op/s 2024-02-19T16:24:49.889 INFO:tasks.workunit.client.0.smithi008.stdout:autoheader 2024-02-19T16:24:50.202 INFO:tasks.workunit.client.0.smithi008.stdout:autoconf 2024-02-19T16:24:50.380 INFO:tasks.workunit.client.0.smithi008.stdout:./configure \ 2024-02-19T16:24:50.380 INFO:tasks.workunit.client.0.smithi008.stdout: --libexecdir=/usr/lib \ 2024-02-19T16:24:50.380 INFO:tasks.workunit.client.0.smithi008.stdout: --exec_prefix=/var/lib 2024-02-19T16:24:50.505 INFO:tasks.workunit.client.0.smithi008.stdout:checking build system type... x86_64-pc-linux-gnu 2024-02-19T16:24:50.505 INFO:tasks.workunit.client.0.smithi008.stdout:checking host system type... x86_64-pc-linux-gnu 2024-02-19T16:24:50.506 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to print strings... printf 2024-02-19T16:24:50.506 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gcc... gcc 2024-02-19T16:24:50.581 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether the C compiler works... yes 2024-02-19T16:24:50.581 INFO:tasks.workunit.client.0.smithi008.stdout:checking for C compiler default output file name... a.out 2024-02-19T16:24:50.618 INFO:tasks.workunit.client.0.smithi008.stdout:checking for suffix of executables... 2024-02-19T16:24:50.654 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether we are cross compiling... no 2024-02-19T16:24:50.687 INFO:tasks.workunit.client.0.smithi008.stdout:checking for suffix of object files... o 2024-02-19T16:24:50.711 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether we are using the GNU C compiler... yes 2024-02-19T16:24:50.732 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether gcc accepts -g... yes 2024-02-19T16:24:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:50 smithi008 ceph-mon[32573]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 601 op/s 2024-02-19T16:24:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:50 smithi191 ceph-mon[37203]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 601 op/s 2024-02-19T16:24:50.805 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gcc option to accept ISO C89... none needed 2024-02-19T16:24:50.809 INFO:tasks.workunit.client.0.smithi008.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-02-19T16:24:50.812 INFO:tasks.workunit.client.0.smithi008.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-02-19T16:24:50.813 INFO:tasks.workunit.client.0.smithi008.stdout:checking for egrep... /usr/bin/grep -E 2024-02-19T16:24:50.814 INFO:tasks.workunit.client.0.smithi008.stdout:checking for fgrep... /usr/bin/grep -F 2024-02-19T16:24:50.817 INFO:tasks.workunit.client.0.smithi008.stdout:checking for ld used by gcc... /usr/bin/ld 2024-02-19T16:24:50.819 INFO:tasks.workunit.client.0.smithi008.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-02-19T16:24:50.823 INFO:tasks.workunit.client.0.smithi008.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-02-19T16:24:50.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:50 smithi156 ceph-mon[36502]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 601 op/s 2024-02-19T16:24:51.725 INFO:tasks.workunit.client.0.smithi008.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-02-19T16:24:51.725 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether ln -s works... yes 2024-02-19T16:24:51.733 INFO:tasks.workunit.client.0.smithi008.stdout:checking the maximum length of command line arguments... 1572864 2024-02-19T16:24:51.736 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-02-19T16:24:51.736 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-02-19T16:24:51.736 INFO:tasks.workunit.client.0.smithi008.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-02-19T16:24:51.736 INFO:tasks.workunit.client.0.smithi008.stdout:checking for objdump... objdump 2024-02-19T16:24:51.736 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to recognize dependent libraries... pass_all 2024-02-19T16:24:51.738 INFO:tasks.workunit.client.0.smithi008.stdout:checking for dlltool... no 2024-02-19T16:24:51.738 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-02-19T16:24:51.738 INFO:tasks.workunit.client.0.smithi008.stdout:checking for ar... ar 2024-02-19T16:24:51.817 INFO:tasks.workunit.client.0.smithi008.stdout:checking for archiver @FILE support... @ 2024-02-19T16:24:51.817 INFO:tasks.workunit.client.0.smithi008.stdout:checking for strip... strip 2024-02-19T16:24:51.817 INFO:tasks.workunit.client.0.smithi008.stdout:checking for ranlib... ranlib 2024-02-19T16:24:51.818 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gawk... gawk 2024-02-19T16:24:51.920 INFO:tasks.workunit.client.0.smithi008.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-02-19T16:24:51.923 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sysroot... no 2024-02-19T16:24:51.932 INFO:tasks.workunit.client.0.smithi008.stdout:checking for a working dd... /usr/bin/dd 2024-02-19T16:24:51.940 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-02-19T16:24:51.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:24:51.954 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:24:51.969 INFO:tasks.workunit.client.0.smithi008.stdout:checking for mt... no 2024-02-19T16:24:51.982 INFO:tasks.workunit.client.0.smithi008.stdout:checking if : is a manifest tool... no 2024-02-19T16:24:52.057 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to run the C preprocessor... gcc -E 2024-02-19T16:24:52.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1194075012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108100610 msgr2=0x7ff108106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1194075012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108100610 0x7ff108106160 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009900 tx=0x7ff0fc02bc30 comp rx=0 tx=0).stop 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1194075012 shutdown_connections 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1194075012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 0x7ff10810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1194075012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108100610 0x7ff108106160 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1194075012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 0x7ff108100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1194075012 >> 172.21.15.8:0/1194075012 conn(0x7ff108076910 msgr2=0x7ff1080fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1194075012 shutdown_connections 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.062+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1194075012 wait complete. 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 Processor -- start 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 -- start start 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 0x7ff10819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 0x7ff10819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 0x7ff108199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10819a490 con 0x7ff108199aa0 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10819a600 con 0x7ff108106890 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff10dfa1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10819a7c0 con 0x7ff1080ffc60 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff107fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 0x7ff108199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff107fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 0x7ff108199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42198/0 (socket says 172.21.15.8:42198) 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff107fff640 1 -- 172.21.15.8:0/1360787882 learned_addr learned my addr 172.21.15.8:0/1360787882 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 0x7ff10819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff106ffd640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 0x7ff10819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff1077fe640 1 -- 172.21.15.8:0/1360787882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 msgr2=0x7ff10819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff1077fe640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 0x7ff10819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff1077fe640 1 -- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 msgr2=0x7ff108199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff1077fe640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 0x7ff108199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff1077fe640 1 -- 172.21.15.8:0/1360787882 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009560 con 0x7ff1080ffc60 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff106ffd640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 0x7ff10819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.063+0000 7ff107fff640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 0x7ff108199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.064+0000 7ff1077fe640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 0x7ff10819f8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e9f0 tx=0x7ff0f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.064+0000 7ff104ff9640 1 -- 172.21.15.8:0/1360787882 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800ccc0 con 0x7ff1080ffc60 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.064+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1081a67b0 con 0x7ff1080ffc60 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.064+0000 7ff104ff9640 1 -- 172.21.15.8:0/1360787882 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0f800ce20 con 0x7ff1080ffc60 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.064+0000 7ff104ff9640 1 -- 172.21.15.8:0/1360787882 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8018920 con 0x7ff1080ffc60 2024-02-19T16:24:52.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.064+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081a6c90 con 0x7ff1080ffc60 2024-02-19T16:24:52.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.066+0000 7ff104ff9640 1 -- 172.21.15.8:0/1360787882 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff0f8018a80 con 0x7ff1080ffc60 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.066+0000 7ff110b80740 1 Processor -- start 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.066+0000 7ff110b80740 1 -- start start 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff104ff9640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff0f00a22b0 0x7ff0f00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff110b80740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ff2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff110b80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ffb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff110b80740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ffdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff110b80740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16f61280 con 0x560b16ff22b0 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff110b80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16f5e7b0 con 0x560b16ff2bd0 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff104ff9640 1 -- 172.21.15.8:0/1360787882 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff0f8014070 con 0x7ff1080ffc60 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff110b80740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16f59f90 con 0x560b16ffb7b0 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff107fff640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff0f00a22b0 0x7ff0f00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ffb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ff2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ffb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37496/0 (socket says 172.21.15.8:37496) 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 -- 172.21.15.8:0/1170795965 learned_addr learned my addr 172.21.15.8:0/1170795965 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff107fff640 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ffdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 -- 172.21.15.8:0/1170795965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 msgr2=0x560b16ffdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ffdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 -- 172.21.15.8:0/1170795965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 msgr2=0x560b16ff2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ff2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff106ffd640 1 -- 172.21.15.8:0/1170795965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b16f18570 con 0x560b16ff2bd0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.067+0000 7ff107fff640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff0f00a22b0 0x7ff0f00a4770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff100009fd0 tx=0x7ff100011810 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff106ffd640 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ffb270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1080695b0 tx=0x7ff0fc02c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a069bf8a7c3bcd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff0edffb640 1 -- 172.21.15.8:0/1170795965 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004af0 con 0x560b16ff2bd0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff0edffb640 1 -- 172.21.15.8:0/1170795965 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc004030 con 0x560b16ff2bd0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff0edffb640 1 -- 172.21.15.8:0/1170795965 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc005430 con 0x560b16ff2bd0 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 -- 172.21.15.8:0/1170795965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 msgr2=0x560b16ffb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ffb270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1080695b0 tx=0x7ff0fc02c680 comp rx=0 tx=0).stop 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 -- 172.21.15.8:0/1170795965 shutdown_connections 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ffdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ffb270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 --2- 172.21.15.8:0/1170795965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ff2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.068+0000 7ff110b80740 1 -- 172.21.15.8:0/1170795965 >> 172.21.15.8:0/1170795965 conn(0x560b16fec990 msgr2=0x560b16fecd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 -- 172.21.15.8:0/1170795965 shutdown_connections 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 -- 172.21.15.8:0/1170795965 wait complete. 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 -- start start 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 Processor -- start 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ffb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ff8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ff8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16c2d430 con 0x560b16ff22b0 2024-02-19T16:24:52.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16be9e00 con 0x560b16ff2bd0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16edefd0 con 0x560b16ffb7b0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff110b80740 10 client.0 _ll_register_callbacks cb 0x560b16f34fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ffb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ffb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42220/0 (socket says 172.21.15.8:42220) 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff1077fe640 1 -- 172.21.15.8:0/3870660227 learned_addr learned my addr 172.21.15.8:0/3870660227 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff107fff640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ff8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.069+0000 7ff106ffd640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ff8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff1077fe640 1 -- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 msgr2=0x560b16ff8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff1077fe640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ff8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff1077fe640 1 -- 172.21.15.8:0/3870660227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 msgr2=0x560b16ff8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff1077fe640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ff8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff1077fe640 1 -- 172.21.15.8:0/3870660227 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10810ec00 con 0x560b16ff22b0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff1077fe640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ffb1f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010eb0 tx=0x7ff0f809af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d967d80eff4d6b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:52.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/3870660227 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80ce530 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/3870660227 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff0f8010bf0 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b16c597e0 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/3870660227 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f80cbf60 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff110b80740 10 client.16074 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560b16f58610 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.070+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/3870660227 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560b16c597e0 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/3870660227 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff0f8099d70 con 0x560b16ff22b0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff110b80740 10 client.16074 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff110b80740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff110b80740 20 client.16074 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff110b80740 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b1700b920 0x560b1700dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff110b80740 4 client.16074 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b16c96720 con 0x560b1700b920 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff106ffd640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b1700b920 0x560b1700dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff0e6ffd640 10 client.16074 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.071+0000 7ff106ffd640 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b1700b920 0x560b1700dd40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2090c1c57d0076a5 server_cookie=cc4ec5045d99a2dc in_seq=0 out_seq=0 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/3870660227 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560b16c96720 con 0x560b1700b920 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff0e6ffd640 10 client.16074 handle_command_reply: tid=0 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 client.16074 shutdown 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 msgr2=0x560b16ffb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ffb1f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010eb0 tx=0x7ff0f809af60 comp rx=0 tx=0).stop 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 shutdown_connections 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b1700b920 0x560b1700dd40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b16ff8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff2bd0 0x560b16ff8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 --2- 172.21.15.8:0/3870660227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff22b0 0x560b16ffb1f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff0e6ffd640 0 client.16074 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 >> 172.21.15.8:0/3870660227 conn(0x560b16fec990 msgr2=0x560b16fee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 shutdown_connections 2024-02-19T16:24:52.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 1 -- 172.21.15.8:0/3870660227 wait complete. 2024-02-19T16:24:52.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.072+0000 7ff110b80740 20 client.16074 trim_cache size 0 max 16384 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 Processor -- start 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 -- start start 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ff8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ff2bd0 0x560b16ffacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 0x560b16ffb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16fed9e0 con 0x560b16ffb7b0 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b16ff1f10 con 0x560b16ff22b0 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff110b80740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b17007da0 con 0x560b16ff2bd0 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ff8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff107fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 0x560b16ffb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ff2bd0 0x560b16ffacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ff8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37506/0 (socket says 172.21.15.8:37506) 2024-02-19T16:24:52.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff107fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 0x560b16ffb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42224/0 (socket says 172.21.15.8:42224) 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 -- 172.21.15.8:0/3224764744 learned_addr learned my addr 172.21.15.8:0/3224764744 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 -- 172.21.15.8:0/3224764744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ff2bd0 msgr2=0x560b16ffacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ff2bd0 0x560b16ffacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 -- 172.21.15.8:0/3224764744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 msgr2=0x560b16ffb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 0x560b16ffb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 -- 172.21.15.8:0/3224764744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b16ef4ac0 con 0x560b16ff22b0 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff107fff640 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 0x560b16ffb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.089+0000 7ff1077fe640 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ff8520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0f809ab20 tx=0x7ff0f809ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c92102d80815b34 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff0ed7fa640 1 -- 172.21.15.8:0/3224764744 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f809a780 con 0x560b16ff22b0 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff0ed7fa640 1 -- 172.21.15.8:0/3224764744 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0f80cbf60 con 0x560b16ff22b0 2024-02-19T16:24:52.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff0ed7fa640 1 -- 172.21.15.8:0/3224764744 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80ce590 con 0x560b16ff22b0 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 -- 172.21.15.8:0/3224764744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 msgr2=0x560b16ff8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ff8520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0f809ab20 tx=0x7ff0f809ad20 comp rx=0 tx=0).stop 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 -- 172.21.15.8:0/3224764744 shutdown_connections 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ffb7b0 0x560b16ffb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ff2bd0 0x560b16ffacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 --2- 172.21.15.8:0/3224764744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ff8520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 -- 172.21.15.8:0/3224764744 >> 172.21.15.8:0/3224764744 conn(0x560b16fec990 msgr2=0x560b16ff7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 -- 172.21.15.8:0/3224764744 shutdown_connections 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.090+0000 7ff110b80740 1 -- 172.21.15.8:0/3224764744 wait complete. 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 -- start start 2024-02-19T16:24:52.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 Processor -- start 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ffc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 0x560b16ffccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b170017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b17001d30 con 0x560b16ff2bd0 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b17001e70 con 0x560b16ff22b0 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b17002030 con 0x560b16ffb7b0 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff110b80740 10 client.0 _ll_register_callbacks cb 0x7ff0f809a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 0x560b16ffccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ffc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 0x560b16ffccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42240/0 (socket says 172.21.15.8:42240) 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ffc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37514/0 (socket says 172.21.15.8:37514) 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff106ffd640 1 -- 172.21.15.8:0/2274666255 learned_addr learned my addr 172.21.15.8:0/2274666255 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.091+0000 7ff107fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b170017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff106ffd640 1 -- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 msgr2=0x560b170017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff106ffd640 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b170017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff106ffd640 1 -- 172.21.15.8:0/2274666255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 msgr2=0x560b16ffc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff106ffd640 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ffc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff106ffd640 1 -- 172.21.15.8:0/2274666255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10000f4e0 con 0x560b16ff2bd0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff1077fe640 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ffc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff106ffd640 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 0x560b16ffccf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff1080695b0 tx=0x7ff0fc004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628731d62996972b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/2274666255 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc051320 con 0x560b16ff2bd0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0fc004030 con 0x560b16ff2bd0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/2274666255 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff0fc04e050 con 0x560b16ff2bd0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/2274666255 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0fc004320 con 0x560b16ff2bd0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff110b80740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560b16f58610 con 0x560b16ff2bd0 2024-02-19T16:24:52.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/2274666255 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc004030 con 0x560b16ff2bd0 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/2274666255 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff0fc03c050 con 0x560b16ff2bd0 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff110b80740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff110b80740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.092+0000 7ff110b80740 20 client.16080 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.093+0000 7ff110b80740 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b17067f40 0x560b1706a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.093+0000 7ff110b80740 4 client.16080 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.093+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b16ce5890 con 0x560b17067f40 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.093+0000 7ff1077fe640 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b17067f40 0x560b1706a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.093+0000 7ff0e6ffd640 10 client.16080 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:52.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.093+0000 7ff1077fe640 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b17067f40 0x560b1706a360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4700b5b7e4bad6 server_cookie=5f52275497e145f2 in_seq=0 out_seq=0 2024-02-19T16:24:52.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.095+0000 7ff0e6ffd640 1 -- 172.21.15.8:0/2274666255 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560b16ce5890 con 0x560b17067f40 2024-02-19T16:24:52.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.095+0000 7ff0e6ffd640 10 client.16080 handle_command_reply: tid=0 2024-02-19T16:24:52.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.095+0000 7ff110b80740 1 client.16080 shutdown 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 msgr2=0x560b16ffccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 0x560b16ffccf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff1080695b0 tx=0x7ff0fc004f70 comp rx=0 tx=0).stop 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 shutdown_connections 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560b17067f40 0x560b1706a360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b16ffb7b0 0x560b170017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560b16ff2bd0 0x560b16ffccf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff0e6ffd640 0 client.16080 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 --2- 172.21.15.8:0/2274666255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b16ff22b0 0x560b16ffc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 >> 172.21.15.8:0/2274666255 conn(0x560b16fec990 msgr2=0x560b16feb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 shutdown_connections 2024-02-19T16:24:52.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 1 -- 172.21.15.8:0/2274666255 wait complete. 2024-02-19T16:24:52.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff110b80740 20 client.16080 trim_cache size 0 max 16384 2024-02-19T16:24:52.102 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:24:52.102 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:24:52.102 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:24:52.102 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff0f00a22b0 msgr2=0x7ff0f00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff0f00a22b0 0x7ff0f00a4770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff100009fd0 tx=0x7ff100011810 comp rx=0 tx=0).stop 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 msgr2=0x7ff10819f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 0x7ff10819f8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e9f0 tx=0x7ff0f800eeb0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 shutdown_connections 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff108199aa0 0x7ff108199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff108106890 0x7ff10819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff0f00a22b0 0x7ff0f00a4770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 --2- 172.21.15.8:0/1360787882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1080ffc60 0x7ff10819f8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.096+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 >> 172.21.15.8:0/1360787882 conn(0x7ff108076910 msgr2=0x7ff108104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.097+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 shutdown_connections 2024-02-19T16:24:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.097+0000 7ff10dfa1640 1 -- 172.21.15.8:0/1360787882 wait complete. 2024-02-19T16:24:52.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:24:52.113 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:24:52.226 INFO:tasks.workunit.client.0.smithi008.stdout:checking for ANSI C header files... yes 2024-02-19T16:24:52.259 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/types.h... yes 2024-02-19T16:24:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 -- 172.21.15.8:0/1358771863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 msgr2=0x7f3560100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 --2- 172.21.15.8:0/1358771863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f3560100a00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3554009930 tx=0x7f355402bc30 comp rx=0 tx=0).stop 2024-02-19T16:24:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 -- 172.21.15.8:0/1358771863 shutdown_connections 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 --2- 172.21.15.8:0/1358771863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35601056f0 0x7f356010d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 --2- 172.21.15.8:0/1358771863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3560100fd0 0x7f3560105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 --2- 172.21.15.8:0/1358771863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f3560100a00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 -- 172.21.15.8:0/1358771863 >> 172.21.15.8:0/1358771863 conn(0x7f3560077490 msgr2=0x7f35600778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 -- 172.21.15.8:0/1358771863 shutdown_connections 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.259+0000 7f35673c9640 1 -- 172.21.15.8:0/1358771863 wait complete. 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 Processor -- start 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 -- start start 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f356019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35601056f0 0x7f356019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35601979b0 0x7f3560197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35601983a0 con 0x7f3560100620 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3560198510 con 0x7f35601979b0 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f35673c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35601986d0 con 0x7f35601056f0 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f3565165640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f356019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f3565165640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f356019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42250/0 (socket says 172.21.15.8:42250) 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f3565165640 1 -- 172.21.15.8:0/2238586987 learned_addr learned my addr 172.21.15.8:0/2238586987 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f3564964640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35601056f0 0x7f356019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.260+0000 7f3565966640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35601979b0 0x7f3560197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f3565165640 1 -- 172.21.15.8:0/2238586987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35601056f0 msgr2=0x7f356019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f3565165640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35601056f0 0x7f356019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f3565165640 1 -- 172.21.15.8:0/2238586987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35601979b0 msgr2=0x7f3560197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f3565165640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35601979b0 0x7f3560197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f3565165640 1 -- 172.21.15.8:0/2238586987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3558009840 con 0x7f3560100620 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f3565165640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f356019d800 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3554009230 tx=0x7f355402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554004350 con 0x7f3560100620 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35540044b0 con 0x7f3560100620 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3554009560 con 0x7f3560100620 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554003bf0 con 0x7f3560100620 2024-02-19T16:24:52.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.261+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35601a49e0 con 0x7f3560100620 2024-02-19T16:24:52.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.262+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3560068730 con 0x7f3560100620 2024-02-19T16:24:52.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.263+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f3554003d50 con 0x7f3560100620 2024-02-19T16:24:52.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.264+0000 7f35527fc640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f354c0a2230 0x7f354c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.264+0000 7f3564964640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f354c0a2230 0x7f354c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.264+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f35540f0ab0 con 0x7f3560100620 2024-02-19T16:24:52.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.264+0000 7f3564964640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f354c0a2230 0x7f354c0a46f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35600fef10 tx=0x7f355c005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.268+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f35540b9440 con 0x7f3560100620 2024-02-19T16:24:52.296 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/stat.h... yes 2024-02-19T16:24:52.331 INFO:tasks.workunit.client.0.smithi008.stdout:checking for stdlib.h... yes 2024-02-19T16:24:52.372 INFO:tasks.workunit.client.0.smithi008.stdout:checking for string.h... yes 2024-02-19T16:24:52.414 INFO:tasks.workunit.client.0.smithi008.stdout:checking for memory.h... yes 2024-02-19T16:24:52.452 INFO:tasks.workunit.client.0.smithi008.stdout:checking for strings.h... yes 2024-02-19T16:24:52.488 INFO:tasks.workunit.client.0.smithi008.stdout:checking for inttypes.h... yes 2024-02-19T16:24:52.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.495+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3560108920 con 0x7f3560100620 2024-02-19T16:24:52.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.496+0000 7f35527fc640 1 -- 172.21.15.8:0/2238586987 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f35540bcb00 con 0x7f3560100620 2024-02-19T16:24:52.497 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:24:52.497 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:24:42.507626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:24:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 27 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f354c0a2230 msgr2=0x7f354c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f354c0a2230 0x7f354c0a46f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35600fef10 tx=0x7f355c005f90 comp rx=0 tx=0).stop 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 msgr2=0x7f356019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f356019d800 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3554009230 tx=0x7f355402c650 comp rx=0 tx=0).stop 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 shutdown_connections 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35601979b0 0x7f3560197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f354c0a2230 0x7f354c0a46f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35601056f0 0x7f356019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 --2- 172.21.15.8:0/2238586987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3560100620 0x7f356019d800 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 >> 172.21.15.8:0/2238586987 conn(0x7f3560077490 msgr2=0x7f35601068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 shutdown_connections 2024-02-19T16:24:52.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.498+0000 7f35673c9640 1 -- 172.21.15.8:0/2238586987 wait complete. 2024-02-19T16:24:52.511 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:24:52.511 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:24:52.527 INFO:tasks.workunit.client.0.smithi008.stdout:checking for stdint.h... yes 2024-02-19T16:24:52.569 INFO:tasks.workunit.client.0.smithi008.stdout:checking for unistd.h... yes 2024-02-19T16:24:52.615 INFO:tasks.workunit.client.0.smithi008.stdout:checking for dlfcn.h... yes 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 -- 172.21.15.8:0/2321015393 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f4103050 msgr2=0x7f15f410b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/2321015393 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f4103050 0x7f15f410b530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009900 tx=0x7f15dc02bcd0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 -- 172.21.15.8:0/2321015393 shutdown_connections 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/2321015393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f410bb00 0x7f15f410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/2321015393 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f4103050 0x7f15f410b530 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/2321015393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f4102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.617+0000 7f15f8ac9640 1 -- 172.21.15.8:0/2321015393 >> 172.21.15.8:0/2321015393 conn(0x7f15f4076770 msgr2=0x7f15f4076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.618+0000 7f15f8ac9640 1 -- 172.21.15.8:0/2321015393 shutdown_connections 2024-02-19T16:24:52.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.618+0000 7f15f8ac9640 1 -- 172.21.15.8:0/2321015393 wait complete. 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.618+0000 7f15f8ac9640 1 Processor -- start 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.618+0000 7f15f8ac9640 1 -- start start 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.618+0000 7f15f8ac9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.618+0000 7f15f8ac9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f4103050 0x7f15f419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f8ac9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 0x7f15f4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f8ac9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4197fc0 con 0x7f15f410bb00 2024-02-19T16:24:52.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f8ac9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4198130 con 0x7f15f4102730 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f8ac9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41982f0 con 0x7f15f4103050 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 0x7f15f4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f4103050 0x7f15f419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 0x7f15f4197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42280/0 (socket says 172.21.15.8:42280) 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37570/0 (socket says 172.21.15.8:37570) 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 -- 172.21.15.8:0/3249823164 learned_addr learned my addr 172.21.15.8:0/3249823164 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 -- 172.21.15.8:0/3249823164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f4103050 msgr2=0x7f15f419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f4103050 0x7f15f419dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 -- 172.21.15.8:0/3249823164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 msgr2=0x7f15f419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 -- 172.21.15.8:0/3249823164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc009560 con 0x7f15f410bb00 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f259c640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15ebfff640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f4103050 0x7f15f419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f2d9d640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 0x7f15f4197970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00e970 tx=0x7f15ec00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15eb7fe640 1 -- 172.21.15.8:0/3249823164 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0096c0 con 0x7f15f410bb00 2024-02-19T16:24:52.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15eb7fe640 1 -- 172.21.15.8:0/3249823164 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15ec005500 con 0x7f15f410bb00 2024-02-19T16:24:52.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f41985d0 con 0x7f15f410bb00 2024-02-19T16:24:52.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15eb7fe640 1 -- 172.21.15.8:0/3249823164 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec010430 con 0x7f15f410bb00 2024-02-19T16:24:52.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.619+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f41a4860 con 0x7f15f410bb00 2024-02-19T16:24:52.624 INFO:tasks.workunit.client.0.smithi008.stdout:checking for objdir... .libs 2024-02-19T16:24:52.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.621+0000 7f15eb7fe640 1 -- 172.21.15.8:0/3249823164 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f15ec005020 con 0x7f15f410bb00 2024-02-19T16:24:52.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 Processor -- start 2024-02-19T16:24:52.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 -- start start 2024-02-19T16:24:52.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6250 0x55f6d51b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 0x55f6d51bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5171960 con 0x55f6d51b6250 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5125310 con 0x55f6d51b6b70 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15fb776740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5122840 con 0x55f6d51bf750 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15eb7fe640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15e00a2230 0x7f15e00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6250 0x55f6d51b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 0x55f6d51bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 0x55f6d51bf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37572/0 (socket says 172.21.15.8:37572) 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15f2d9d640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15e00a2230 0x7f15e00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15f259c640 1 -- 172.21.15.8:0/3779881287 learned_addr learned my addr 172.21.15.8:0/3779881287 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15ebfff640 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.622+0000 7f15eb7fe640 1 -- 172.21.15.8:0/3249823164 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f15ec014070 con 0x7f15f410bb00 2024-02-19T16:24:52.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f259c640 1 -- 172.21.15.8:0/3779881287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 msgr2=0x55f6d51c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f259c640 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f259c640 1 -- 172.21.15.8:0/3779881287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6250 msgr2=0x55f6d51b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f259c640 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6250 0x55f6d51b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f259c640 1 -- 172.21.15.8:0/3779881287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d50e0060 con 0x55f6d51b6b70 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f2d9d640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15e00a2230 0x7f15e00a46f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0cbb80 tx=0x7f15ec01bb30 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15f259c640 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 0x55f6d51bf210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15e4011820 tx=0x7f15e4011ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912c510c244af1ec server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15e8ff9640 1 -- 172.21.15.8:0/3779881287 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011220 con 0x55f6d51b6b70 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15e8ff9640 1 -- 172.21.15.8:0/3779881287 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15e400fa60 con 0x55f6d51b6b70 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.623+0000 7f15e8ff9640 1 -- 172.21.15.8:0/3779881287 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e40055d0 con 0x55f6d51b6b70 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 -- 172.21.15.8:0/3779881287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 msgr2=0x55f6d51bf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 0x55f6d51bf210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15e4011820 tx=0x7f15e4011ce0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 -- 172.21.15.8:0/3779881287 shutdown_connections 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6b70 0x55f6d51bf210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 --2- 172.21.15.8:0/3779881287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6250 0x55f6d51b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 -- 172.21.15.8:0/3779881287 >> 172.21.15.8:0/3779881287 conn(0x55f6d51b0980 msgr2=0x55f6d51b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 -- 172.21.15.8:0/3779881287 shutdown_connections 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 -- 172.21.15.8:0/3779881287 wait complete. 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.624+0000 7f15fb776740 1 -- start start 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 Processor -- start 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51bc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d51bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d511e020 con 0x55f6d51b6b70 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d4df1430 con 0x55f6d51b6250 2024-02-19T16:24:52.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d4dade00 con 0x55f6d51bf750 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15fb776740 10 client.0 _ll_register_callbacks cb 0x55f6d50c5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15f259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d51bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15f259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d51bcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42290/0 (socket says 172.21.15.8:42290) 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51bc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15f259c640 1 -- 172.21.15.8:0/882515540 learned_addr learned my addr 172.21.15.8:0/882515540 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 -- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 msgr2=0x55f6d51bd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51bd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 -- 172.21.15.8:0/882515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 msgr2=0x55f6d51bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d51bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 -- 172.21.15.8:0/882515540 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e4011480 con 0x55f6d51b6250 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15f259c640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d51bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15ebfff640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51bc7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15dc000c00 tx=0x7f15dc02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75787462989abab server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.625+0000 7f15d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15d9ffb640 1 -- 172.21.15.8:0/882515540 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc04a070 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6d4e1d7e0 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15d9ffb640 1 -- 172.21.15.8:0/882515540 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f15dc005000 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15d9ffb640 1 -- 172.21.15.8:0/882515540 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15dc0051c0 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 10 client.25871 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f6d511c6a0 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15d9ffb640 1 -- 172.21.15.8:0/882515540 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6d4e1d7e0 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15d9ffb640 1 -- 172.21.15.8:0/882515540 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f15dc03c040 con 0x55f6d51b6250 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 10 client.25871 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 10 client.25871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 20 client.25871 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d51cf9c0 0x55f6d51d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 4 client.25871 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:52.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6d4e5a720 con 0x55f6d51cf9c0 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.626+0000 7f15f259c640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d51cf9c0 0x55f6d51d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.627+0000 7f15f259c640 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d51cf9c0 0x55f6d51d1db0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fccff0a97c5c30c server_cookie=fb4d4598aef73d23 in_seq=0 out_seq=0 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.627+0000 7f15d9ffb640 10 client.25871 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15d9ffb640 1 -- 172.21.15.8:0/882515540 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f6d4e5a720 con 0x55f6d51cf9c0 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15d9ffb640 10 client.25871 handle_command_reply: tid=0 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 client.25871 shutdown 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 msgr2=0x55f6d51bc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51bc7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15dc000c00 tx=0x7f15dc02bcd0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 shutdown_connections 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d51cf9c0 0x55f6d51d1db0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51bd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d51bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15d9ffb640 0 client.25871 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:52.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 --2- 172.21.15.8:0/882515540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51bc7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 >> 172.21.15.8:0/882515540 conn(0x55f6d51b0980 msgr2=0x55f6d51b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 shutdown_connections 2024-02-19T16:24:52.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 1 -- 172.21.15.8:0/882515540 wait complete. 2024-02-19T16:24:52.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.628+0000 7f15fb776740 20 client.25871 trim_cache size 0 max 16384 2024-02-19T16:24:52.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.652+0000 7f15fb776740 1 Processor -- start 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 -- start start 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51b6b70 0x55f6d51aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 0x55f6d51afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d50a3130 con 0x55f6d51bf750 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d51b5eb0 con 0x55f6d51b6250 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15fb776740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d51b83a0 con 0x55f6d51b6b70 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15f259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51b6b70 0x55f6d51aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 0x55f6d51afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15f259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51b6b70 0x55f6d51aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:50968/0 (socket says 172.21.15.8:50968) 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 0x55f6d51afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42304/0 (socket says 172.21.15.8:42304) 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15f259c640 1 -- 172.21.15.8:0/1309415063 learned_addr learned my addr 172.21.15.8:0/1309415063 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 -- 172.21.15.8:0/1309415063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51b6b70 msgr2=0x55f6d51aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51b6b70 0x55f6d51aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 -- 172.21.15.8:0/1309415063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 msgr2=0x55f6d51beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 -- 172.21.15.8:0/1309415063 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d50b8c20 con 0x55f6d51bf750 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.653+0000 7f15ebfff640 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 0x55f6d51afa00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005c40 tx=0x7f15dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150b7f72f78b077b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15dbfff640 1 -- 172.21.15.8:0/1309415063 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc04a030 con 0x55f6d51bf750 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15dbfff640 1 -- 172.21.15.8:0/1309415063 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15dc04b040 con 0x55f6d51bf750 2024-02-19T16:24:52.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15dbfff640 1 -- 172.21.15.8:0/1309415063 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc005260 con 0x55f6d51bf750 2024-02-19T16:24:52.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 -- 172.21.15.8:0/1309415063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 msgr2=0x55f6d51afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 0x55f6d51afa00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005c40 tx=0x7f15dc005fd0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 -- 172.21.15.8:0/1309415063 shutdown_connections 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51bf750 0x55f6d51afa00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51b6b70 0x55f6d51aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 --2- 172.21.15.8:0/1309415063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d51beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 -- 172.21.15.8:0/1309415063 >> 172.21.15.8:0/1309415063 conn(0x55f6d51b0980 msgr2=0x55f6d51af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 -- 172.21.15.8:0/1309415063 shutdown_connections 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.654+0000 7f15fb776740 1 -- 172.21.15.8:0/1309415063 wait complete. 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 -- start start 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 Processor -- start 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d52245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d5224b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d51c3840 con 0x55f6d51b6b70 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d51c39b0 con 0x55f6d51b6250 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d51c3b70 con 0x55f6d51bf750 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15fb776740 10 client.0 _ll_register_callbacks cb 0x7f15dc0041c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d5224b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d5224b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42308/0 (socket says 172.21.15.8:42308) 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 -- 172.21.15.8:0/2171150755 learned_addr learned my addr 172.21.15.8:0/2171150755 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15ebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d52245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15ebfff640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 -- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 msgr2=0x55f6d51c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 -- 172.21.15.8:0/2171150755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 msgr2=0x55f6d52245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d52245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15f259c640 1 -- 172.21.15.8:0/2171150755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6d511c6a0 con 0x55f6d51b6b70 2024-02-19T16:24:52.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.655+0000 7f15ebfff640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15f259c640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d5224b20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f15e4000f80 tx=0x7f15e4009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66028ac949c2a9fb server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15d9ffb640 1 -- 172.21.15.8:0/2171150755 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4029040 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15d9ffb640 1 -- 172.21.15.8:0/2171150755 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f15e4003970 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6d51ac0f0 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15d9ffb640 1 -- 172.21.15.8:0/2171150755 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e4004b20 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f6d50de8c0 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15d9ffb640 1 -- 172.21.15.8:0/2171150755 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6d51ac0f0 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15d9ffb640 1 -- 172.21.15.8:0/2171150755 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f15e4004030 con 0x55f6d51b6b70 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 10 client.16104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 20 client.16104 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d522c4a0 0x55f6d522e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 4 client.16104 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f15e400fa60 con 0x55f6d522c4a0 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.656+0000 7f15ebfff640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d522c4a0 0x55f6d522e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15ebfff640 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d522c4a0 0x55f6d522e8c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ed4919eaff6d63d server_cookie=6176916358ad5bd5 in_seq=0 out_seq=0 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15d9ffb640 10 client.16104 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15d9ffb640 1 -- 172.21.15.8:0/2171150755 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f15e400fa60 con 0x55f6d522c4a0 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15d9ffb640 10 client.16104 handle_command_reply: tid=0 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 client.16104 shutdown 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 msgr2=0x55f6d5224b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d5224b20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f15e4000f80 tx=0x7f15e4009990 comp rx=0 tx=0).stop 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 shutdown_connections 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f6d522c4a0 0x55f6d522e8c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f6d51bf750 0x55f6d51c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f6d51b6b70 0x55f6d5224b20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15d9ffb640 0 client.16104 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:24:52.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 --2- 172.21.15.8:0/2171150755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d51b6250 0x55f6d52245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 >> 172.21.15.8:0/2171150755 conn(0x55f6d51b0980 msgr2=0x55f6d51b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 shutdown_connections 2024-02-19T16:24:52.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.657+0000 7f15fb776740 1 -- 172.21.15.8:0/2171150755 wait complete. 2024-02-19T16:24:52.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15fb776740 20 client.16104 trim_cache size 0 max 16384 2024-02-19T16:24:52.665 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:24:52.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15e00a2230 msgr2=0x7f15e00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15e00a2230 0x7f15e00a46f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0cbb80 tx=0x7f15ec01bb30 comp rx=0 tx=0).stop 2024-02-19T16:24:52.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 msgr2=0x7f15f4197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 0x7f15f4197970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00e970 tx=0x7f15ec00ee30 comp rx=0 tx=0).stop 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 shutdown_connections 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15f410bb00 0x7f15f4197970 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15e00a2230 0x7f15e00a46f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f4103050 0x7f15f419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 --2- 172.21.15.8:0/3249823164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4102730 0x7f15f419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 >> 172.21.15.8:0/3249823164 conn(0x7f15f4076770 msgr2=0x7f15f40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 shutdown_connections 2024-02-19T16:24:52.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.658+0000 7f15f8ac9640 1 -- 172.21.15.8:0/3249823164 wait complete. 2024-02-19T16:24:52.670 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:52 smithi008 ceph-mon[32573]: pgmap v62: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 296 op/s 2024-02-19T16:24:52.671 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:24:52.671 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:24:52.744 INFO:tasks.workunit.client.0.smithi008.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-02-19T16:24:52.744 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-02-19T16:24:52.776 INFO:tasks.workunit.client.0.smithi008.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-02-19T16:24:52.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:52 smithi191 ceph-mon[37203]: pgmap v62: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 296 op/s 2024-02-19T16:24:52.814 INFO:tasks.workunit.client.0.smithi008.stdout:checking if gcc static flag -static works... no 2024-02-19T16:24:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 -- 172.21.15.8:0/3585213154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3688101e60 msgr2=0x7f368810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 --2- 172.21.15.8:0/3585213154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3688101e60 0x7f368810e320 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ca0 tx=0x7f368002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 -- 172.21.15.8:0/3585213154 shutdown_connections 2024-02-19T16:24:52.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 --2- 172.21.15.8:0/3585213154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3688101e60 0x7f368810e320 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 --2- 172.21.15.8:0/3585213154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36881014a0 0x7f3688101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 --2- 172.21.15.8:0/3585213154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3688107560 0x7f3688107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 -- 172.21.15.8:0/3585213154 >> 172.21.15.8:0/3585213154 conn(0x7f36880788f0 msgr2=0x7f3688078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 -- 172.21.15.8:0/3585213154 shutdown_connections 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 -- 172.21.15.8:0/3585213154 wait complete. 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 Processor -- start 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.817+0000 7f368ef79640 1 -- start start 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 0x7f368819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3688101e60 0x7f368819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3688107560 0x7f3688197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3688197ea0 con 0x7f36881014a0 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3688198010 con 0x7f3688101e60 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881981d0 con 0x7f3688107560 2024-02-19T16:24:52.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 0x7f368819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 0x7f368819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42326/0 (socket says 172.21.15.8:42326) 2024-02-19T16:24:52.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 -- 172.21.15.8:0/2837252617 learned_addr learned my addr 172.21.15.8:0/2837252617 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368d516640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3688107560 0x7f3688197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f3687fff640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3688101e60 0x7f368819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 -- 172.21.15.8:0/2837252617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3688107560 msgr2=0x7f3688197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3688107560 0x7f3688197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 -- 172.21.15.8:0/2837252617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3688101e60 msgr2=0x7f368819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3688101e60 0x7f368819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 -- 172.21.15.8:0/2837252617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c009490 con 0x7f36881014a0 2024-02-19T16:24:52.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368cd15640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 0x7f368819d690 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f367c00ea60 tx=0x7f367c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f3685ffb640 1 -- 172.21.15.8:0/2837252617 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c00cde0 con 0x7f36881014a0 2024-02-19T16:24:52.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f3685ffb640 1 -- 172.21.15.8:0/2837252617 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f367c0035f0 con 0x7f36881014a0 2024-02-19T16:24:52.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3680009900 con 0x7f36881014a0 2024-02-19T16:24:52.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.818+0000 7f3685ffb640 1 -- 172.21.15.8:0/2837252617 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c010690 con 0x7f36881014a0 2024-02-19T16:24:52.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.819+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36881986a0 con 0x7f36881014a0 2024-02-19T16:24:52.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:52 smithi156 ceph-mon[36502]: pgmap v62: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 296 op/s 2024-02-19T16:24:52.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.821+0000 7f3685ffb640 1 -- 172.21.15.8:0/2837252617 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f367c003af0 con 0x7f36881014a0 2024-02-19T16:24:52.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.821+0000 7f3691c2a740 1 Processor -- start 2024-02-19T16:24:52.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.821+0000 7f3691c2a740 1 -- start start 2024-02-19T16:24:52.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3685ffb640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f36780a22b0 0x7f36780a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 0x55e789185630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e78918e750 0x55e789190b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3691c2a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e789140960 con 0x55e789185250 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3691c2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7890f4310 con 0x55e78918e750 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3691c2a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7890f1840 con 0x55e789185b70 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368d516640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f36780a22b0 0x7f36780a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 0x55e789185630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 0x55e789185630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42340/0 (socket says 172.21.15.8:42340) 2024-02-19T16:24:52.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 -- 172.21.15.8:0/185168011 learned_addr learned my addr 172.21.15.8:0/185168011 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3685ffb640 1 -- 172.21.15.8:0/2837252617 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f367c014070 con 0x7f36881014a0 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f3687fff640 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368d516640 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e78918e750 0x55e789190b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 -- 172.21.15.8:0/185168011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 msgr2=0x55e78918e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 -- 172.21.15.8:0/185168011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e78918e750 msgr2=0x55e789190b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e78918e750 0x55e789190b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 -- 172.21.15.8:0/185168011 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7890af060 con 0x55e789185250 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368d516640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f36780a22b0 0x7f36780a4770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3680004af0 tx=0x7f36800325f0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.822+0000 7f368cd15640 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 0x55e789185630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f367c0a3130 tx=0x7f367c09aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9f3b40e15b1b74 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3672ffd640 1 -- 172.21.15.8:0/185168011 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c0481a0 con 0x55e789185250 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3672ffd640 1 -- 172.21.15.8:0/185168011 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f367c09aec0 con 0x55e789185250 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3672ffd640 1 -- 172.21.15.8:0/185168011 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c0cbea0 con 0x55e789185250 2024-02-19T16:24:52.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 -- 172.21.15.8:0/185168011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 msgr2=0x55e789185630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 0x55e789185630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f367c0a3130 tx=0x7f367c09aa00 comp rx=0 tx=0).stop 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 -- 172.21.15.8:0/185168011 shutdown_connections 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e78918e750 0x55e789190b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 --2- 172.21.15.8:0/185168011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185250 0x55e789185630 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 -- 172.21.15.8:0/185168011 >> 172.21.15.8:0/185168011 conn(0x55e78917f980 msgr2=0x55e78917fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 -- 172.21.15.8:0/185168011 shutdown_connections 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.823+0000 7f3691c2a740 1 -- 172.21.15.8:0/185168011 wait complete. 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 -- start start 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 Processor -- start 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78918b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e78918c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7890ed020 con 0x55e78918e750 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e788dc0430 con 0x55e789185250 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e788d7ce00 con 0x55e789185b70 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f3691c2a740 10 client.0 _ll_register_callbacks cb 0x55e789094cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f368d516640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e78918c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.824+0000 7f368cd15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78918b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78918b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37652/0 (socket says 172.21.15.8:37652) 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 -- 172.21.15.8:0/2840328882 learned_addr learned my addr 172.21.15.8:0/2840328882 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3687fff640 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 -- 172.21.15.8:0/2840328882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 msgr2=0x55e78918bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 -- 172.21.15.8:0/2840328882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 msgr2=0x55e78918c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e78918c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 -- 172.21.15.8:0/2840328882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36881986a0 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f368cd15640 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78918b760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f367c00ea30 tx=0x7f367c097780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6171a0c55046306 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3667fff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3667fff640 1 -- 172.21.15.8:0/2840328882 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c003970 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3667fff640 1 -- 172.21.15.8:0/2840328882 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f367c09f010 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e788dec7e0 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3667fff640 1 -- 172.21.15.8:0/2840328882 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f367c0cc0a0 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3691c2a740 10 client.25889 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.825+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7890eb6a0 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3667fff640 1 -- 172.21.15.8:0/2840328882 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e788dec7e0 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3667fff640 1 -- 172.21.15.8:0/2840328882 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f367c096ca0 con 0x55e789185250 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3691c2a740 10 client.25889 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3691c2a740 10 client.25889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:24:52.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3691c2a740 20 client.25889 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7891a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3691c2a740 4 client.25889 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e788e29720 con 0x55e78919e830 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3687fff640 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7891a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3667fff640 10 client.25889 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.826+0000 7f3687fff640 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7891a0c20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc710129cc46a938 server_cookie=23e8eb6711ca68e8 in_seq=0 out_seq=0 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3667fff640 1 -- 172.21.15.8:0/2840328882 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e788e29720 con 0x55e78919e830 2024-02-19T16:24:52.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3667fff640 10 client.25889 handle_command_reply: tid=0 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 client.25889 shutdown 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 msgr2=0x55e78918b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78918b760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f367c00ea30 tx=0x7f367c097780 comp rx=0 tx=0).stop 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 shutdown_connections 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7891a0c20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e78918c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3667fff640 0 client.25889 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185b70 0x55e78918bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2840328882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78918b760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 >> 172.21.15.8:0/2840328882 conn(0x55e78917f980 msgr2=0x55e7891816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 shutdown_connections 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.827+0000 7f3691c2a740 1 -- 172.21.15.8:0/2840328882 wait complete. 2024-02-19T16:24:52.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.828+0000 7f3691c2a740 20 client.25889 trim_cache size 0 max 16384 2024-02-19T16:24:52.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 Processor -- start 2024-02-19T16:24:52.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 -- start start 2024-02-19T16:24:52.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78917aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 0x55e789199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e78918e750 0x55e78919a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e789072130 con 0x55e789185b70 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e78919e630 con 0x55e789185250 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3691c2a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e789180980 con 0x55e78918e750 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3687fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 0x55e789199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368d516640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e78918e750 0x55e78919a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3687fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 0x55e789199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42362/0 (socket says 172.21.15.8:42362) 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3687fff640 1 -- 172.21.15.8:0/1306811990 learned_addr learned my addr 172.21.15.8:0/1306811990 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368cd15640 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78917aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368d516640 1 -- 172.21.15.8:0/1306811990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 msgr2=0x55e78917aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368d516640 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78917aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368d516640 1 -- 172.21.15.8:0/1306811990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 msgr2=0x55e789199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368d516640 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 0x55e789199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368d516640 1 -- 172.21.15.8:0/1306811990 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e789087c20 con 0x55e78918e750 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f368cd15640 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78917aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.841+0000 7f3687fff640 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 0x55e789199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.842+0000 7f368d516640 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e78918e750 0x55e78919a010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f368002c700 tx=0x7f368002e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4596fd30613921b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.842+0000 7f36727fc640 1 -- 172.21.15.8:0/1306811990 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680034a90 con 0x55e78918e750 2024-02-19T16:24:52.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.842+0000 7f36727fc640 1 -- 172.21.15.8:0/1306811990 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3680034bf0 con 0x55e78918e750 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.842+0000 7f36727fc640 1 -- 172.21.15.8:0/1306811990 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36800042a0 con 0x55e78918e750 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- 172.21.15.8:0/1306811990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e78918e750 msgr2=0x55e78919a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e78918e750 0x55e78919a010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f368002c700 tx=0x7f368002e050 comp rx=0 tx=0).stop 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- 172.21.15.8:0/1306811990 shutdown_connections 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e78918e750 0x55e78919a010 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e789185b70 0x55e789199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- 172.21.15.8:0/1306811990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185250 0x55e78917aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- 172.21.15.8:0/1306811990 >> 172.21.15.8:0/1306811990 conn(0x55e78917f980 msgr2=0x55e78917eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- 172.21.15.8:0/1306811990 shutdown_connections 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- 172.21.15.8:0/1306811990 wait complete. 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- start start 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 Processor -- start 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185250 0x55e789189370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185b70 0x55e7891935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e789193b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7891940e0 con 0x55e78918e750 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e789194250 con 0x55e789185b70 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.843+0000 7f3691c2a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e789194410 con 0x55e789185250 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3691c2a740 10 client.0 _ll_register_callbacks cb 0x7f3674009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e789193b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368cd15640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185250 0x55e789189370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3687fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185b70 0x55e7891935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368cd15640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185250 0x55e789189370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51014/0 (socket says 172.21.15.8:51014) 2024-02-19T16:24:52.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368cd15640 1 -- 172.21.15.8:0/2617032264 learned_addr learned my addr 172.21.15.8:0/2617032264 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 -- 172.21.15.8:0/2617032264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185250 msgr2=0x55e789189370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185250 0x55e789189370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 -- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185b70 msgr2=0x55e7891935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185b70 0x55e7891935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 -- 172.21.15.8:0/2617032264 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7890eb6a0 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3687fff640 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185b70 0x55e7891935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f368d516640 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e789193b20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f36800344b0 tx=0x7f3680032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd0c2cd520adb82 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3667fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3667fff640 1 -- 172.21.15.8:0/2617032264 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680032ab0 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e78917b0f0 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3667fff640 1 -- 172.21.15.8:0/2617032264 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f368004e430 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3667fff640 1 -- 172.21.15.8:0/2617032264 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f368004e5f0 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3691c2a740 10 client.16122 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.844+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3674010430 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3667fff640 1 -- 172.21.15.8:0/2617032264 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e78917b0f0 con 0x55e78918e750 2024-02-19T16:24:52.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3667fff640 1 -- 172.21.15.8:0/2617032264 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3680055070 con 0x55e78918e750 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3691c2a740 10 client.16122 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3691c2a740 10 client.16122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3691c2a740 20 client.16122 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7892018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3691c2a740 4 client.16122 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f367c09aec0 con 0x55e78919e830 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.845+0000 7f368cd15640 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7892018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3667fff640 10 client.16122 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f368cd15640 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7892018e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62432370c28c7d0e server_cookie=9113a32ffc311c3b in_seq=0 out_seq=0 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3667fff640 1 -- 172.21.15.8:0/2617032264 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f367c09aec0 con 0x55e78919e830 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3667fff640 10 client.16122 handle_command_reply: tid=0 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 client.16122 shutdown 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 msgr2=0x55e789193b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e789193b20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f36800344b0 tx=0x7f3680032fc0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 shutdown_connections 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55e78919e830 0x55e7892018e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e78918e750 0x55e789193b20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e789185b70 0x55e7891935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 --2- 172.21.15.8:0/2617032264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e789185250 0x55e789189370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 >> 172.21.15.8:0/2617032264 conn(0x55e78917f980 msgr2=0x55e789180b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3667fff640 0 client.16122 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 shutdown_connections 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 1 -- 172.21.15.8:0/2617032264 wait complete. 2024-02-19T16:24:52.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.846+0000 7f3691c2a740 20 client.16122 trim_cache size 0 max 16384 2024-02-19T16:24:52.857 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f36780a22b0 msgr2=0x7f36780a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f36780a22b0 0x7f36780a4770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3680004af0 tx=0x7f36800325f0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 msgr2=0x7f368819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 0x7f368819d690 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f367c00ea60 tx=0x7f367c00ef20 comp rx=0 tx=0).stop 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 shutdown_connections 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f36780a22b0 0x7f36780a4770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3688107560 0x7f3688197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3688101e60 0x7f368819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 --2- 172.21.15.8:0/2837252617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f36881014a0 0x7f368819d690 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 >> 172.21.15.8:0/2837252617 conn(0x7f36880788f0 msgr2=0x7f36880fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 shutdown_connections 2024-02-19T16:24:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.847+0000 7f368ef79640 1 -- 172.21.15.8:0/2837252617 wait complete. 2024-02-19T16:24:52.859 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:24:52.860 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:24:52.870 INFO:tasks.workunit.client.0.smithi008.stdout:checking if gcc supports -c -o file.o... yes 2024-02-19T16:24:52.870 INFO:tasks.workunit.client.0.smithi008.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-02-19T16:24:52.876 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-02-19T16:24:52.932 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether -lc should be explicitly linked in... no 2024-02-19T16:24:52.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 -- 172.21.15.8:0/123181766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc100f00 msgr2=0x7f8ccc101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 --2- 172.21.15.8:0/123181766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc100f00 0x7f8ccc101380 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009930 tx=0x7f8cb802bc30 comp rx=0 tx=0).stop 2024-02-19T16:24:52.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 -- 172.21.15.8:0/123181766 shutdown_connections 2024-02-19T16:24:52.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 --2- 172.21.15.8:0/123181766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 --2- 172.21.15.8:0/123181766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc100f00 0x7f8ccc101380 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 --2- 172.21.15.8:0/123181766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ccc105390 0x7f8ccc105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 -- 172.21.15.8:0/123181766 >> 172.21.15.8:0/123181766 conn(0x7f8ccc075650 msgr2=0x7f8ccc075a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 -- 172.21.15.8:0/123181766 shutdown_connections 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.976+0000 7f8cd309a640 1 -- 172.21.15.8:0/123181766 wait complete. 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 Processor -- start 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 -- start start 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ccc100f00 0x7f8ccc19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ccc105390 0x7f8ccc1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc197ef0 con 0x7f8ccc1018c0 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc198060 con 0x7f8ccc100f00 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd309a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc198220 con 0x7f8ccc105390 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc19dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42388/0 (socket says 172.21.15.8:42388) 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8ccbfff640 1 -- 172.21.15.8:0/4240783819 learned_addr learned my addr 172.21.15.8:0/4240783819 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ccc100f00 0x7f8ccc19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8cd1637640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ccc105390 0x7f8ccc1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8ccbfff640 1 -- 172.21.15.8:0/4240783819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ccc105390 msgr2=0x7f8ccc1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.977+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ccc105390 0x7f8ccc1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8ccbfff640 1 -- 172.21.15.8:0/4240783819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ccc100f00 msgr2=0x7f8ccc19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ccc100f00 0x7f8ccc19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8ccbfff640 1 -- 172.21.15.8:0/4240783819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009490 con 0x7f8ccc1018c0 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc19dc30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8000c00 tx=0x7f8cb802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8cc9ffb640 1 -- 172.21.15.8:0/4240783819 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb802dea0 con 0x7f8ccc1018c0 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8cc9ffb640 1 -- 172.21.15.8:0/4240783819 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cb80042d0 con 0x7f8ccc1018c0 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb8009560 con 0x7f8ccc1018c0 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8cc9ffb640 1 -- 172.21.15.8:0/4240783819 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8003b50 con 0x7f8ccc1018c0 2024-02-19T16:24:52.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.978+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1a45b0 con 0x7f8ccc1018c0 2024-02-19T16:24:52.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.980+0000 7f8cc9ffb640 1 -- 172.21.15.8:0/4240783819 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8cb8003cb0 con 0x7f8ccc1018c0 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.980+0000 7f8cd5c76740 1 Processor -- start 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.980+0000 7f8cd5c76740 1 -- start start 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 0x55acebe4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe54210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe54750 0x55acebe56b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebe06960 con 0x55acebe4b250 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebdba310 con 0x55acebe54750 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebdb7840 con 0x55acebe4bb70 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 0x55acebe4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 0x55acebe4b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42396/0 (socket says 172.21.15.8:42396) 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 -- 172.21.15.8:0/2697641608 learned_addr learned my addr 172.21.15.8:0/2697641608 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cc9ffb640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8cbc0a22b0 0x7f8cbc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe54210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe54750 0x55acebe56b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8cbc0a22b0 0x7f8cbc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 -- 172.21.15.8:0/2697641608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 msgr2=0x55acebe54210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe54210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 -- 172.21.15.8:0/2697641608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe54750 msgr2=0x55acebe56b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe54750 0x55acebe56b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cc9ffb640 1 -- 172.21.15.8:0/4240783819 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8cb80f08c0 con 0x7f8ccc1018c0 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 -- 172.21.15.8:0/2697641608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acebd75060 con 0x55acebe4b250 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe54210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.981+0000 7f8cd1637640 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 0x55acebe4b630 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005c40 tx=0x7f8cc400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d6650865aea123 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cb6ffd640 1 -- 172.21.15.8:0/2697641608 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400fd30 con 0x55acebe4b250 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd1637640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8cbc0a22b0 0x7f8cbc0a4770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4008000 tx=0x7f8cc401f000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cb6ffd640 1 -- 172.21.15.8:0/2697641608 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc400fe90 con 0x55acebe4b250 2024-02-19T16:24:52.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cb6ffd640 1 -- 172.21.15.8:0/2697641608 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40109b0 con 0x55acebe4b250 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 -- 172.21.15.8:0/2697641608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 msgr2=0x55acebe4b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 0x55acebe4b630 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005c40 tx=0x7f8cc400f3c0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 -- 172.21.15.8:0/2697641608 shutdown_connections 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe54750 0x55acebe56b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe54210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/2697641608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4b250 0x55acebe4b630 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 -- 172.21.15.8:0/2697641608 >> 172.21.15.8:0/2697641608 conn(0x55acebe45980 msgr2=0x55acebe45d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 -- 172.21.15.8:0/2697641608 shutdown_connections 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.982+0000 7f8cd5c76740 1 -- 172.21.15.8:0/2697641608 wait complete. 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 -- start start 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 Processor -- start 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe54750 0x55acebe52230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebdb3020 con 0x55acebe54750 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aceba86430 con 0x55acebe4b250 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aceba42e00 con 0x55acebe4bb70 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd5c76740 10 client.0 _ll_register_callbacks cb 0x55acebd5acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe54750 0x55acebe52230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd0e36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe54750 0x55acebe52230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42404/0 (socket says 172.21.15.8:42404) 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8cd1637640 1 -- 172.21.15.8:0/1491662684 learned_addr learned my addr 172.21.15.8:0/1491662684 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.983+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 msgr2=0x55acebe51cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe51cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1491662684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe54750 msgr2=0x55acebe52230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:52.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe54750 0x55acebe52230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1491662684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc4013700 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe517b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc076c70 tx=0x7f8cc0012ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e205835627d9294 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cabfff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cabfff640 1 -- 172.21.15.8:0/1491662684 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0017070 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acebab27e0 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cabfff640 1 -- 172.21.15.8:0/1491662684 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8cc0004030 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cabfff640 1 -- 172.21.15.8:0/1491662684 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc00041f0 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd5c76740 10 client.25907 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55acebdb16a0 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cabfff640 1 -- 172.21.15.8:0/1491662684 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55acebab27e0 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.984+0000 7f8cabfff640 1 -- 172.21.15.8:0/1491662684 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8cc0003a00 con 0x55acebe4b250 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cd5c76740 10 client.25907 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cd5c76740 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cd5c76740 20 client.25907 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:52.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebe64c70 0x55acebe67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cd5c76740 4 client.25907 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acebaef720 con 0x55acebe64c70 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebe64c70 0x55acebe67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8cabfff640 10 client.25907 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.985+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebe64c70 0x55acebe67090 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb352426cedf357e server_cookie=f846d62884147203 in_seq=0 out_seq=0 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cabfff640 1 -- 172.21.15.8:0/1491662684 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55acebaef720 con 0x55acebe64c70 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cabfff640 10 client.25907 handle_command_reply: tid=0 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 client.25907 shutdown 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 msgr2=0x55acebe517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe517b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc076c70 tx=0x7f8cc0012ca0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 shutdown_connections 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe54750 0x55acebe52230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebe64c70 0x55acebe67090 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe4bb70 0x55acebe51cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1491662684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe517b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:52.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cabfff640 0 client.25907 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:52.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 >> 172.21.15.8:0/1491662684 conn(0x55acebe45980 msgr2=0x55acebe476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:52.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 shutdown_connections 2024-02-19T16:24:52.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1491662684 wait complete. 2024-02-19T16:24:52.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.986+0000 7f8cd5c76740 20 client.25907 trim_cache size 0 max 16384 2024-02-19T16:24:52.997 INFO:tasks.workunit.client.0.smithi008.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-02-19T16:24:52.997 INFO:tasks.workunit.client.0.smithi008.stdout:checking how to hardcode library paths into programs... immediate 2024-02-19T16:24:52.998 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether stripping libraries is possible... yes 2024-02-19T16:24:52.998 INFO:tasks.workunit.client.0.smithi008.stdout:checking if libtool supports shared libraries... yes 2024-02-19T16:24:52.998 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether to build shared libraries... yes 2024-02-19T16:24:52.999 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether to build static libraries... yes 2024-02-19T16:24:53.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 Processor -- start 2024-02-19T16:24:53.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 -- start start 2024-02-19T16:24:53.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebe5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebd38130 con 0x55acebe4bb70 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebe4c620 con 0x55acebe4b250 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:52.999+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebe60520 con 0x55acebe54750 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe52050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37716/0 (socket says 172.21.15.8:37716) 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebe5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1371102797 learned_addr learned my addr 172.21.15.8:0/1371102797 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1371102797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 msgr2=0x55acebe5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebe5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1371102797 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 msgr2=0x55acebe5fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe5fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 -- 172.21.15.8:0/1371102797 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acebd4dc20 con 0x55acebe4b250 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe52050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0003e60 tx=0x7f8cc000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c3df3a25d9decd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cb67fc640 1 -- 172.21.15.8:0/1371102797 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0017070 con 0x55acebe4b250 2024-02-19T16:24:53.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cb67fc640 1 -- 172.21.15.8:0/1371102797 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0018040 con 0x55acebe4b250 2024-02-19T16:24:53.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.000+0000 7f8cb67fc640 1 -- 172.21.15.8:0/1371102797 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0005550 con 0x55acebe4b250 2024-02-19T16:24:53.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1371102797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 msgr2=0x55acebe52050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:53.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe52050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0003e60 tx=0x7f8cc000f790 comp rx=0 tx=0).stop 2024-02-19T16:24:53.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1371102797 shutdown_connections 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebe5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/1371102797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe52050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1371102797 >> 172.21.15.8:0/1371102797 conn(0x55acebe45980 msgr2=0x55acebe6b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1371102797 shutdown_connections 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 -- 172.21.15.8:0/1371102797 wait complete. 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 -- start start 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.001+0000 7f8cd5c76740 1 Processor -- start 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe4ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebeb84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebeb8a50 con 0x55acebe4bb70 2024-02-19T16:24:53.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebeb8bc0 con 0x55acebe4b250 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebeb8d80 con 0x55acebe54750 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd5c76740 10 client.0 _ll_register_callbacks cb 0x7f8cc00099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebeb84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe4ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebeb84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51066/0 (socket says 172.21.15.8:51066) 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 -- 172.21.15.8:0/335145555 learned_addr learned my addr 172.21.15.8:0/335145555 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 -- 172.21.15.8:0/335145555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 msgr2=0x55acebe4e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe4e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 -- 172.21.15.8:0/335145555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 msgr2=0x55acebe4ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe4ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 -- 172.21.15.8:0/335145555 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc0021040 con 0x55acebe54750 2024-02-19T16:24:53.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8ccbfff640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cd1637640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebeb84c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40104a0 tx=0x7f8cc400fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df4b1b05faa157e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.002+0000 7f8cabfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cabfff640 1 -- 172.21.15.8:0/335145555 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4013320 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cc400fe90 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cabfff640 1 -- 172.21.15.8:0/335145555 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8cc40106a0 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cabfff640 1 -- 172.21.15.8:0/335145555 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc4010860 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 10 client.26008 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55acebd738c0 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cabfff640 1 -- 172.21.15.8:0/335145555 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc400fe90 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cabfff640 1 -- 172.21.15.8:0/335145555 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8cc4027960 con 0x55acebe54750 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 10 client.26008 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:53.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 10 client.26008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 20 client.26008 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebec17c0 0x55acebec3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 4 client.26008 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acebb3e890 con 0x55acebec17c0 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.003+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebec17c0 0x55acebec3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cabfff640 10 client.26008 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd0e36640 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebec17c0 0x55acebec3be0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369abce4776442dc server_cookie=626dda292a65b80f in_seq=0 out_seq=0 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cabfff640 1 -- 172.21.15.8:0/335145555 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acebb3e890 con 0x55acebec17c0 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cabfff640 10 client.26008 handle_command_reply: tid=0 2024-02-19T16:24:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 client.26008 shutdown 2024-02-19T16:24:53.011 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gcc... (cached) gcc 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 msgr2=0x55acebeb84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebeb84c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40104a0 tx=0x7f8cc400fd10 comp rx=0 tx=0).stop 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 shutdown_connections 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55acebec17c0 0x55acebec3be0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55acebe54750 0x55acebeb84c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cabfff640 0 client.26008 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55acebe4bb70 0x55acebe4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 --2- 172.21.15.8:0/335145555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acebe4b250 0x55acebe4e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 >> 172.21.15.8:0/335145555 conn(0x55acebe45980 msgr2=0x55acebe6b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 shutdown_connections 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 1 -- 172.21.15.8:0/335145555 wait complete. 2024-02-19T16:24:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.004+0000 7f8cd5c76740 20 client.26008 trim_cache size 0 max 16384 2024-02-19T16:24:53.013 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:24:53.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8cbc0a22b0 msgr2=0x7f8cbc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8cbc0a22b0 0x7f8cbc0a4770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4008000 tx=0x7f8cc401f000 comp rx=0 tx=0).stop 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 msgr2=0x7f8ccc19dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc19dc30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8000c00 tx=0x7f8cb802c670 comp rx=0 tx=0).stop 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 shutdown_connections 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8cbc0a22b0 0x7f8cbc0a4770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ccc105390 0x7f8ccc1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ccc1018c0 0x7f8ccc19dc30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 --2- 172.21.15.8:0/4240783819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ccc100f00 0x7f8ccc19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 >> 172.21.15.8:0/4240783819 conn(0x7f8ccc075650 msgr2=0x7f8ccc0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 shutdown_connections 2024-02-19T16:24:53.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:53.005+0000 7f8cd309a640 1 -- 172.21.15.8:0/4240783819 wait complete. 2024-02-19T16:24:53.033 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-02-19T16:24:53.034 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether gcc accepts -g... (cached) yes 2024-02-19T16:24:53.034 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-02-19T16:24:53.034 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gmake... /usr/bin/gmake 2024-02-19T16:24:53.034 INFO:tasks.workunit.client.0.smithi008.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-02-19T16:24:53.035 INFO:tasks.workunit.client.0.smithi008.stdout:checking for tar... /bin/tar 2024-02-19T16:24:53.035 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gzip... /bin/gzip 2024-02-19T16:24:53.053 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether gcc -MM is supported... yes 2024-02-19T16:24:53.053 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sort... /bin/sort 2024-02-19T16:24:53.053 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether ln -s works... yes 2024-02-19T16:24:53.053 INFO:tasks.workunit.client.0.smithi008.stdout:checking for rpm... /bin/rpm 2024-02-19T16:24:53.056 INFO:tasks.workunit.client.0.smithi008.stdout:checking for rpmbuild... no 2024-02-19T16:24:53.056 INFO:tasks.workunit.client.0.smithi008.stdout:checking for ANSI C header files... (cached) yes 2024-02-19T16:24:53.101 INFO:tasks.workunit.client.0.smithi008.stdout:checking assert.h usability... yes 2024-02-19T16:24:53.115 INFO:tasks.workunit.client.0.smithi008.stdout:checking assert.h presence... yes 2024-02-19T16:24:53.115 INFO:tasks.workunit.client.0.smithi008.stdout:checking for assert.h... yes 2024-02-19T16:24:53.161 INFO:tasks.workunit.client.0.smithi008.stdout:checking bstring.h usability... no 2024-02-19T16:24:53.190 INFO:tasks.workunit.client.0.smithi008.stdout:checking bstring.h presence... no 2024-02-19T16:24:53.191 INFO:tasks.workunit.client.0.smithi008.stdout:checking for bstring.h... no 2024-02-19T16:24:53.229 INFO:tasks.workunit.client.0.smithi008.stdout:checking libgen.h usability... yes 2024-02-19T16:24:53.244 INFO:tasks.workunit.client.0.smithi008.stdout:checking libgen.h presence... yes 2024-02-19T16:24:53.244 INFO:tasks.workunit.client.0.smithi008.stdout:checking for libgen.h... yes 2024-02-19T16:24:53.294 INFO:tasks.workunit.client.0.smithi008.stdout:checking dirent.h usability... yes 2024-02-19T16:24:53.309 INFO:tasks.workunit.client.0.smithi008.stdout:checking dirent.h presence... yes 2024-02-19T16:24:53.309 INFO:tasks.workunit.client.0.smithi008.stdout:checking for dirent.h... yes 2024-02-19T16:24:53.352 INFO:tasks.workunit.client.0.smithi008.stdout:checking errno.h usability... yes 2024-02-19T16:24:53.367 INFO:tasks.workunit.client.0.smithi008.stdout:checking errno.h presence... yes 2024-02-19T16:24:53.367 INFO:tasks.workunit.client.0.smithi008.stdout:checking for errno.h... yes 2024-02-19T16:24:53.414 INFO:tasks.workunit.client.0.smithi008.stdout:checking malloc.h usability... yes 2024-02-19T16:24:53.430 INFO:tasks.workunit.client.0.smithi008.stdout:checking malloc.h presence... yes 2024-02-19T16:24:53.430 INFO:tasks.workunit.client.0.smithi008.stdout:checking for malloc.h... yes 2024-02-19T16:24:53.477 INFO:tasks.workunit.client.0.smithi008.stdout:checking uuid.h usability... no 2024-02-19T16:24:53.500 INFO:tasks.workunit.client.0.smithi008.stdout:checking uuid.h presence... no 2024-02-19T16:24:53.500 INFO:tasks.workunit.client.0.smithi008.stdout:checking for uuid.h... no 2024-02-19T16:24:53.545 INFO:tasks.workunit.client.0.smithi008.stdout:checking uuid/uuid.h usability... yes 2024-02-19T16:24:53.560 INFO:tasks.workunit.client.0.smithi008.stdout:checking uuid/uuid.h presence... yes 2024-02-19T16:24:53.560 INFO:tasks.workunit.client.0.smithi008.stdout:checking for uuid/uuid.h... yes 2024-02-19T16:24:53.608 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/uuid.h usability... no 2024-02-19T16:24:53.634 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/uuid.h presence... no 2024-02-19T16:24:53.634 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/uuid.h... no 2024-02-19T16:24:53.673 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/file.h usability... yes 2024-02-19T16:24:53.687 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/file.h presence... yes 2024-02-19T16:24:53.687 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/file.h... yes 2024-02-19T16:24:53.737 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/fcntl.h usability... yes 2024-02-19T16:24:53.754 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/fcntl.h presence... yes 2024-02-19T16:24:53.754 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/fcntl.h... yes 2024-02-19T16:24:53.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:53 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2238586987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:24:53.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:53 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2238586987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:24:53.801 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/syssgi.h usability... no 2024-02-19T16:24:53.829 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/syssgi.h presence... no 2024-02-19T16:24:53.829 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/syssgi.h... no 2024-02-19T16:24:53.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:53 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2238586987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:24:53.924 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/param.h usability... yes 2024-02-19T16:24:53.944 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/param.h presence... yes 2024-02-19T16:24:53.944 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/param.h... yes 2024-02-19T16:24:53.958 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/stat.h... (cached) yes 2024-02-19T16:24:53.993 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/statvfs.h usability... yes 2024-02-19T16:24:54.009 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/statvfs.h presence... yes 2024-02-19T16:24:54.010 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/statvfs.h... yes 2024-02-19T16:24:54.018 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:24:54.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:24:54.018 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:24:54.053 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/time.h usability... yes 2024-02-19T16:24:54.071 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/time.h presence... yes 2024-02-19T16:24:54.071 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/time.h... yes 2024-02-19T16:24:54.115 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/ioctl.h usability... yes 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 -- 172.21.15.8:0/3201636261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a41056e0 msgr2=0x7fc7a410d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 --2- 172.21.15.8:0/3201636261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a41056e0 0x7fc7a410d2b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009cd0 tx=0x7fc79c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 -- 172.21.15.8:0/3201636261 shutdown_connections 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 --2- 172.21.15.8:0/3201636261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a41056e0 0x7fc7a410d2b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 --2- 172.21.15.8:0/3201636261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100fc0 0x7fc7a4105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 --2- 172.21.15.8:0/3201636261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a4100610 0x7fc7a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 -- 172.21.15.8:0/3201636261 >> 172.21.15.8:0/3201636261 conn(0x7fc7a4076770 msgr2=0x7fc7a4076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 -- 172.21.15.8:0/3201636261 shutdown_connections 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 -- 172.21.15.8:0/3201636261 wait complete. 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 Processor -- start 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.126+0000 7fc7aa965640 1 -- start start 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7aa965640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100610 0x7fc7a41020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7aa965640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a41056e0 0x7fc7a41025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7aa965640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 0x7fc7a41a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7aa965640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41a62e0 con 0x7fc7a4102b20 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7aa965640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41a6450 con 0x7fc7a41056e0 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7aa965640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41a6610 con 0x7fc7a4100610 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 0x7fc7a41a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a41056e0 0x7fc7a41025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100610 0x7fc7a41020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 0x7fc7a41a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42448/0 (socket says 172.21.15.8:42448) 2024-02-19T16:24:54.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2598727224 learned_addr learned my addr 172.21.15.8:0/2598727224 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 -- 172.21.15.8:0/2598727224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100610 msgr2=0x7fc7a41020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100610 0x7fc7a41020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 -- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 msgr2=0x7fc7a41a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 0x7fc7a41a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 -- 172.21.15.8:0/2598727224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009900 con 0x7fc7a41056e0 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 0x7fc7a41a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100610 0x7fc7a41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.127+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a41056e0 0x7fc7a41025e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00e9c0 tx=0x7fc78c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.128+0000 7fc7a17fa640 1 -- 172.21.15.8:0/2598727224 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00ccc0 con 0x7fc7a41056e0 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.128+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a41a68f0 con 0x7fc7a41056e0 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.128+0000 7fc7a17fa640 1 -- 172.21.15.8:0/2598727224 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc78c00ce20 con 0x7fc7a41056e0 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.128+0000 7fc7a17fa640 1 -- 172.21.15.8:0/2598727224 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c018880 con 0x7fc7a41056e0 2024-02-19T16:24:54.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.128+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a41a6e50 con 0x7fc7a41056e0 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7a17fa640 1 -- 172.21.15.8:0/2598727224 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fc78c0189e0 con 0x7fc7a41056e0 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 Processor -- start 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 -- start start 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd279760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd284d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd1e7f90 con 0x55a8cd279d30 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd1e54c0 con 0x55a8cd2828b0 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd1e0ca0 con 0x55a8cd279380 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7a17fa640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc7900a2260 0x7fc7900a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.130+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd279760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd279760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51084/0 (socket says 172.21.15.8:51084) 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2822e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42452/0 (socket says 172.21.15.8:42452) 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a8f02640 1 -- 172.21.15.8:0/569461029 learned_addr learned my addr 172.21.15.8:0/569461029 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd284d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc7900a2260 0x7fc7900a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a17fa640 1 -- 172.21.15.8:0/2598727224 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fc78c014070 con 0x7fc7a41056e0 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 -- 172.21.15.8:0/569461029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 msgr2=0x55a8cd279760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd279760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 -- 172.21.15.8:0/569461029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 msgr2=0x55a8cd284d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd284d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 -- 172.21.15.8:0/569461029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8cd1a30b0 con 0x55a8cd279d30 2024-02-19T16:24:54.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc7900a2260 0x7fc7900a4720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc79c033000 tx=0x7fc79c0095c0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.136 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/ioctl.h presence... yes 2024-02-19T16:24:54.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2822e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc794009fd0 tx=0x7fc7940118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92789e7c102602d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.136 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/ioctl.h... yes 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc79a7fc640 1 -- 172.21.15.8:0/569461029 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794004030 con 0x55a8cd279d30 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc79a7fc640 1 -- 172.21.15.8:0/569461029 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc794005500 con 0x55a8cd279d30 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.131+0000 7fc79a7fc640 1 -- 172.21.15.8:0/569461029 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794004c40 con 0x55a8cd279d30 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 -- 172.21.15.8:0/569461029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 msgr2=0x55a8cd2822e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2822e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc794009fd0 tx=0x7fc7940118d0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 -- 172.21.15.8:0/569461029 shutdown_connections 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd284d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2822e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/569461029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd279760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 -- 172.21.15.8:0/569461029 >> 172.21.15.8:0/569461029 conn(0x55a8cd273840 msgr2=0x55a8cd273c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 -- 172.21.15.8:0/569461029 shutdown_connections 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 -- 172.21.15.8:0/569461029 wait complete. 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.132+0000 7fc7ad61d740 1 -- start start 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 Processor -- start 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd27f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279d30 0x55a8cd27fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 0x55a8cd2802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd185f40 con 0x55a8cd2828b0 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cceb44b0 con 0x55a8cd279d30 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd166670 con 0x55a8cd279380 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7ad61d740 10 client.0 _ll_register_callbacks cb 0x55a8cd1d3f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:54.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 0x55a8cd2802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 0x55a8cd2802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42464/0 (socket says 172.21.15.8:42464) 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2740497649 learned_addr learned my addr 172.21.15.8:0/2740497649 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd27f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279d30 0x55a8cd27fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 msgr2=0x55a8cd27f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd27f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2740497649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279d30 msgr2=0x55a8cd27fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279d30 0x55a8cd27fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2740497649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc794009c00 con 0x55a8cd2828b0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 0x55a8cd2802d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc79c037040 tx=0x7fc79c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3128254dc23081 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.133+0000 7fc7837fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7837fe640 1 -- 172.21.15.8:0/2740497649 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c003b80 con 0x55a8cd2828b0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7837fe640 1 -- 172.21.15.8:0/2740497649 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fc79c003ce0 con 0x55a8cd2828b0 2024-02-19T16:24:54.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8cd1baf40 con 0x55a8cd2828b0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7837fe640 1 -- 172.21.15.8:0/2740497649 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c003ea0 con 0x55a8cd2828b0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 10 client.16152 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a8cd1df320 con 0x55a8cd2828b0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7837fe640 1 -- 172.21.15.8:0/2740497649 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc79c05b020 con 0x55a8cd2828b0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7837fe640 1 -- 172.21.15.8:0/2740497649 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc79c058070 con 0x55a8cd2828b0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 10 client.16152 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 20 client.16152 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd292ed0 0x55a8cd2952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 4 client.16152 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.134+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8cd1d7310 con 0x55a8cd292ed0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.135+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd292ed0 0x55a8cd2952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.135+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd292ed0 0x55a8cd2952f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66948debab9340b server_cookie=cd2b4a72759f7f8a in_seq=0 out_seq=0 2024-02-19T16:24:54.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.135+0000 7fc7837fe640 10 client.16152 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7837fe640 1 -- 172.21.15.8:0/2740497649 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a8cd1d7310 con 0x55a8cd292ed0 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7837fe640 10 client.16152 handle_command_reply: tid=0 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 client.16152 shutdown 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 msgr2=0x55a8cd2802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 0x55a8cd2802d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc79c037040 tx=0x7fc79c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 shutdown_connections 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd2828b0 0x55a8cd2802d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279d30 0x55a8cd27fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd292ed0 0x55a8cd2952f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2740497649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd27f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 >> 172.21.15.8:0/2740497649 conn(0x55a8cd273840 msgr2=0x55a8cd2757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7837fe640 0 client.16152 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.136+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 shutdown_connections 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.137+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2740497649 wait complete. 2024-02-19T16:24:54.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.137+0000 7fc7ad61d740 20 client.16152 trim_cache size 0 max 16384 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 Processor -- start 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 -- start start 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd281f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd2754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd26f110 con 0x55a8cd279d30 2024-02-19T16:24:54.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd278640 con 0x55a8cd2828b0 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd27dda0 con 0x55a8cd279380 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd281f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2800f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42472/0 (socket says 172.21.15.8:42472) 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a37fe640 1 -- 172.21.15.8:0/1531792037 learned_addr learned my addr 172.21.15.8:0/1531792037 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd281f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51100/0 (socket says 172.21.15.8:51100) 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd2754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 -- 172.21.15.8:0/1531792037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 msgr2=0x55a8cd2754e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd2754e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 -- 172.21.15.8:0/1531792037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 msgr2=0x55a8cd2800f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2800f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a3fff640 1 -- 172.21.15.8:0/1531792037 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8cd17bc00 con 0x55a8cd279380 2024-02-19T16:24:54.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2800f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.153+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd2754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.154+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd281f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55a8cd274ac0 tx=0x7fc794009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a87fe94a2c3865 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.154+0000 7fc799ffb640 1 -- 172.21.15.8:0/1531792037 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794030070 con 0x55a8cd279380 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.154+0000 7fc799ffb640 1 -- 172.21.15.8:0/1531792037 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc794004160 con 0x55a8cd279380 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.154+0000 7fc799ffb640 1 -- 172.21.15.8:0/1531792037 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794003b80 con 0x55a8cd279380 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- 172.21.15.8:0/1531792037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 msgr2=0x55a8cd281f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd281f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55a8cd274ac0 tx=0x7fc794009b50 comp rx=0 tx=0).stop 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- 172.21.15.8:0/1531792037 shutdown_connections 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd2828b0 0x55a8cd2754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd2800f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/1531792037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd279380 0x55a8cd281f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- 172.21.15.8:0/1531792037 >> 172.21.15.8:0/1531792037 conn(0x55a8cd273840 msgr2=0x55a8cd274dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- 172.21.15.8:0/1531792037 shutdown_connections 2024-02-19T16:24:54.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- 172.21.15.8:0/1531792037 wait complete. 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- start start 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 Processor -- start 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279380 0x55a8cd28a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd28a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 0x55a8cd286800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd286ed0 con 0x55a8cd279d30 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd287040 con 0x55a8cd279380 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cd287200 con 0x55a8cd2828b0 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.155+0000 7fc7ad61d740 10 client.0 _ll_register_callbacks cb 0x7fc78c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 0x55a8cd286800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd28a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279380 0x55a8cd28a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 0x55a8cd286800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51110/0 (socket says 172.21.15.8:51110) 2024-02-19T16:24:54.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2307893694 learned_addr learned my addr 172.21.15.8:0/2307893694 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2307893694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279380 msgr2=0x55a8cd28a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279380 0x55a8cd28a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2307893694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 msgr2=0x55a8cd28a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd28a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 -- 172.21.15.8:0/2307893694 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a41a6e50 con 0x55a8cd2828b0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a37fe640 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd28a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7a8f02640 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 0x55a8cd286800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009430 tx=0x7fc79c032e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885b7129b55949e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7837fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7837fe640 1 -- 172.21.15.8:0/2307893694 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c009b20 con 0x55a8cd2828b0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8cd206900 con 0x55a8cd2828b0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7837fe640 1 -- 172.21.15.8:0/2307893694 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fc79c05c080 con 0x55a8cd2828b0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7837fe640 1 -- 172.21.15.8:0/2307893694 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c050670 con 0x55a8cd2828b0 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7ad61d740 10 client.26026 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:54.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.156+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a8cd1df320 con 0x55a8cd2828b0 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7837fe640 1 -- 172.21.15.8:0/2307893694 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a8cd206900 con 0x55a8cd2828b0 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7837fe640 1 -- 172.21.15.8:0/2307893694 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc79c05e070 con 0x55a8cd2828b0 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7ad61d740 10 client.26026 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7ad61d740 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7ad61d740 20 client.26026 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd2ef860 0x55a8cd2f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7ad61d740 4 client.26026 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a8ccee0860 con 0x55a8cd2ef860 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd2ef860 0x55a8cd2f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7837fe640 10 client.26026 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.157+0000 7fc7a3fff640 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd2ef860 0x55a8cd2f1c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d3825e38db74bc server_cookie=262dca1777a71bc1 in_seq=0 out_seq=0 2024-02-19T16:24:54.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7837fe640 1 -- 172.21.15.8:0/2307893694 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a8ccee0860 con 0x55a8cd2ef860 2024-02-19T16:24:54.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7837fe640 10 client.26026 handle_command_reply: tid=0 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 client.26026 shutdown 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 msgr2=0x55a8cd286800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 0x55a8cd286800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009430 tx=0x7fc79c032e30 comp rx=0 tx=0).stop 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 shutdown_connections 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a8cd2ef860 0x55a8cd2f1c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cd2828b0 0x55a8cd286800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a8cd279d30 0x55a8cd28a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7837fe640 0 client.26026 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 --2- 172.21.15.8:0/2307893694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8cd279380 0x55a8cd28a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 >> 172.21.15.8:0/2307893694 conn(0x55a8cd273840 msgr2=0x55a8cd275660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 shutdown_connections 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 1 -- 172.21.15.8:0/2307893694 wait complete. 2024-02-19T16:24:54.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.159+0000 7fc7ad61d740 20 client.26026 trim_cache size 0 max 16384 2024-02-19T16:24:54.167 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:24:54.167 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:24:54.167 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "1720634c-7ab9-4f34-bb9b-d607d0e4067e", 2024-02-19T16:24:54.167 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:24:54.167 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc7900a2260 msgr2=0x7fc7900a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc7900a2260 0x7fc7900a4720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc79c033000 tx=0x7fc79c0095c0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a41056e0 msgr2=0x7fc7a41025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a41056e0 0x7fc7a41025e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00e9c0 tx=0x7fc78c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 shutdown_connections 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7a4102b20 0x7fc7a41a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7a41056e0 0x7fc7a41025e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc7900a2260 0x7fc7900a4720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 --2- 172.21.15.8:0/2598727224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a4100610 0x7fc7a41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 >> 172.21.15.8:0/2598727224 conn(0x7fc7a4076770 msgr2=0x7fc7a4069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 shutdown_connections 2024-02-19T16:24:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.160+0000 7fc7aa965640 1 -- 172.21.15.8:0/2598727224 wait complete. 2024-02-19T16:24:54.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:24:54.172 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:24:54.193 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/wait.h usability... yes 2024-02-19T16:24:54.212 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/wait.h presence... yes 2024-02-19T16:24:54.213 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/wait.h... yes 2024-02-19T16:24:54.283 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/types.h... (cached) yes 2024-02-19T16:24:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:checking for strings.h... (cached) yes 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 -- 172.21.15.8:0/1556230275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d54100f90 msgr2=0x7f0d54104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 --2- 172.21.15.8:0/1556230275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d54100f90 0x7f0d54104fd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009890 tx=0x7f0d3c02bb70 comp rx=0 tx=0).stop 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 -- 172.21.15.8:0/1556230275 shutdown_connections 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 --2- 172.21.15.8:0/1556230275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d541056b0 0x7f0d5410d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 --2- 172.21.15.8:0/1556230275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d54100f90 0x7f0d54104fd0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 --2- 172.21.15.8:0/1556230275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d541005e0 0x7f0d541009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 -- 172.21.15.8:0/1556230275 >> 172.21.15.8:0/1556230275 conn(0x7f0d540775a0 msgr2=0x7f0d540779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 -- 172.21.15.8:0/1556230275 shutdown_connections 2024-02-19T16:24:54.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 -- 172.21.15.8:0/1556230275 wait complete. 2024-02-19T16:24:54.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 Processor -- start 2024-02-19T16:24:54.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.299+0000 7f0d59a50640 1 -- start start 2024-02-19T16:24:54.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d59a50640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 0x7f0d54196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d59a50640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d541056b0 0x7f0d54196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d59a50640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d541971a0 0x7f0d541a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d59a50640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d541a62e0 con 0x7f0d541005e0 2024-02-19T16:24:54.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d59a50640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d541a6420 con 0x7f0d541971a0 2024-02-19T16:24:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d59a50640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d541a6560 con 0x7f0d541056b0 2024-02-19T16:24:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 0x7f0d54196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 0x7f0d54196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42484/0 (socket says 172.21.15.8:42484) 2024-02-19T16:24:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 -- 172.21.15.8:0/853456894 learned_addr learned my addr 172.21.15.8:0/853456894 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d541056b0 0x7f0d54196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d537fe640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d541971a0 0x7f0d541a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 -- 172.21.15.8:0/853456894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d541056b0 msgr2=0x7f0d54196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d541056b0 0x7f0d54196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 -- 172.21.15.8:0/853456894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d541971a0 msgr2=0x7f0d541a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d541971a0 0x7f0d541a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 -- 172.21.15.8:0/853456894 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44009560 con 0x7f0d541005e0 2024-02-19T16:24:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.300+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 0x7f0d54196720 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400e9f0 tx=0x7f0d4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.301+0000 7f0d58a4e640 1 -- 172.21.15.8:0/853456894 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400cde0 con 0x7f0d541005e0 2024-02-19T16:24:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.301+0000 7f0d58a4e640 1 -- 172.21.15.8:0/853456894 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d440035f0 con 0x7f0d541005e0 2024-02-19T16:24:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.301+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3c009490 con 0x7f0d541005e0 2024-02-19T16:24:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.301+0000 7f0d58a4e640 1 -- 172.21.15.8:0/853456894 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44010670 con 0x7f0d541005e0 2024-02-19T16:24:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.301+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d541a6be0 con 0x7f0d541005e0 2024-02-19T16:24:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.303+0000 7f0d58a4e640 1 -- 172.21.15.8:0/853456894 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f0d44003ad0 con 0x7f0d541005e0 2024-02-19T16:24:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 Processor -- start 2024-02-19T16:24:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 -- start start 2024-02-19T16:24:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68e2b0 0x56166b68e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b697270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 0x56166b699ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b5fd280 con 0x56166b68ebd0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b5fa7b0 con 0x56166b68e2b0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b5f5f90 con 0x56166b6977b0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d58a4e640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0d400a22b0 0x7f0d400a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d537fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68e2b0 0x56166b68e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d52ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b697270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 0x56166b699ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 0x56166b699ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51148/0 (socket says 172.21.15.8:51148) 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/2620719384 learned_addr learned my addr 172.21.15.8:0/2620719384 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d58a4e640 1 -- 172.21.15.8:0/853456894 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f0d44014070 con 0x7f0d541005e0 2024-02-19T16:24:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.304+0000 7f0d537fe640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0d400a22b0 0x7f0d400a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/2620719384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68e2b0 msgr2=0x56166b68e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68e2b0 0x56166b68e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/2620719384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 msgr2=0x56166b697270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b697270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/2620719384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166b5b4570 con 0x56166b6977b0 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 0x56166b699ba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009860 tx=0x7f0d3c0323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0899dfbf3c375d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d537fe640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0d400a22b0 0x7f0d400a4770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c005fd0 tx=0x7f0d4c009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d49ffb640 1 -- 172.21.15.8:0/2620719384 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c004020 con 0x56166b6977b0 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d49ffb640 1 -- 172.21.15.8:0/2620719384 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c034440 con 0x56166b6977b0 2024-02-19T16:24:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.305+0000 7f0d49ffb640 1 -- 172.21.15.8:0/2620719384 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c02c9e0 con 0x56166b6977b0 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 -- 172.21.15.8:0/2620719384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 msgr2=0x56166b699ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 0x56166b699ba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009860 tx=0x7f0d3c0323c0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 -- 172.21.15.8:0/2620719384 shutdown_connections 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b6977b0 0x56166b699ba0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b697270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/2620719384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68e2b0 0x56166b68e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 -- 172.21.15.8:0/2620719384 >> 172.21.15.8:0/2620719384 conn(0x56166b688990 msgr2=0x56166b688d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 -- 172.21.15.8:0/2620719384 shutdown_connections 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 -- 172.21.15.8:0/2620719384 wait complete. 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.306+0000 7f0d5c61f740 1 -- start start 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 Processor -- start 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 0x56166b697160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68ebd0 0x56166b6944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b694a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b2c9430 con 0x56166b68e2b0 2024-02-19T16:24:54.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b285e00 con 0x56166b6977b0 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b57afd0 con 0x56166b68ebd0 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d5c61f740 10 client.0 _ll_register_callbacks cb 0x56166b5d0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 0x56166b697160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 0x56166b697160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42500/0 (socket says 172.21.15.8:42500) 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 -- 172.21.15.8:0/40540980 learned_addr learned my addr 172.21.15.8:0/40540980 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68ebd0 0x56166b6944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d537fe640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b694a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 -- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68ebd0 msgr2=0x56166b6944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68ebd0 0x56166b6944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 -- 172.21.15.8:0/40540980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 msgr2=0x56166b694a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b694a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.307+0000 7f0d52ffd640 1 -- 172.21.15.8:0/40540980 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5410da90 con 0x56166b68e2b0 2024-02-19T16:24:54.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 0x56166b697160 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0d440cbbf0 tx=0x7f0d440663d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913d4c8bd9e624fe server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:54.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d32ffd640 1 -- 172.21.15.8:0/40540980 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440ce5f0 con 0x56166b68e2b0 2024-02-19T16:24:54.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d32ffd640 1 -- 172.21.15.8:0/40540980 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f0d440a35d0 con 0x56166b68e2b0 2024-02-19T16:24:54.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56166b2f57e0 con 0x56166b68e2b0 2024-02-19T16:24:54.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d32ffd640 1 -- 172.21.15.8:0/40540980 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d4409ae30 con 0x56166b68e2b0 2024-02-19T16:24:54.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d5c61f740 10 client.16170 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:54.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56166b5f4610 con 0x56166b68e2b0 2024-02-19T16:24:54.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d32ffd640 1 -- 172.21.15.8:0/40540980 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d440b7020 con 0x56166b68e2b0 2024-02-19T16:24:54.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d32ffd640 1 -- 172.21.15.8:0/40540980 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0d4401f060 con 0x56166b68e2b0 2024-02-19T16:24:54.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d5c61f740 10 client.16170 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:54.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.308+0000 7f0d5c61f740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d5c61f740 20 client.16170 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b6a78b0 0x56166b6a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d5c61f740 4 client.16170 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:24:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56166b332720 con 0x56166b6a78b0 2024-02-19T16:24:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b6a78b0 0x56166b6a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d32ffd640 10 client.16170 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.309+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b6a78b0 0x56166b6a9ca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606711e01bf275e1 server_cookie=64f5655941cd8687 in_seq=0 out_seq=0 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.310+0000 7f0d32ffd640 1 -- 172.21.15.8:0/40540980 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56166b332720 con 0x56166b6a78b0 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.310+0000 7f0d32ffd640 10 client.16170 handle_command_reply: tid=0 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.310+0000 7f0d5c61f740 1 client.16170 shutdown 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 msgr2=0x56166b697160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 0x56166b697160 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0d440cbbf0 tx=0x7f0d440663d0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 shutdown_connections 2024-02-19T16:24:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b694a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b6a78b0 0x56166b6a9ca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68ebd0 0x56166b6944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/40540980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68e2b0 0x56166b697160 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 >> 172.21.15.8:0/40540980 conn(0x56166b688990 msgr2=0x56166b68a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d32ffd640 0 client.16170 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 shutdown_connections 2024-02-19T16:24:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 1 -- 172.21.15.8:0/40540980 wait complete. 2024-02-19T16:24:54.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.311+0000 7f0d5c61f740 20 client.16170 trim_cache size 0 max 16384 2024-02-19T16:24:54.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 Processor -- start 2024-02-19T16:24:54.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 -- start start 2024-02-19T16:24:54.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b694490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 0x56166b696c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 0x56166b697160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b68d3a0 con 0x56166b6977b0 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b68df10 con 0x56166b68ebd0 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b699f20 con 0x56166b68e2b0 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 0x56166b697160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 0x56166b696c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 0x56166b697160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42510/0 (socket says 172.21.15.8:42510) 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d52ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b694490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 0x56166b696c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:37826/0 (socket says 172.21.15.8:37826) 2024-02-19T16:24:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 -- 172.21.15.8:0/3171781353 learned_addr learned my addr 172.21.15.8:0/3171781353 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 -- 172.21.15.8:0/3171781353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 msgr2=0x56166b694490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b694490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 -- 172.21.15.8:0/3171781353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 msgr2=0x56166b696c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 0x56166b696c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d537fe640 1 -- 172.21.15.8:0/3171781353 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166b590ac0 con 0x56166b6977b0 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.330+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 0x56166b696c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.331+0000 7f0d537fe640 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 0x56166b697160 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00fe00 tx=0x7f0d4c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99caa5c9a76cb3b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.331+0000 7f0d497fa640 1 -- 172.21.15.8:0/3171781353 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c025040 con 0x56166b6977b0 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.331+0000 7f0d497fa640 1 -- 172.21.15.8:0/3171781353 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d4c01f990 con 0x56166b6977b0 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.331+0000 7f0d497fa640 1 -- 172.21.15.8:0/3171781353 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c005530 con 0x56166b6977b0 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.331+0000 7f0d5c61f740 1 -- 172.21.15.8:0/3171781353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 msgr2=0x56166b697160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.331+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 0x56166b697160 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00fe00 tx=0x7f0d4c018000 comp rx=0 tx=0).stop 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- 172.21.15.8:0/3171781353 shutdown_connections 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b6977b0 0x56166b697160 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b68ebd0 0x56166b696c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/3171781353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b694490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- 172.21.15.8:0/3171781353 >> 172.21.15.8:0/3171781353 conn(0x56166b688990 msgr2=0x56166b693d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- 172.21.15.8:0/3171781353 shutdown_connections 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- 172.21.15.8:0/3171781353 wait complete. 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- start start 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 Processor -- start 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b69a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b69ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b69b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b692430 con 0x56166b68ebd0 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b6925a0 con 0x56166b6977b0 2024-02-19T16:24:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166b703170 con 0x56166b68e2b0 2024-02-19T16:24:54.343 INFO:tasks.workunit.client.0.smithi008.stdout:checking err.h usability... yes 2024-02-19T16:24:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d5c61f740 10 client.0 _ll_register_callbacks cb 0x7f0d4409a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:24:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b69ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b69ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42524/0 (socket says 172.21.15.8:42524) 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/1930474961 learned_addr learned my addr 172.21.15.8:0/1930474961 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.332+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b69a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d537fe640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b69b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 msgr2=0x56166b69a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b69a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/1930474961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 msgr2=0x56166b69b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b69b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d4a7fc640 1 -- 172.21.15.8:0/1930474961 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56166b5b4570 con 0x56166b68ebd0 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d537fe640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b69b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d4a7fc640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b69ab10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005b60 tx=0x7f0d3c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9be2b0b0101dde5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:24:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d32ffd640 1 -- 172.21.15.8:0/1930474961 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c04d040 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d32ffd640 1 -- 172.21.15.8:0/1930474961 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f0d3c04c040 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d3c034440 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d32ffd640 1 -- 172.21.15.8:0/1930474961 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d3c053030 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d5c61f740 10 client.16176 fetch_fsmap learned FSMap version 27 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f0d4c019040 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.333+0000 7f0d32ffd640 1 -- 172.21.15.8:0/1930474961 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c034440 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d32ffd640 1 -- 172.21.15.8:0/1930474961 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0d3c050050 con 0x56166b68ebd0 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d5c61f740 10 client.16176 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d5c61f740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d5c61f740 20 client.16176 populate_metadata read hostname 'smithi008' 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b704770 0x56166b706b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d5c61f740 4 client.16176 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:24:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56166b381890 con 0x56166b704770 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b704770 0x56166b706b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d52ffd640 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b704770 0x56166b706b90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f9017d39beb58e server_cookie=84cdcfbc34cd941c in_seq=0 out_seq=0 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.334+0000 7f0d32ffd640 10 client.16176 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d32ffd640 1 -- 172.21.15.8:0/1930474961 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x56166b381890 con 0x56166b704770 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d32ffd640 10 client.16176 handle_command_reply: tid=0 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 client.16176 shutdown 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 msgr2=0x56166b69ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b69ab10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005b60 tx=0x7f0d3c005770 comp rx=0 tx=0).stop 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 shutdown_connections 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56166b6977b0 0x56166b69b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56166b68ebd0 0x56166b69ab10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56166b704770 0x56166b706b90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 --2- 172.21.15.8:0/1930474961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56166b68e2b0 0x56166b69a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 >> 172.21.15.8:0/1930474961 conn(0x56166b688990 msgr2=0x56166b6871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d32ffd640 0 client.16176 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:24:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 shutdown_connections 2024-02-19T16:24:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 1 -- 172.21.15.8:0/1930474961 wait complete. 2024-02-19T16:24:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.335+0000 7f0d5c61f740 20 client.16176 trim_cache size 0 max 16384 2024-02-19T16:24:54.350 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:24:54.350 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2848 inodes in the stack)", 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: "1720634c-7ab9-4f34-bb9b-d607d0e4067e": { 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "1720634c-7ab9-4f34-bb9b-d607d0e4067e", 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:24:54.351 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:24:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0d400a22b0 msgr2=0x7f0d400a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0d400a22b0 0x7f0d400a4770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c005fd0 tx=0x7f0d4c009500 comp rx=0 tx=0).stop 2024-02-19T16:24:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 msgr2=0x7f0d54196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:24:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 0x7f0d54196720 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400e9f0 tx=0x7f0d4400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 shutdown_connections 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d541971a0 0x7f0d541a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0d400a22b0 0x7f0d400a4770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d541056b0 0x7f0d54196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 --2- 172.21.15.8:0/853456894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d541005e0 0x7f0d54196720 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 >> 172.21.15.8:0/853456894 conn(0x7f0d540775a0 msgr2=0x7f0d540fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 shutdown_connections 2024-02-19T16:24:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:24:54.336+0000 7f0d59a50640 1 -- 172.21.15.8:0/853456894 wait complete. 2024-02-19T16:24:54.353 INFO:tasks.cephfs.filesystem:scrub status for tag:1720634c-7ab9-4f34-bb9b-d607d0e4067e - {'path': '/', 'tag': '1720634c-7ab9-4f34-bb9b-d607d0e4067e', 'options': 'recursive,force'} 2024-02-19T16:24:54.354 INFO:tasks.workunit.client.0.smithi008.stdout:checking err.h presence... yes 2024-02-19T16:24:54.354 INFO:tasks.workunit.client.0.smithi008.stdout:checking for err.h... yes 2024-02-19T16:24:54.402 INFO:tasks.workunit.client.0.smithi008.stdout:checking linux/falloc.h usability... yes 2024-02-19T16:24:54.416 INFO:tasks.workunit.client.0.smithi008.stdout:checking linux/falloc.h presence... yes 2024-02-19T16:24:54.416 INFO:tasks.workunit.client.0.smithi008.stdout:checking for linux/falloc.h... yes 2024-02-19T16:24:54.472 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-02-19T16:24:54.498 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-02-19T16:24:54.498 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/fs/xfs_fsops.h... no 2024-02-19T16:24:54.542 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/fs/xfs_itable.h usability... no 2024-02-19T16:24:54.570 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/fs/xfs_itable.h presence... no 2024-02-19T16:24:54.570 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/fs/xfs_itable.h... no 2024-02-19T16:24:54.613 INFO:tasks.workunit.client.0.smithi008.stdout:checking xfs/platform_defs.h usability... no 2024-02-19T16:24:54.639 INFO:tasks.workunit.client.0.smithi008.stdout:checking xfs/platform_defs.h presence... no 2024-02-19T16:24:54.639 INFO:tasks.workunit.client.0.smithi008.stdout:checking for xfs/platform_defs.h... no 2024-02-19T16:24:54.687 INFO:tasks.workunit.client.0.smithi008.stdout:checking btrfs/ioctl.h usability... no 2024-02-19T16:24:54.715 INFO:tasks.workunit.client.0.smithi008.stdout:checking btrfs/ioctl.h presence... no 2024-02-19T16:24:54.715 INFO:tasks.workunit.client.0.smithi008.stdout:checking for btrfs/ioctl.h... no 2024-02-19T16:24:54.762 INFO:tasks.workunit.client.0.smithi008.stdout:checking cifs/ioctl.h usability... no 2024-02-19T16:24:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:54 smithi008 ceph-mon[32573]: pgmap v63: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 296 op/s 2024-02-19T16:24:54.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:54 smithi191 ceph-mon[37203]: pgmap v63: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 296 op/s 2024-02-19T16:24:54.792 INFO:tasks.workunit.client.0.smithi008.stdout:checking cifs/ioctl.h presence... no 2024-02-19T16:24:54.792 INFO:tasks.workunit.client.0.smithi008.stdout:checking for cifs/ioctl.h... no 2024-02-19T16:24:54.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:54 smithi156 ceph-mon[36502]: pgmap v63: 129 pgs: 129 active+clean; 53 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 296 op/s 2024-02-19T16:24:54.832 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/mman.h usability... yes 2024-02-19T16:24:54.850 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/mman.h presence... yes 2024-02-19T16:24:54.851 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/mman.h... yes 2024-02-19T16:24:54.891 INFO:tasks.workunit.client.0.smithi008.stdout:checking for xfs/xfs_log_format.h... no 2024-02-19T16:24:54.944 INFO:tasks.workunit.client.0.smithi008.stdout:checking for uuid_compare... no 2024-02-19T16:24:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:checking for uuid_compare in -luuid... yes 2024-02-19T16:24:55.072 INFO:tasks.workunit.client.0.smithi008.stdout:checking for xfs/xfs.h... yes 2024-02-19T16:24:55.117 INFO:tasks.workunit.client.0.smithi008.stdout:checking for xfs/libxfs.h... no 2024-02-19T16:24:55.146 INFO:tasks.workunit.client.0.smithi008.stdout:checking whether xlog_assign_lsn is declared... no 2024-02-19T16:24:55.186 INFO:tasks.workunit.client.0.smithi008.stdout:checking for xfs/xqm.h... yes 2024-02-19T16:24:55.237 INFO:tasks.workunit.client.0.smithi008.stdout:checking xfsctl from xfs/xfs.h... ok 2024-02-19T16:24:55.274 INFO:tasks.workunit.client.0.smithi008.stdout:checking xfs/handle.h usability... yes 2024-02-19T16:24:55.290 INFO:tasks.workunit.client.0.smithi008.stdout:checking xfs/handle.h presence... yes 2024-02-19T16:24:55.290 INFO:tasks.workunit.client.0.smithi008.stdout:checking for xfs/handle.h... yes 2024-02-19T16:24:55.331 INFO:tasks.workunit.client.0.smithi008.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-02-19T16:24:55.403 INFO:tasks.workunit.client.0.smithi008.stdout:checking attr/attributes.h usability... yes 2024-02-19T16:24:55.419 INFO:tasks.workunit.client.0.smithi008.stdout:checking attr/attributes.h presence... yes 2024-02-19T16:24:55.419 INFO:tasks.workunit.client.0.smithi008.stdout:checking for attr/attributes.h... yes 2024-02-19T16:24:55.469 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/attributes.h usability... no 2024-02-19T16:24:55.498 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/attributes.h presence... no 2024-02-19T16:24:55.498 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/attributes.h... no 2024-02-19T16:24:55.537 INFO:tasks.workunit.client.0.smithi008.stdout:checking for attr_list in -lattr... yes 2024-02-19T16:24:55.577 INFO:tasks.workunit.client.0.smithi008.stdout:checking for attr_set in -lattr... yes 2024-02-19T16:24:55.608 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/acl.h usability... yes 2024-02-19T16:24:55.627 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/acl.h presence... yes 2024-02-19T16:24:55.627 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/acl.h... yes 2024-02-19T16:24:55.657 INFO:tasks.workunit.client.0.smithi008.stdout:checking acl/libacl.h usability... yes 2024-02-19T16:24:55.674 INFO:tasks.workunit.client.0.smithi008.stdout:checking acl/libacl.h presence... yes 2024-02-19T16:24:55.674 INFO:tasks.workunit.client.0.smithi008.stdout:checking for acl/libacl.h... yes 2024-02-19T16:24:55.714 INFO:tasks.workunit.client.0.smithi008.stdout:checking for acl_init in -lacl... yes 2024-02-19T16:24:55.767 INFO:tasks.workunit.client.0.smithi008.stdout:checking gdbm-ndbm.h usability... no 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:55.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.792 INFO:tasks.workunit.client.0.smithi008.stdout:checking gdbm-ndbm.h presence... no 2024-02-19T16:24:55.792 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gdbm-ndbm.h... no 2024-02-19T16:24:55.825 INFO:tasks.workunit.client.0.smithi008.stdout:checking gdbm/ndbm.h usability... no 2024-02-19T16:24:55.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:24:55.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:55.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:55.849 INFO:tasks.workunit.client.0.smithi008.stdout:checking gdbm/ndbm.h presence... no 2024-02-19T16:24:55.850 INFO:tasks.workunit.client.0.smithi008.stdout:checking for gdbm/ndbm.h... no 2024-02-19T16:24:55.885 INFO:tasks.workunit.client.0.smithi008.stdout:checking ndbm.h usability... no 2024-02-19T16:24:55.913 INFO:tasks.workunit.client.0.smithi008.stdout:checking ndbm.h presence... no 2024-02-19T16:24:55.913 INFO:tasks.workunit.client.0.smithi008.stdout:checking for ndbm.h... no 2024-02-19T16:24:55.945 INFO:tasks.workunit.client.0.smithi008.stdout:checking libaio.h usability... yes 2024-02-19T16:24:55.961 INFO:tasks.workunit.client.0.smithi008.stdout:checking libaio.h presence... yes 2024-02-19T16:24:55.961 INFO:tasks.workunit.client.0.smithi008.stdout:checking for libaio.h... yes 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout:checking for dm_make_handle in -ldm... no 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout: 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout:WARNING: could not find a valid DMAPI base library. 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout:If you want DMAPI support please install the data migration 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout:from the dmapi source. 2024-02-19T16:24:56.006 INFO:tasks.workunit.client.0.smithi008.stdout: 2024-02-19T16:24:56.037 INFO:tasks.workunit.client.0.smithi008.stdout:checking linux/fiemap.h usability... yes 2024-02-19T16:24:56.052 INFO:tasks.workunit.client.0.smithi008.stdout:checking linux/fiemap.h presence... yes 2024-02-19T16:24:56.052 INFO:tasks.workunit.client.0.smithi008.stdout:checking for linux/fiemap.h... yes 2024-02-19T16:24:56.087 INFO:tasks.workunit.client.0.smithi008.stdout:checking for fallocate... yes 2024-02-19T16:24:56.137 INFO:tasks.workunit.client.0.smithi008.stdout:checking for open_by_handle_at... yes 2024-02-19T16:24:56.181 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/prctl.h usability... yes 2024-02-19T16:24:56.198 INFO:tasks.workunit.client.0.smithi008.stdout:checking sys/prctl.h presence... yes 2024-02-19T16:24:56.198 INFO:tasks.workunit.client.0.smithi008.stdout:checking for sys/prctl.h... yes 2024-02-19T16:24:56.230 INFO:tasks.workunit.client.0.smithi008.stdout:checking linux/fs.h usability... yes 2024-02-19T16:24:56.248 INFO:tasks.workunit.client.0.smithi008.stdout:checking linux/fs.h presence... yes 2024-02-19T16:24:56.248 INFO:tasks.workunit.client.0.smithi008.stdout:checking for linux/fs.h... yes 2024-02-19T16:24:56.312 INFO:tasks.workunit.client.0.smithi008.stdout:checking btrfsutil.h usability... no 2024-02-19T16:24:56.337 INFO:tasks.workunit.client.0.smithi008.stdout:checking btrfsutil.h presence... no 2024-02-19T16:24:56.337 INFO:tasks.workunit.client.0.smithi008.stdout:checking for btrfsutil.h... no 2024-02-19T16:24:56.386 INFO:tasks.workunit.client.0.smithi008.stdout:checking for copy_file_range... yes 2024-02-19T16:24:56.428 INFO:tasks.workunit.client.0.smithi008.stdout:checking for renameat2... yes 2024-02-19T16:24:56.446 INFO:tasks.workunit.client.0.smithi008.stdout:configure: creating ./config.status 2024-02-19T16:24:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:56 smithi008 ceph-mon[32573]: pgmap v64: 129 pgs: 129 active+clean; 53 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-02-19T16:24:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:56 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:24:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:56 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:56.773 INFO:tasks.workunit.client.0.smithi008.stdout:config.status: creating include/builddefs 2024-02-19T16:24:56.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:56 smithi191 ceph-mon[37203]: pgmap v64: 129 pgs: 129 active+clean; 53 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-02-19T16:24:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:56 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:24:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:56 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:56.786 INFO:tasks.workunit.client.0.smithi008.stdout:config.status: creating include/config.h 2024-02-19T16:24:56.797 INFO:tasks.workunit.client.0.smithi008.stdout:config.status: executing libtool commands 2024-02-19T16:24:56.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:56 smithi156 ceph-mon[36502]: pgmap v64: 129 pgs: 129 active+clean; 53 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-02-19T16:24:56.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:56 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:24:56.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:56 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:24:56.874 INFO:tasks.workunit.client.0.smithi008.stdout:Building include 2024-02-19T16:24:56.874 INFO:tasks.workunit.client.0.smithi008.stdout:Building lib 2024-02-19T16:24:56.874 INFO:tasks.workunit.client.0.smithi008.stdout:Building tests 2024-02-19T16:24:56.878 INFO:tasks.workunit.client.0.smithi008.stdout: [LTDEP] 2024-02-19T16:24:57.043 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] dataascii.lo 2024-02-19T16:24:57.043 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] databin.lo 2024-02-19T16:24:57.043 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] datapid.lo 2024-02-19T16:24:57.044 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] file_lock.lo 2024-02-19T16:24:57.173 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] forker.lo 2024-02-19T16:24:57.189 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] pattern.lo 2024-02-19T16:24:57.228 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] open_flags.lo 2024-02-19T16:24:57.228 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] random_range.lo 2024-02-19T16:24:57.334 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] string_to_tokens.lo 2024-02-19T16:24:57.362 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] str_to_bytes.lo 2024-02-19T16:24:57.391 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] tlibio.lo 2024-02-19T16:24:57.473 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] write_log.lo 2024-02-19T16:24:57.500 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] random.lo 2024-02-19T16:24:57.746 INFO:tasks.workunit.client.0.smithi008.stdout: [LD] 2024-02-19T16:24:57.854 INFO:tasks.workunit.client.0.smithi008.stdout:Building ltp 2024-02-19T16:24:57.854 INFO:tasks.workunit.client.0.smithi008.stdout:Building src 2024-02-19T16:24:57.854 INFO:tasks.workunit.client.0.smithi008.stdout:Building m4 2024-02-19T16:24:57.854 INFO:tasks.workunit.client.0.smithi008.stdout:Building common 2024-02-19T16:24:57.863 INFO:tasks.workunit.client.0.smithi008.stdout: [DEP] 2024-02-19T16:24:57.881 INFO:tasks.workunit.client.0.smithi008.stdout: [DEP] 2024-02-19T16:24:58.080 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] doio 2024-02-19T16:24:58.080 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] fsstress 2024-02-19T16:24:58.081 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] fsx 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c: In function 'attr_list_path': 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~ 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr:In file included from ../src/global.h:25, 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr: from fsstress.c:12: 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~ 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c: In function 'attr_remove_path': 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-02-19T16:24:58.217 INFO:tasks.workunit.client.0.smithi008.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~ 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr:In file included from ../src/global.h:25, 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: from fsstress.c:12: 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~ 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c: In function 'attr_set_path': 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~ 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr:In file included from ../src/global.h:25, 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: from fsstress.c:12: 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-02-19T16:24:58.218 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~ 2024-02-19T16:24:58.228 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c: In function 'allocsp_f': 2024-02-19T16:24:58.228 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-02-19T16:24:58.228 INFO:tasks.workunit.client.0.smithi008.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-02-19T16:24:58.228 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~~~ 2024-02-19T16:24:58.228 INFO:tasks.workunit.client.0.smithi008.stderr: | XFS_IOC_RESVSP64 2024-02-19T16:24:58.228 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-02-19T16:24:58.245 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c: In function 'freesp_f': 2024-02-19T16:24:58.246 INFO:tasks.workunit.client.0.smithi008.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-02-19T16:24:58.246 INFO:tasks.workunit.client.0.smithi008.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-02-19T16:24:58.246 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~~ 2024-02-19T16:24:58.246 INFO:tasks.workunit.client.0.smithi008.stderr: | XFS_IOC_RESVSP64 2024-02-19T16:24:58.277 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-02-19T16:24:58.277 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr:doio.c: In function 'parse_cmdline': 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr: 3113 | fprintf(stderr, 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~ 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr: 3115 | Prog, TagName, tok); 2024-02-19T16:24:58.655 INFO:tasks.workunit.client.0.smithi008.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-02-19T16:24:58.700 INFO:tasks.workunit.client.0.smithi008.stderr:fsx.c: In function 'main': 2024-02-19T16:24:58.700 INFO:tasks.workunit.client.0.smithi008.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-02-19T16:24:58.701 INFO:tasks.workunit.client.0.smithi008.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-02-19T16:24:58.701 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-19T16:24:58.701 INFO:tasks.workunit.client.0.smithi008.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-02-19T16:24:58.701 INFO:tasks.workunit.client.0.smithi008.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-02-19T16:24:58.701 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-19T16:24:58.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:58 smithi008 ceph-mon[32573]: pgmap v65: 129 pgs: 129 active+clean; 53 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 946 KiB/s wr, 163 op/s 2024-02-19T16:24:58.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:58 smithi191 ceph-mon[37203]: pgmap v65: 129 pgs: 129 active+clean; 53 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 946 KiB/s wr, 163 op/s 2024-02-19T16:24:58.800 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-02-19T16:24:58.800 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-02-19T16:24:58.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:58 smithi156 ceph-mon[36502]: pgmap v65: 129 pgs: 129 active+clean; 53 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 946 KiB/s wr, 163 op/s 2024-02-19T16:24:59.380 INFO:tasks.workunit.client.0.smithi008.stdout: [DEP] 2024-02-19T16:24:59.381 INFO:tasks.workunit.client.0.smithi008.stdout: [DEP] 2024-02-19T16:24:59.381 INFO:tasks.workunit.client.0.smithi008.stdout: [DEP] 2024-02-19T16:24:59.625 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] dirstress 2024-02-19T16:24:59.626 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] fill 2024-02-19T16:24:59.627 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] fill2 2024-02-19T16:24:59.627 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] getpagesize 2024-02-19T16:24:59.762 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] holes 2024-02-19T16:24:59.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:24:59 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:24:59.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:24:59 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:24:59.780 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] lstat64 2024-02-19T16:24:59.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:24:59 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:24:59.862 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] nametest 2024-02-19T16:24:59.878 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] permname 2024-02-19T16:24:59.957 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] randholes 2024-02-19T16:24:59.972 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] runas 2024-02-19T16:25:00.067 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] truncfile 2024-02-19T16:25:00.121 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] usemem 2024-02-19T16:25:00.169 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] mmapcat 2024-02-19T16:25:00.234 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] append_reader 2024-02-19T16:25:00.252 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] append_writer 2024-02-19T16:25:00.285 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] dirperf 2024-02-19T16:25:00.335 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] metaperf 2024-02-19T16:25:00.394 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] devzero 2024-02-19T16:25:00.409 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] feature 2024-02-19T16:25:00.520 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] alloc 2024-02-19T16:25:00.742 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] fault 2024-02-19T16:25:00.743 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] fstest 2024-02-19T16:25:00.743 INFO:tasks.workunit.client.0.smithi008.stdout: [CC] t_access_root 2024-02-19T16:25:00.756 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c: In function 'main': 2024-02-19T16:25:00.756 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-02-19T16:25:00.756 INFO:tasks.workunit.client.0.smithi008.stderr: 154 | bozo! 2024-02-19T16:25:00.756 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~ 2024-02-19T16:25:00.756 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-02-19T16:25:00.756 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-02-19T16:25:00.757 INFO:tasks.workunit.client.0.smithi008.stderr: 154 | bozo! 2024-02-19T16:25:00.757 INFO:tasks.workunit.client.0.smithi008.stderr: | ^ 2024-02-19T16:25:00.757 INFO:tasks.workunit.client.0.smithi008.stderr: | ; 2024-02-19T16:25:00.757 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-02-19T16:25:00.757 INFO:tasks.workunit.client.0.smithi008.stderr: 184 | rflag = 1; 2024-02-19T16:25:00.757 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~ 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr: | nflag 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr: 320 | bozo! 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr: | ^ 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr: | ; 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr: 128 | struct flock64 f; 2024-02-19T16:25:00.758 INFO:tasks.workunit.client.0.smithi008.stderr: | ^ 2024-02-19T16:25:00.761 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-02-19T16:25:00.761 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-02-19T16:25:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:00 smithi008 ceph-mon[32573]: pgmap v66: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-02-19T16:25:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:25:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:25:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:25:00.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:00 smithi191 ceph-mon[37203]: pgmap v66: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-02-19T16:25:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:25:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:25:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:25:00.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:00 smithi156 ceph-mon[36502]: pgmap v66: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-02-19T16:25:00.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:25:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:25:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:25:00.887 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c: In function 'run_child': 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~ 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~ 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~ 2024-02-19T16:25:00.888 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-02-19T16:25:00.889 INFO:tasks.workunit.client.0.smithi008.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-19T16:25:00.889 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~ 2024-02-19T16:25:00.889 INFO:tasks.workunit.client.0.smithi008.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-02-19T16:25:00.889 INFO:tasks.workunit.client.0.smithi008.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-02-19T16:25:00.889 INFO:tasks.workunit.client.0.smithi008.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-02-19T16:25:00.958 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-02-19T16:25:00.959 INFO:tasks.workunit.client.0.smithi008.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-02-19T16:25:00.959 INFO:tasks.workunit.client.0.smithi008.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-02-19T16:25:00.959 INFO:tasks.workunit.client.0.smithi008.stderr:make: *** [Makefile:51: default] Error 2 2024-02-19T16:25:00.959 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-02-19T16:25:00.960 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-19T16:25:01.285 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi008 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-02-19T16:25:01.499 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f6d8b14e1eb54b03870dc0e937d46b1f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9f8908eed2b2d7e8c316bd5be7ed6e49024931ee/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi008 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-02-19T16:25:01.501 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-19T16:25:01.511 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-19T16:25:01.512 INFO:teuthology.task.exec:Executing custom commands... 2024-02-19T16:25:01.512 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:25:01.512 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-19T16:25:01.557 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-19T16:25:02.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:02 smithi008 ceph-mon[32573]: pgmap v67: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 307 KiB/s wr, 61 op/s 2024-02-19T16:25:02.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:02 smithi191 ceph-mon[37203]: pgmap v67: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 307 KiB/s wr, 61 op/s 2024-02-19T16:25:02.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:02 smithi156 ceph-mon[36502]: pgmap v67: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 307 KiB/s wr, 61 op/s 2024-02-19T16:25:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:04 smithi008 ceph-mon[32573]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 307 KiB/s wr, 61 op/s 2024-02-19T16:25:04.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:04 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:25:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:04 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:25:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:04 smithi191 ceph-mon[37203]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 307 KiB/s wr, 61 op/s 2024-02-19T16:25:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:04 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:25:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:04 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:25:04.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:04 smithi156 ceph-mon[36502]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 307 KiB/s wr, 61 op/s 2024-02-19T16:25:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:04 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:25:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:04 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:25:05.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:05 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:25:05.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:05 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:25:05.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:05 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:25:06.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:06 smithi008 ceph-mon[32573]: pgmap v69: 129 pgs: 129 active+clean; 69 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 269 op/s 2024-02-19T16:25:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:06 smithi191 ceph-mon[37203]: pgmap v69: 129 pgs: 129 active+clean; 69 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 269 op/s 2024-02-19T16:25:06.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:06 smithi156 ceph-mon[36502]: pgmap v69: 129 pgs: 129 active+clean; 69 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 269 op/s 2024-02-19T16:25:08.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:08 smithi008 ceph-mon[32573]: pgmap v70: 129 pgs: 129 active+clean; 69 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-02-19T16:25:08.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:08 smithi191 ceph-mon[37203]: pgmap v70: 129 pgs: 129 active+clean; 69 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-02-19T16:25:08.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:08 smithi156 ceph-mon[36502]: pgmap v70: 129 pgs: 129 active+clean; 69 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-02-19T16:25:10.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:10 smithi008 ceph-mon[32573]: pgmap v71: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-02-19T16:25:10.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:10 smithi191 ceph-mon[37203]: pgmap v71: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-02-19T16:25:10.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:10 smithi156 ceph-mon[36502]: pgmap v71: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-02-19T16:25:12.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:12 smithi008 ceph-mon[32573]: pgmap v72: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 244 op/s 2024-02-19T16:25:12.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:12 smithi191 ceph-mon[37203]: pgmap v72: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 244 op/s 2024-02-19T16:25:12.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:12 smithi156 ceph-mon[36502]: pgmap v72: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 244 op/s 2024-02-19T16:25:14.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:14 smithi008 ceph-mon[32573]: pgmap v73: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 244 op/s 2024-02-19T16:25:14.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:14 smithi191 ceph-mon[37203]: pgmap v73: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 244 op/s 2024-02-19T16:25:14.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:14 smithi156 ceph-mon[36502]: pgmap v73: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 244 op/s 2024-02-19T16:25:16.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:16 smithi008 ceph-mon[32573]: pgmap v74: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 251 op/s 2024-02-19T16:25:16.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:16 smithi191 ceph-mon[37203]: pgmap v74: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 251 op/s 2024-02-19T16:25:16.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:16 smithi156 ceph-mon[36502]: pgmap v74: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 251 op/s 2024-02-19T16:25:18.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:18 smithi008 ceph-mon[32573]: pgmap v75: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 989 KiB/s wr, 43 op/s 2024-02-19T16:25:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:18 smithi191 ceph-mon[37203]: pgmap v75: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 989 KiB/s wr, 43 op/s 2024-02-19T16:25:18.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:18 smithi156 ceph-mon[36502]: pgmap v75: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 989 KiB/s wr, 43 op/s 2024-02-19T16:25:20.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:20 smithi008 ceph-mon[32573]: pgmap v76: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 991 KiB/s wr, 47 op/s 2024-02-19T16:25:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:20 smithi191 ceph-mon[37203]: pgmap v76: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 991 KiB/s wr, 47 op/s 2024-02-19T16:25:20.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:20 smithi156 ceph-mon[36502]: pgmap v76: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 991 KiB/s wr, 47 op/s 2024-02-19T16:25:22.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:22 smithi008 ceph-mon[32573]: pgmap v77: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-02-19T16:25:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:22 smithi191 ceph-mon[37203]: pgmap v77: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-02-19T16:25:22.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:22 smithi156 ceph-mon[36502]: pgmap v77: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-02-19T16:25:24.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:25:24.355 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:25:24.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 -- 172.21.15.8:0/2423274489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1005e0 msgr2=0x7efc4c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 --2- 172.21.15.8:0/2423274489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1005e0 0x7efc4c1009c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efc40009930 tx=0x7efc4002bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:24.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 -- 172.21.15.8:0/2423274489 shutdown_connections 2024-02-19T16:25:24.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 --2- 172.21.15.8:0/2423274489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1056b0 0x7efc4c10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 --2- 172.21.15.8:0/2423274489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc4c100f90 0x7efc4c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 --2- 172.21.15.8:0/2423274489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1005e0 0x7efc4c1009c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 -- 172.21.15.8:0/2423274489 >> 172.21.15.8:0/2423274489 conn(0x7efc4c0775a0 msgr2=0x7efc4c0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 -- 172.21.15.8:0/2423274489 shutdown_connections 2024-02-19T16:25:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.472+0000 7efc52c91640 1 -- 172.21.15.8:0/2423274489 wait complete. 2024-02-19T16:25:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 Processor -- start 2024-02-19T16:25:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 -- start start 2024-02-19T16:25:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1005e0 0x7efc4c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 0x7efc4c19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc4c197930 0x7efc4c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c198320 con 0x7efc4c1056b0 2024-02-19T16:25:24.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c198490 con 0x7efc4c197930 2024-02-19T16:25:24.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc52c91640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c198650 con 0x7efc4c1005e0 2024-02-19T16:25:24.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 0x7efc4c19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 0x7efc4c19dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57202/0 (socket says 172.21.15.8:57202) 2024-02-19T16:25:24.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc50a2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1005e0 0x7efc4c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 -- 172.21.15.8:0/3208807805 learned_addr learned my addr 172.21.15.8:0/3208807805 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc5122e640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc4c197930 0x7efc4c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 -- 172.21.15.8:0/3208807805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1005e0 msgr2=0x7efc4c19d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1005e0 0x7efc4c19d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 -- 172.21.15.8:0/3208807805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc4c197930 msgr2=0x7efc4c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc4c197930 0x7efc4c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 -- 172.21.15.8:0/3208807805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40009560 con 0x7efc4c1056b0 2024-02-19T16:25:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc50a2d640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1005e0 0x7efc4c19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.473+0000 7efc4bfff640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 0x7efc4c19dcc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efc3800e9c0 tx=0x7efc3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.474+0000 7efc49ffb640 1 -- 172.21.15.8:0/3208807805 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38009700 con 0x7efc4c1056b0 2024-02-19T16:25:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.474+0000 7efc49ffb640 1 -- 172.21.15.8:0/3208807805 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc3800cd80 con 0x7efc4c1056b0 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.474+0000 7efc49ffb640 1 -- 172.21.15.8:0/3208807805 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38010430 con 0x7efc4c1056b0 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.474+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c1a4640 con 0x7efc4c1056b0 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.474+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c1a4b70 con 0x7efc4c1056b0 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.476+0000 7efc49ffb640 1 -- 172.21.15.8:0/3208807805 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7efc38005000 con 0x7efc4c1056b0 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 Processor -- start 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 -- start start 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc49ffb640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc3c0a22b0 0x7efc3c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff42428280 con 0x55ff424c6bc0 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424257b0 con 0x55ff424b9900 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc55860740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff42420f90 con 0x55ff424b8fe0 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc5122e640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc3c0a22b0 0x7efc3c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc49ffb640 1 -- 172.21.15.8:0/3208807805 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7efc38014070 con 0x7efc4c1056b0 2024-02-19T16:25:24.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc50a2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc5122e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc50a2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:44980/0 (socket says 172.21.15.8:44980) 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc5122e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57208/0 (socket says 172.21.15.8:57208) 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.477+0000 7efc50a2d640 1 -- 172.21.15.8:0/2352702731 learned_addr learned my addr 172.21.15.8:0/2352702731 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc50a2d640 1 -- 172.21.15.8:0/2352702731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 msgr2=0x55ff424c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc50a2d640 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc50a2d640 1 -- 172.21.15.8:0/2352702731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 msgr2=0x55ff424c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc50a2d640 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc50a2d640 1 -- 172.21.15.8:0/2352702731 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff423df570 con 0x55ff424b8fe0 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc5122e640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc3c0a22b0 0x7efc3c0a4770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efc44005fd0 tx=0x7efc44009e90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc50a2d640 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424b93c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efc40009930 tx=0x7efc4002c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ac4f485c615a4a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc36ffd640 1 -- 172.21.15.8:0/2352702731 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4002c7e0 con 0x55ff424b8fe0 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc36ffd640 1 -- 172.21.15.8:0/2352702731 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc40004510 con 0x55ff424b8fe0 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.478+0000 7efc36ffd640 1 -- 172.21.15.8:0/2352702731 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400052d0 con 0x55ff424b8fe0 2024-02-19T16:25:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 -- 172.21.15.8:0/2352702731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 msgr2=0x55ff424b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424b93c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efc40009930 tx=0x7efc4002c6d0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 -- 172.21.15.8:0/2352702731 shutdown_connections 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 --2- 172.21.15.8:0/2352702731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424b93c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 -- 172.21.15.8:0/2352702731 >> 172.21.15.8:0/2352702731 conn(0x55ff424b3990 msgr2=0x55ff424b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 -- 172.21.15.8:0/2352702731 shutdown_connections 2024-02-19T16:25:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 -- 172.21.15.8:0/2352702731 wait complete. 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 -- start start 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.479+0000 7efc55860740 1 Processor -- start 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff420f4430 con 0x55ff424c6bc0 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff420b0e00 con 0x55ff424b9900 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff423a5fd0 con 0x55ff424b8fe0 2024-02-19T16:25:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc55860740 10 client.0 _ll_register_callbacks cb 0x55ff423fbfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:24.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc5122e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:44982/0 (socket says 172.21.15.8:44982) 2024-02-19T16:25:24.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 -- 172.21.15.8:0/3763056560 learned_addr learned my addr 172.21.15.8:0/3763056560 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 -- 172.21.15.8:0/3763056560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 msgr2=0x55ff424c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 -- 172.21.15.8:0/3763056560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 msgr2=0x55ff424c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc50a2d640 1 -- 172.21.15.8:0/3763056560 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c10da90 con 0x55ff424b8fe0 2024-02-19T16:25:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc5122e640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.480+0000 7efc4bfff640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc50a2d640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c65c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efc40036040 tx=0x7efc4002c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3626da1288d33bb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:25:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc2bfff640 1 -- 172.21.15.8:0/3763056560 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400054e0 con 0x55ff424b8fe0 2024-02-19T16:25:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc2bfff640 1 -- 172.21.15.8:0/3763056560 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7efc4004d070 con 0x55ff424b8fe0 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff421207e0 con 0x55ff424b8fe0 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc2bfff640 1 -- 172.21.15.8:0/3763056560 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc40005640 con 0x55ff424b8fe0 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 10 client.26056 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ff4241f610 con 0x55ff424b8fe0 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc2bfff640 1 -- 172.21.15.8:0/3763056560 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ff421207e0 con 0x55ff424b8fe0 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc2bfff640 1 -- 172.21.15.8:0/3763056560 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc4003c050 con 0x55ff424b8fe0 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 10 client.26056 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 20 client.26056 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff424d69d0 0x55ff424d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 4 client.26056 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.481+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff4215d720 con 0x55ff424d69d0 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.482+0000 7efc4bfff640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff424d69d0 0x55ff424d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.482+0000 7efc4bfff640 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff424d69d0 0x55ff424d8df0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4ce27bc4214a61 server_cookie=904730b422828f91 in_seq=0 out_seq=0 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.482+0000 7efc2bfff640 10 client.26056 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc2bfff640 1 -- 172.21.15.8:0/3763056560 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff4215d720 con 0x55ff424d69d0 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc2bfff640 10 client.26056 handle_command_reply: tid=0 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 client.26056 shutdown 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 msgr2=0x55ff424c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c65c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efc40036040 tx=0x7efc4002c3b0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 shutdown_connections 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff424d69d0 0x55ff424d8df0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 --2- 172.21.15.8:0/3763056560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c65c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc2bfff640 0 client.26056 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 >> 172.21.15.8:0/3763056560 conn(0x55ff424b3990 msgr2=0x55ff424b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 shutdown_connections 2024-02-19T16:25:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 1 -- 172.21.15.8:0/3763056560 wait complete. 2024-02-19T16:25:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.483+0000 7efc55860740 20 client.26056 trim_cache size 0 max 16384 2024-02-19T16:25:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 Processor -- start 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 -- start start 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424af120 con 0x55ff424c6bc0 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424b4c40 con 0x55ff424b9900 2024-02-19T16:25:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc55860740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424d67d0 con 0x55ff424b8fe0 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc5122e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc5122e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57220/0 (socket says 172.21.15.8:57220) 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc5122e640 1 -- 172.21.15.8:0/2936143223 learned_addr learned my addr 172.21.15.8:0/2936143223 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.499+0000 7efc50a2d640 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc4bfff640 1 -- 172.21.15.8:0/2936143223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 msgr2=0x55ff424c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc4bfff640 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc4bfff640 1 -- 172.21.15.8:0/2936143223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 msgr2=0x55ff424c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc4bfff640 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc4bfff640 1 -- 172.21.15.8:0/2936143223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff423bbac0 con 0x55ff424b9900 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc5122e640 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc4bfff640 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc380cbb60 tx=0x7efc38097120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71c9383c5bf2f3f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc367fc640 1 -- 172.21.15.8:0/2936143223 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3809d040 con 0x55ff424b9900 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc367fc640 1 -- 172.21.15.8:0/2936143223 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc3809e040 con 0x55ff424b9900 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.500+0000 7efc367fc640 1 -- 172.21.15.8:0/2936143223 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380184c0 con 0x55ff424b9900 2024-02-19T16:25:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- 172.21.15.8:0/2936143223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 msgr2=0x55ff424c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc380cbb60 tx=0x7efc38097120 comp rx=0 tx=0).stop 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- 172.21.15.8:0/2936143223 shutdown_connections 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424c6bc0 0x55ff424c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b9900 0x55ff424c6080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- 172.21.15.8:0/2936143223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424b8fe0 0x55ff424c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- 172.21.15.8:0/2936143223 >> 172.21.15.8:0/2936143223 conn(0x55ff424b3990 msgr2=0x55ff424c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- 172.21.15.8:0/2936143223 shutdown_connections 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- 172.21.15.8:0/2936143223 wait complete. 2024-02-19T16:25:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- start start 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 Processor -- start 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b8fe0 0x55ff424c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 0x55ff424c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424c6bc0 0x55ff424c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424c8880 con 0x55ff424b9900 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424c89f0 con 0x55ff424b8fe0 2024-02-19T16:25:24.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff424c8bb0 con 0x55ff424c6bc0 2024-02-19T16:25:24.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc55860740 10 client.0 _ll_register_callbacks cb 0x7efc44011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:24.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.501+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 0x55ff424c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 0x55ff424c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57222/0 (socket says 172.21.15.8:57222) 2024-02-19T16:25:24.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc5122e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424c6bc0 0x55ff424c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 -- 172.21.15.8:0/648614828 learned_addr learned my addr 172.21.15.8:0/648614828 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc50a2d640 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b8fe0 0x55ff424c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 -- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424c6bc0 msgr2=0x55ff424c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424c6bc0 0x55ff424c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 -- 172.21.15.8:0/648614828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b8fe0 msgr2=0x55ff424c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b8fe0 0x55ff424c1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 -- 172.21.15.8:0/648614828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc44019040 con 0x55ff424b9900 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc4bfff640 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 0x55ff424c7e00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efc3801b040 tx=0x7efc38018520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1dc6de4960bada server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc2bfff640 1 -- 172.21.15.8:0/648614828 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3809d070 con 0x55ff424b9900 2024-02-19T16:25:24.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc2bfff640 1 -- 172.21.15.8:0/648614828 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7efc380b3070 con 0x55ff424b9900 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc40004510 con 0x55ff424b9900 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc2bfff640 1 -- 172.21.15.8:0/648614828 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc3809ef50 con 0x55ff424b9900 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc55860740 10 client.16194 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7efc4c1a4b70 con 0x55ff424b9900 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.502+0000 7efc2bfff640 1 -- 172.21.15.8:0/648614828 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc40004510 con 0x55ff424b9900 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc2bfff640 1 -- 172.21.15.8:0/648614828 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc380cd040 con 0x55ff424b9900 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 10 client.16194 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:24.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 20 client.16194 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:24.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff42533390 0x55ff425357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 4 client.16194 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:25:24.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ff421ac890 con 0x55ff42533390 2024-02-19T16:25:24.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc50a2d640 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff42533390 0x55ff425357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc2bfff640 10 client.16194 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc50a2d640 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff42533390 0x55ff425357b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=808023dc376640f2 server_cookie=5f745a9c2202744 in_seq=0 out_seq=0 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc2bfff640 1 -- 172.21.15.8:0/648614828 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ff421ac890 con 0x55ff42533390 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc2bfff640 10 client.16194 handle_command_reply: tid=0 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 client.16194 shutdown 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 msgr2=0x55ff424c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 0x55ff424c7e00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efc3801b040 tx=0x7efc38018520 comp rx=0 tx=0).stop 2024-02-19T16:25:24.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 shutdown_connections 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ff42533390 0x55ff425357b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff424c6bc0 0x55ff424c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc55860740 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff424b9900 0x55ff424c7e00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.503+0000 7efc2bfff640 0 client.16194 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc55860740 1 --2- 172.21.15.8:0/648614828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ff424b8fe0 0x55ff424c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 >> 172.21.15.8:0/648614828 conn(0x55ff424b3990 msgr2=0x55ff424b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 shutdown_connections 2024-02-19T16:25:24.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc55860740 1 -- 172.21.15.8:0/648614828 wait complete. 2024-02-19T16:25:24.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc55860740 20 client.16194 trim_cache size 0 max 16384 2024-02-19T16:25:24.530 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:25:24.530 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:25:24.530 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:25:24.530 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:25:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc3c0a22b0 msgr2=0x7efc3c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc3c0a22b0 0x7efc3c0a4770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efc44005fd0 tx=0x7efc44009e90 comp rx=0 tx=0).stop 2024-02-19T16:25:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 msgr2=0x7efc4c19dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 0x7efc4c19dcc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efc3800e9c0 tx=0x7efc3800ee80 comp rx=0 tx=0).stop 2024-02-19T16:25:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 shutdown_connections 2024-02-19T16:25:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc4c197930 0x7efc4c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc4c1056b0 0x7efc4c19dcc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efc3c0a22b0 0x7efc3c0a4770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 --2- 172.21.15.8:0/3208807805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc4c1005e0 0x7efc4c19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 >> 172.21.15.8:0/3208807805 conn(0x7efc4c0775a0 msgr2=0x7efc4c0689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 shutdown_connections 2024-02-19T16:25:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.504+0000 7efc52c91640 1 -- 172.21.15.8:0/3208807805 wait complete. 2024-02-19T16:25:24.534 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:25:24.534 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:25:24.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 -- 172.21.15.8:0/1939204965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 msgr2=0x7f2b9c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 --2- 172.21.15.8:0/1939204965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 0x7f2b9c1009e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009860 tx=0x7f2b9802bb70 comp rx=0 tx=0).stop 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 -- 172.21.15.8:0/1939204965 shutdown_connections 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 --2- 172.21.15.8:0/1939204965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c10d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 --2- 172.21.15.8:0/1939204965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c100fb0 0x7f2b9c104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 --2- 172.21.15.8:0/1939204965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 0x7f2b9c1009e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 -- 172.21.15.8:0/1939204965 >> 172.21.15.8:0/1939204965 conn(0x7f2b9c076770 msgr2=0x7f2b9c076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 -- 172.21.15.8:0/1939204965 shutdown_connections 2024-02-19T16:25:24.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.636+0000 7f2ba4072640 1 -- 172.21.15.8:0/1939204965 wait complete. 2024-02-19T16:25:24.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 Processor -- start 2024-02-19T16:25:24.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 -- start start 2024-02-19T16:25:24.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 0x7f2b9c102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c1025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c102b10 0x7f2b9c1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1a6280 con 0x7f2b9c1056d0 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1a63f0 con 0x7f2b9c102b10 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba4072640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1a65b0 con 0x7f2b9c100600 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba160d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c1025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba160d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c1025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57228/0 (socket says 172.21.15.8:57228) 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba160d640 1 -- 172.21.15.8:0/3919114517 learned_addr learned my addr 172.21.15.8:0/3919114517 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba1e0e640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 0x7f2b9c102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba260f640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c102b10 0x7f2b9c1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.637+0000 7f2ba160d640 1 -- 172.21.15.8:0/3919114517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 msgr2=0x7f2b9c102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba160d640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 0x7f2b9c102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba160d640 1 -- 172.21.15.8:0/3919114517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c102b10 msgr2=0x7f2b9c1a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba160d640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c102b10 0x7f2b9c1a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba160d640 1 -- 172.21.15.8:0/3919114517 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009900 con 0x7f2b9c1056d0 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba160d640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c1025d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00e9c0 tx=0x7f2b8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c009700 con 0x7f2b9c1056d0 2024-02-19T16:25:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b8c00cd80 con 0x7f2b9c1056d0 2024-02-19T16:25:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c010430 con 0x7f2b9c1056d0 2024-02-19T16:25:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b98009490 con 0x7f2b9c1056d0 2024-02-19T16:25:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.638+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c1a6bc0 con 0x7f2b9c1056d0 2024-02-19T16:25:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.639+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c101ad0 con 0x7f2b9c1056d0 2024-02-19T16:25:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.641+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f2b8c005000 con 0x7f2b9c1056d0 2024-02-19T16:25:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.641+0000 7f2b92ffd640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2b880a23b0 0x7f2b880a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.641+0000 7f2ba1e0e640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2b880a23b0 0x7f2b880a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.641+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f2b8c014070 con 0x7f2b9c1056d0 2024-02-19T16:25:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.642+0000 7f2ba1e0e640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2b880a23b0 0x7f2b880a4870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b9802c180 tx=0x7f2b980058d0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.646+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2b8c097050 con 0x7f2b9c1056d0 2024-02-19T16:25:24.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:24 smithi008 ceph-mon[32573]: pgmap v78: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-02-19T16:25:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:24 smithi191 ceph-mon[37203]: pgmap v78: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-02-19T16:25:24.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:24 smithi156 ceph-mon[36502]: pgmap v78: 129 pgs: 129 active+clean; 79 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-02-19T16:25:24.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.868+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b9c109910 con 0x7f2b9c1056d0 2024-02-19T16:25:24.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.869+0000 7f2b92ffd640 1 -- 172.21.15.8:0/3919114517 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f2b8c09a710 con 0x7f2b9c1056d0 2024-02-19T16:25:24.870 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:25:24.871 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:24:42.507626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:25:24.871 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 27 2024-02-19T16:25:24.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2b880a23b0 msgr2=0x7f2b880a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2b880a23b0 0x7f2b880a4870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b9802c180 tx=0x7f2b980058d0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 msgr2=0x7f2b9c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c1025d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00e9c0 tx=0x7f2b8c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 shutdown_connections 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c102b10 0x7f2b9c1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c1056d0 0x7f2b9c1025d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2b880a23b0 0x7f2b880a4870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 --2- 172.21.15.8:0/3919114517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c100600 0x7f2b9c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 >> 172.21.15.8:0/3919114517 conn(0x7f2b9c076770 msgr2=0x7f2b9c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 shutdown_connections 2024-02-19T16:25:24.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.871+0000 7f2ba4072640 1 -- 172.21.15.8:0/3919114517 wait complete. 2024-02-19T16:25:24.883 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:25:24.883 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:25:24.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.986+0000 7fabe4b72640 1 -- 172.21.15.8:0/789232409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe0100f80 msgr2=0x7fabe0104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.986+0000 7fabe4b72640 1 --2- 172.21.15.8:0/789232409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe0100f80 0x7fabe0104fc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009930 tx=0x7fabc802bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:24.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 -- 172.21.15.8:0/789232409 shutdown_connections 2024-02-19T16:25:24.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 --2- 172.21.15.8:0/789232409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01056a0 0x7fabe010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 --2- 172.21.15.8:0/789232409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe0100f80 0x7fabe0104fc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 --2- 172.21.15.8:0/789232409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01005d0 0x7fabe01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 -- 172.21.15.8:0/789232409 >> 172.21.15.8:0/789232409 conn(0x7fabe00774f0 msgr2=0x7fabe0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 -- 172.21.15.8:0/789232409 shutdown_connections 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 -- 172.21.15.8:0/789232409 wait complete. 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 Processor -- start 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 -- start start 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 0x7fabe019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.987+0000 7fabe4b72640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01056a0 0x7fabe019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabe4b72640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01979c0 0x7fabe0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabe4b72640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01983b0 con 0x7fabe01005d0 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabe4b72640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe0198520 con 0x7fabe01979c0 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabe4b72640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01986e0 con 0x7fabe01056a0 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 0x7fabe019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 0x7fabe019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57248/0 (socket says 172.21.15.8:57248) 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 -- 172.21.15.8:0/2958263313 learned_addr learned my addr 172.21.15.8:0/2958263313 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:24.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabd7fff640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01056a0 0x7fabe019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabded9d640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01979c0 0x7fabe0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:24.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 -- 172.21.15.8:0/2958263313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01056a0 msgr2=0x7fabe019dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01056a0 0x7fabe019dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 -- 172.21.15.8:0/2958263313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01979c0 msgr2=0x7fabe0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:24.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01979c0 0x7fabe0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:24.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 -- 172.21.15.8:0/2958263313 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8009560 con 0x7fabe01005d0 2024-02-19T16:25:24.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabde59c640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 0x7fabe019d810 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fabd000e9f0 tx=0x7fabd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:24.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabd77fe640 1 -- 172.21.15.8:0/2958263313 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0009720 con 0x7fabe01005d0 2024-02-19T16:25:24.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe01a46d0 con 0x7fabe01005d0 2024-02-19T16:25:24.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabd77fe640 1 -- 172.21.15.8:0/2958263313 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabd000cd80 con 0x7fabe01005d0 2024-02-19T16:25:24.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabd77fe640 1 -- 172.21.15.8:0/2958263313 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0010690 con 0x7fabe01005d0 2024-02-19T16:25:24.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.988+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe01a4bb0 con 0x7fabe01005d0 2024-02-19T16:25:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabd77fe640 1 -- 172.21.15.8:0/2958263313 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fabd00046c0 con 0x7fabe01005d0 2024-02-19T16:25:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 Processor -- start 2024-02-19T16:25:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 -- start start 2024-02-19T16:25:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea62630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea6b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bea1d960 con 0x5627bea6b750 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627be9d1310 con 0x5627bea62250 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabe7820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627be9ce840 con 0x5627bea62b70 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabd77fe640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fabcc0a2230 0x7fabcc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabded9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabde59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea6b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabd7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabded9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea62630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:34516/0 (socket says 172.21.15.8:34516) 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabd7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea6db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57264/0 (socket says 172.21.15.8:57264) 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.991+0000 7fabded9d640 1 -- 172.21.15.8:0/852393414 learned_addr learned my addr 172.21.15.8:0/852393414 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fabcc0a2230 0x7fabcc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabd77fe640 1 -- 172.21.15.8:0/2958263313 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fabd0014070 con 0x7fabe01005d0 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 -- 172.21.15.8:0/852393414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 msgr2=0x5627bea6b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea6b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 -- 172.21.15.8:0/852393414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 msgr2=0x5627bea6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 -- 172.21.15.8:0/852393414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627be98c060 con 0x5627bea62250 2024-02-19T16:25:25.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabde59c640 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea6b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:25.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea62630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011da0 tx=0x7fabd800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725ee4d39e881d32 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabded9d640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fabcc0a2230 0x7fabcc0a46f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabd800fc90 tx=0x7fabd801e000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabd4ff9640 1 -- 172.21.15.8:0/852393414 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd801dbf0 con 0x5627bea62250 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabd4ff9640 1 -- 172.21.15.8:0/852393414 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabd8009dc0 con 0x5627bea62250 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.992+0000 7fabd4ff9640 1 -- 172.21.15.8:0/852393414 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8004290 con 0x5627bea62250 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 -- 172.21.15.8:0/852393414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 msgr2=0x5627bea62630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea62630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011da0 tx=0x7fabd800f790 comp rx=0 tx=0).stop 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 -- 172.21.15.8:0/852393414 shutdown_connections 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea6b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 --2- 172.21.15.8:0/852393414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea62630 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 -- 172.21.15.8:0/852393414 >> 172.21.15.8:0/852393414 conn(0x5627bea5c980 msgr2=0x5627bea5cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 -- 172.21.15.8:0/852393414 shutdown_connections 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 -- 172.21.15.8:0/852393414 wait complete. 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 -- start start 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.993+0000 7fabe7820740 1 Processor -- start 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 0x5627bea68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62b70 0x5627bea68ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea6b750 0x5627bea691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627be9ca020 con 0x5627bea62250 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627be69d430 con 0x5627bea62b70 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627be659e00 con 0x5627bea6b750 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 10 client.0 _ll_register_callbacks cb 0x5627be971cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 0x5627bea68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 0x5627bea68760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57272/0 (socket says 172.21.15.8:57272) 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 -- 172.21.15.8:0/3642208128 learned_addr learned my addr 172.21.15.8:0/3642208128 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabded9d640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea6b750 0x5627bea691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabd7fff640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62b70 0x5627bea68ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 -- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea6b750 msgr2=0x5627bea691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea6b750 0x5627bea691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 -- 172.21.15.8:0/3642208128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62b70 msgr2=0x5627bea68ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62b70 0x5627bea68ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 -- 172.21.15.8:0/3642208128 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd8019040 con 0x5627bea62250 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabde59c640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 0x5627bea68760 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fabd0004ae0 tx=0x7fabd006a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95ebc3aefb85b92 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabc5ffb640 1 -- 172.21.15.8:0/3642208128 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00a3330 con 0x5627bea62250 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabc5ffb640 1 -- 172.21.15.8:0/3642208128 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fabd009ae20 con 0x5627bea62250 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.994+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627be6c97e0 con 0x5627bea62250 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabc5ffb640 1 -- 172.21.15.8:0/3642208128 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabd009ef70 con 0x5627bea62250 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 10 client.16212 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5627be9c86a0 con 0x5627bea62250 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabc5ffb640 1 -- 172.21.15.8:0/3642208128 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5627be6c97e0 con 0x5627bea62250 2024-02-19T16:25:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabc5ffb640 1 -- 172.21.15.8:0/3642208128 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fabd001f060 con 0x5627bea62250 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 10 client.16212 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 10 client.16212 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 20 client.16212 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627bea7baf0 0x5627bea7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 4 client.16212 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627be706720 con 0x5627bea7baf0 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabd7fff640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627bea7baf0 0x5627bea7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabc5ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.995+0000 7fabd7fff640 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627bea7baf0 0x5627bea7dee0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e93259cbac84d8a9 server_cookie=9df60c1991533b17 in_seq=0 out_seq=0 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.996+0000 7fabc5ffb640 1 -- 172.21.15.8:0/3642208128 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5627be706720 con 0x5627bea7baf0 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.996+0000 7fabc5ffb640 10 client.16212 handle_command_reply: tid=0 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.996+0000 7fabe7820740 1 client.16212 shutdown 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 msgr2=0x5627bea68760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 0x5627bea68760 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fabd0004ae0 tx=0x7fabd006a080 comp rx=0 tx=0).stop 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 shutdown_connections 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627bea7baf0 0x5627bea7dee0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea6b750 0x5627bea691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabc5ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62b70 0x5627bea68ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 --2- 172.21.15.8:0/3642208128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62250 0x5627bea68760 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 >> 172.21.15.8:0/3642208128 conn(0x5627bea5c980 msgr2=0x5627bea5e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 shutdown_connections 2024-02-19T16:25:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 1 -- 172.21.15.8:0/3642208128 wait complete. 2024-02-19T16:25:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:24.997+0000 7fabe7820740 20 client.16212 trim_cache size 0 max 16384 2024-02-19T16:25:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 Processor -- start 2024-02-19T16:25:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 -- start start 2024-02-19T16:25:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea6ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea69020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea76a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627be94f130 con 0x5627bea6b750 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bea67800 con 0x5627bea62250 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabe7820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bea57a90 con 0x5627bea62b70 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea76a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabd7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea69020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea76a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57286/0 (socket says 172.21.15.8:57286) 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabded9d640 1 -- 172.21.15.8:0/3178993727 learned_addr learned my addr 172.21.15.8:0/3178993727 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.009+0000 7fabde59c640 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea6ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabded9d640 1 -- 172.21.15.8:0/3178993727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 msgr2=0x5627bea69020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabded9d640 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea69020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabded9d640 1 -- 172.21.15.8:0/3178993727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 msgr2=0x5627bea6ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabded9d640 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea6ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabded9d640 1 -- 172.21.15.8:0/3178993727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627be964c20 con 0x5627bea6b750 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabded9d640 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea76a70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fabd8003fd0 tx=0x7fabd8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c77734dba8a5887 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabc7fff640 1 -- 172.21.15.8:0/3178993727 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8011990 con 0x5627bea6b750 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabc7fff640 1 -- 172.21.15.8:0/3178993727 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabd8004050 con 0x5627bea6b750 2024-02-19T16:25:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabc7fff640 1 -- 172.21.15.8:0/3178993727 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8011040 con 0x5627bea6b750 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 -- 172.21.15.8:0/3178993727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 msgr2=0x5627bea76a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea76a70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fabd8003fd0 tx=0x7fabd8011c50 comp rx=0 tx=0).stop 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 -- 172.21.15.8:0/3178993727 shutdown_connections 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea6b750 0x5627bea76a70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62b70 0x5627bea69020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 --2- 172.21.15.8:0/3178993727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea62250 0x5627bea6ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 -- 172.21.15.8:0/3178993727 >> 172.21.15.8:0/3178993727 conn(0x5627bea5c980 msgr2=0x5627bea5e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 -- 172.21.15.8:0/3178993727 shutdown_connections 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.010+0000 7fabe7820740 1 -- 172.21.15.8:0/3178993727 wait complete. 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 -- start start 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 Processor -- start 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62250 0x5627bea65220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 0x5627bea65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea6b750 0x5627beadb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627beadb930 con 0x5627bea62b70 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627beadba70 con 0x5627bea6b750 2024-02-19T16:25:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627beadbbb0 con 0x5627bea62250 2024-02-19T16:25:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabe7820740 10 client.0 _ll_register_callbacks cb 0x7fabd000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 0x5627bea65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 0x5627bea65760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57298/0 (socket says 172.21.15.8:57298) 2024-02-19T16:25:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 -- 172.21.15.8:0/2317153187 learned_addr learned my addr 172.21.15.8:0/2317153187 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabded9d640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea6b750 0x5627beadb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabde59c640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62250 0x5627bea65220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 -- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62250 msgr2=0x5627bea65220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62250 0x5627bea65220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 -- 172.21.15.8:0/2317153187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea6b750 msgr2=0x5627beadb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea6b750 0x5627beadb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.011+0000 7fabd7fff640 1 -- 172.21.15.8:0/2317153187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd802a040 con 0x5627bea62b70 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabd7fff640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 0x5627bea65760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fabc802bc70 tx=0x7fabc80049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8e130a48d1aa98 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabc5ffb640 1 -- 172.21.15.8:0/2317153187 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004ae0 con 0x5627bea62b70 2024-02-19T16:25:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabc5ffb640 1 -- 172.21.15.8:0/2317153187 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fabc8046040 con 0x5627bea62b70 2024-02-19T16:25:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabd8009dc0 con 0x5627bea62b70 2024-02-19T16:25:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabc5ffb640 1 -- 172.21.15.8:0/2317153187 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabc8004250 con 0x5627bea62b70 2024-02-19T16:25:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 10 client.16218 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5627be98a8c0 con 0x5627bea62b70 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabc5ffb640 1 -- 172.21.15.8:0/2317153187 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabd8009dc0 con 0x5627bea62b70 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabc5ffb640 1 -- 172.21.15.8:0/2317153187 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fabc803c040 con 0x5627bea62b70 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 10 client.16218 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 10 client.16218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 20 client.16218 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627beadd360 0x5627beadf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 4 client.16218 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:25:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627be755890 con 0x5627beadd360 2024-02-19T16:25:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.012+0000 7fabde59c640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627beadd360 0x5627beadf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabde59c640 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627beadd360 0x5627beadf780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=731ba2deea56efc7 server_cookie=2625cb2b5de465db in_seq=0 out_seq=0 2024-02-19T16:25:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabc5ffb640 10 client.16218 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabc5ffb640 1 -- 172.21.15.8:0/2317153187 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627be755890 con 0x5627beadd360 2024-02-19T16:25:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabc5ffb640 10 client.16218 handle_command_reply: tid=0 2024-02-19T16:25:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 client.16218 shutdown 2024-02-19T16:25:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 msgr2=0x5627bea65760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 0x5627bea65760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fabc802bc70 tx=0x7fabc80049f0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 shutdown_connections 2024-02-19T16:25:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5627beadd360 0x5627beadf780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627bea6b750 0x5627beadb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabc5ffb640 0 client.16218 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5627bea62b70 0x5627bea65760 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 --2- 172.21.15.8:0/2317153187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627bea62250 0x5627bea65220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 >> 172.21.15.8:0/2317153187 conn(0x5627bea5c980 msgr2=0x5627bea72980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 shutdown_connections 2024-02-19T16:25:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 1 -- 172.21.15.8:0/2317153187 wait complete. 2024-02-19T16:25:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.013+0000 7fabe7820740 20 client.16218 trim_cache size 0 max 16384 2024-02-19T16:25:25.034 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:25:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fabcc0a2230 msgr2=0x7fabcc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fabcc0a2230 0x7fabcc0a46f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabd800fc90 tx=0x7fabd801e000 comp rx=0 tx=0).stop 2024-02-19T16:25:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 msgr2=0x7fabe019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 0x7fabe019d810 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fabd000e9f0 tx=0x7fabd000edf0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 shutdown_connections 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabe01979c0 0x7fabe0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fabcc0a2230 0x7fabcc0a46f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe01056a0 0x7fabe019dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 --2- 172.21.15.8:0/2958263313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabe01005d0 0x7fabe019d810 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 >> 172.21.15.8:0/2958263313 conn(0x7fabe00774f0 msgr2=0x7fabe00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 shutdown_connections 2024-02-19T16:25:25.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.014+0000 7fabe4b72640 1 -- 172.21.15.8:0/2958263313 wait complete. 2024-02-19T16:25:25.039 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:25:25.039 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:25:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 -- 172.21.15.8:0/2415188170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 msgr2=0x7f7170100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 --2- 172.21.15.8:0/2415188170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f7170100890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f716402bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:25.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 -- 172.21.15.8:0/2415188170 shutdown_connections 2024-02-19T16:25:25.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 --2- 172.21.15.8:0/2415188170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7170100dd0 0x7f717010d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 --2- 172.21.15.8:0/2415188170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f7170100890 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 --2- 172.21.15.8:0/2415188170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71701064b0 0x7f7170106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 -- 172.21.15.8:0/2415188170 >> 172.21.15.8:0/2415188170 conn(0x7f7170077550 msgr2=0x7f7170077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 -- 172.21.15.8:0/2415188170 shutdown_connections 2024-02-19T16:25:25.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 -- 172.21.15.8:0/2415188170 wait complete. 2024-02-19T16:25:25.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.143+0000 7f71761c9640 1 Processor -- start 2024-02-19T16:25:25.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 -- start start 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f71700fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170100dd0 0x7f71700fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71701064b0 0x7f71700fde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701a5f40 con 0x7f7170100410 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701a60b0 con 0x7f71701064b0 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f71761c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701a6270 con 0x7f7170100dd0 2024-02-19T16:25:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f71700fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f71700fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57324/0 (socket says 172.21.15.8:57324) 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 -- 172.21.15.8:0/2764281367 learned_addr learned my addr 172.21.15.8:0/2764281367 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716effd640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170100dd0 0x7f71700fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716ffff640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71701064b0 0x7f71700fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 -- 172.21.15.8:0/2764281367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170100dd0 msgr2=0x7f71700fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170100dd0 0x7f71700fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 -- 172.21.15.8:0/2764281367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71701064b0 msgr2=0x7f71700fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71701064b0 0x7f71700fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 -- 172.21.15.8:0/2764281367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009490 con 0x7f7170100410 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.144+0000 7f716f7fe640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f71700fd390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f716000ea60 tx=0x7f716000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.145+0000 7f716cff9640 1 -- 172.21.15.8:0/2764281367 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000cde0 con 0x7f7170100410 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.145+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7164009560 con 0x7f7170100410 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.145+0000 7f716cff9640 1 -- 172.21.15.8:0/2764281367 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71600035f0 con 0x7f7170100410 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.145+0000 7f716cff9640 1 -- 172.21.15.8:0/2764281367 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160010690 con 0x7f7170100410 2024-02-19T16:25:25.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.145+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71701a68f0 con 0x7f7170100410 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.147+0000 7f7178da6740 1 Processor -- start 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 -- start start 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716cff9640 1 -- 172.21.15.8:0/2764281367 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f7160003af0 con 0x7f7170100410 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e945ab960 con 0x558e945f9750 2024-02-19T16:25:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e9455f310 con 0x558e945f0250 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f7178da6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e9455c840 con 0x558e945f0b70 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:34566/0 (socket says 172.21.15.8:34566) 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945f9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:45106/0 (socket says 172.21.15.8:45106) 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57334/0 (socket says 172.21.15.8:57334) 2024-02-19T16:25:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 -- 172.21.15.8:0/388594224 learned_addr learned my addr 172.21.15.8:0/388594224 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716cff9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71580a2230 0x7f71580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 -- 172.21.15.8:0/388594224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 msgr2=0x558e945f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 -- 172.21.15.8:0/388594224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 msgr2=0x558e945fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 -- 172.21.15.8:0/388594224 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e9451a060 con 0x558e945f0250 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716effd640 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716cff9640 1 -- 172.21.15.8:0/2764281367 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f7160014070 con 0x7f7170100410 2024-02-19T16:25:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.148+0000 7f716ffff640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71580a2230 0x7f71580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f716ffff640 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f0630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7168009fd0 tx=0x7f71680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e9697cb47376121 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f7155ffb640 1 -- 172.21.15.8:0/388594224 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168020030 con 0x558e945f0250 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f7155ffb640 1 -- 172.21.15.8:0/388594224 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f716800fe90 con 0x558e945f0250 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f7155ffb640 1 -- 172.21.15.8:0/388594224 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716801ba70 con 0x558e945f0250 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f7178da6740 1 -- 172.21.15.8:0/388594224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 msgr2=0x558e945f0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f716ffff640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71580a2230 0x7f71580a46f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7168020000 tx=0x7f7168021000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.149+0000 7f7178da6740 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f0630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7168009fd0 tx=0x7f71680118d0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- 172.21.15.8:0/388594224 shutdown_connections 2024-02-19T16:25:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 --2- 172.21.15.8:0/388594224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f0630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- 172.21.15.8:0/388594224 >> 172.21.15.8:0/388594224 conn(0x558e945ea980 msgr2=0x558e945ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- 172.21.15.8:0/388594224 shutdown_connections 2024-02-19T16:25:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- 172.21.15.8:0/388594224 wait complete. 2024-02-19T16:25:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- start start 2024-02-19T16:25:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 Processor -- start 2024-02-19T16:25:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0b70 0x558e945f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f9750 0x558e945f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e94558020 con 0x558e945f0250 2024-02-19T16:25:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e9422b430 con 0x558e945f0b70 2024-02-19T16:25:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e941e7e00 con 0x558e945f9750 2024-02-19T16:25:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.150+0000 7f7178da6740 10 client.0 _ll_register_callbacks cb 0x558e944ffcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57336/0 (socket says 172.21.15.8:57336) 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 -- 172.21.15.8:0/633544151 learned_addr learned my addr 172.21.15.8:0/633544151 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716ffff640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f9750 0x558e945f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716effd640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0b70 0x558e945f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 -- 172.21.15.8:0/633544151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f9750 msgr2=0x558e945f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f9750 0x558e945f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 -- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0b70 msgr2=0x558e945f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0b70 0x558e945f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 -- 172.21.15.8:0/633544151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7168009c80 con 0x558e945f0250 2024-02-19T16:25:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f716f7fe640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6730 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f716001f000 tx=0x7f71600971e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e455219190cd684 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f714effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f714effd640 1 -- 172.21.15.8:0/633544151 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716009b580 con 0x558e945f0250 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f714effd640 1 -- 172.21.15.8:0/633544151 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f716009ae80 con 0x558e945f0250 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e942577e0 con 0x558e945f0250 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f714effd640 1 -- 172.21.15.8:0/633544151 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f716009efd0 con 0x558e945f0250 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f7178da6740 10 client.16230 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e945566a0 con 0x558e945f0250 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.151+0000 7f714effd640 1 -- 172.21.15.8:0/633544151 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558e942577e0 con 0x558e945f0250 2024-02-19T16:25:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f714effd640 1 -- 172.21.15.8:0/633544151 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f71600d6070 con 0x558e945f0250 2024-02-19T16:25:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f7178da6740 10 client.16230 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f7178da6740 10 client.16230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:25:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f7178da6740 20 client.16230 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f7178da6740 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94609af0 0x558e9460bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f7178da6740 4 client.16230 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e94294720 con 0x558e94609af0 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f716effd640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94609af0 0x558e9460bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f714effd640 10 client.16230 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.152+0000 7f716effd640 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94609af0 0x558e9460bee0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e973e4cfc99a1414 server_cookie=2dc8f8db9370f436 in_seq=0 out_seq=0 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f714effd640 1 -- 172.21.15.8:0/633544151 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558e94294720 con 0x558e94609af0 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f714effd640 10 client.16230 handle_command_reply: tid=0 2024-02-19T16:25:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 client.16230 shutdown 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 msgr2=0x558e945f6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6730 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f716001f000 tx=0x7f71600971e0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 shutdown_connections 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94609af0 0x558e9460bee0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f9750 0x558e945f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0b70 0x558e945f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 --2- 172.21.15.8:0/633544151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6730 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f714effd640 0 client.16230 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 >> 172.21.15.8:0/633544151 conn(0x558e945ea980 msgr2=0x558e945ec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 shutdown_connections 2024-02-19T16:25:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 1 -- 172.21.15.8:0/633544151 wait complete. 2024-02-19T16:25:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.153+0000 7f7178da6740 20 client.16230 trim_cache size 0 max 16384 2024-02-19T16:25:25.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 Processor -- start 2024-02-19T16:25:25.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 -- start start 2024-02-19T16:25:25.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f9750 0x558e94604ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e944dd130 con 0x558e945f0250 2024-02-19T16:25:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e945ef3a0 con 0x558e945f9750 2024-02-19T16:25:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f7178da6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e945e62b0 con 0x558e945f0b70 2024-02-19T16:25:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57350/0 (socket says 172.21.15.8:57350) 2024-02-19T16:25:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 -- 172.21.15.8:0/3820879825 learned_addr learned my addr 172.21.15.8:0/3820879825 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716effd640 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716ffff640 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f9750 0x558e94604ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 -- 172.21.15.8:0/3820879825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 msgr2=0x558e945e9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945e9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 -- 172.21.15.8:0/3820879825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f9750 msgr2=0x558e94604ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f9750 0x558e94604ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 -- 172.21.15.8:0/3820879825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e944f2c20 con 0x558e945f0250 2024-02-19T16:25:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f716f7fe640 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6fd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f716000c760 tx=0x7f7160003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cac882c53a6fee server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f71557fa640 1 -- 172.21.15.8:0/3820879825 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716009a9c0 con 0x558e945f0250 2024-02-19T16:25:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f71557fa640 1 -- 172.21.15.8:0/3820879825 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f716009b580 con 0x558e945f0250 2024-02-19T16:25:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.166+0000 7f71557fa640 1 -- 172.21.15.8:0/3820879825 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71600974f0 con 0x558e945f0250 2024-02-19T16:25:25.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 -- 172.21.15.8:0/3820879825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 msgr2=0x558e945f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6fd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f716000c760 tx=0x7f7160003ef0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 -- 172.21.15.8:0/3820879825 shutdown_connections 2024-02-19T16:25:25.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f9750 0x558e94604ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 --2- 172.21.15.8:0/3820879825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f0250 0x558e945f6fd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 -- 172.21.15.8:0/3820879825 >> 172.21.15.8:0/3820879825 conn(0x558e945ea980 msgr2=0x558e945e92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 -- 172.21.15.8:0/3820879825 shutdown_connections 2024-02-19T16:25:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 -- 172.21.15.8:0/3820879825 wait complete. 2024-02-19T16:25:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.167+0000 7f7178da6740 1 -- start start 2024-02-19T16:25:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 Processor -- start 2024-02-19T16:25:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945feb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e945ff0e0 con 0x558e945f9750 2024-02-19T16:25:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e945ff250 con 0x558e945f0250 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e945ff410 con 0x558e945f0b70 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f7178da6740 10 client.0 _ll_register_callbacks cb 0x7f71600d4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945feb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57362/0 (socket says 172.21.15.8:57362) 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 -- 172.21.15.8:0/3989238697 learned_addr learned my addr 172.21.15.8:0/3989238697 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716f7fe640 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 -- 172.21.15.8:0/3989238697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 msgr2=0x558e945fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 -- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 msgr2=0x558e945f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 -- 172.21.15.8:0/3989238697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e945566a0 con 0x558e945f9750 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f716ffff640 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945feb20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7168003d80 tx=0x7f7168003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b6f6fc73484904 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.168+0000 7f714effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f714effd640 1 -- 172.21.15.8:0/3989238697 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680039d0 con 0x558e945f9750 2024-02-19T16:25:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f716800fe90 con 0x558e945f9750 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f714effd640 1 -- 172.21.15.8:0/3989238697 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f7168021070 con 0x558e945f9750 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f714effd640 1 -- 172.21.15.8:0/3989238697 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f716800f930 con 0x558e945f9750 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e945188c0 con 0x558e945f9750 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f714effd640 1 -- 172.21.15.8:0/3989238697 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f716800fe90 con 0x558e945f9750 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f714effd640 1 -- 172.21.15.8:0/3989238697 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7168030550 con 0x558e945f9750 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 10 client.16236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:25:25.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 20 client.16236 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94666a80 0x558e94668ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 4 client.16236 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e942e3890 con 0x558e94666a80 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.169+0000 7f716f7fe640 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94666a80 0x558e94668ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f714effd640 10 client.16236 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f716f7fe640 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94666a80 0x558e94668ea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b05d4ab4b84a00a server_cookie=f68d0c39f20b05cd in_seq=0 out_seq=0 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f714effd640 1 -- 172.21.15.8:0/3989238697 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e942e3890 con 0x558e94666a80 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f714effd640 10 client.16236 handle_command_reply: tid=0 2024-02-19T16:25:25.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 client.16236 shutdown 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 msgr2=0x558e945feb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945feb20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7168003d80 tx=0x7f7168003db0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 shutdown_connections 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558e94666a80 0x558e94668ea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558e945f9750 0x558e945feb20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e945f0b70 0x558e945fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 --2- 172.21.15.8:0/3989238697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e945f0250 0x558e945f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f714effd640 0 client.16236 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:25.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 >> 172.21.15.8:0/3989238697 conn(0x558e945ea980 msgr2=0x558e945e9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 shutdown_connections 2024-02-19T16:25:25.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 1 -- 172.21.15.8:0/3989238697 wait complete. 2024-02-19T16:25:25.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.170+0000 7f7178da6740 20 client.16236 trim_cache size 0 max 16384 2024-02-19T16:25:25.193 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:25:25.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71580a2230 msgr2=0x7f71580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71580a2230 0x7f71580a46f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7168020000 tx=0x7f7168021000 comp rx=0 tx=0).stop 2024-02-19T16:25:25.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 msgr2=0x7f71700fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f71700fd390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f716000ea60 tx=0x7f716000ef20 comp rx=0 tx=0).stop 2024-02-19T16:25:25.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 shutdown_connections 2024-02-19T16:25:25.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71701064b0 0x7f71700fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71580a2230 0x7f71580a46f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170100dd0 0x7f71700fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 --2- 172.21.15.8:0/2764281367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7170100410 0x7f71700fd390 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 >> 172.21.15.8:0/2764281367 conn(0x7f7170077550 msgr2=0x7f7170069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 shutdown_connections 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.171+0000 7f71761c9640 1 -- 172.21.15.8:0/2764281367 wait complete. 2024-02-19T16:25:25.196 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:25:25.197 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:25:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 -- 172.21.15.8:0/1557099545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc101e60 msgr2=0x7f87cc10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 --2- 172.21.15.8:0/1557099545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc101e60 0x7f87cc10e320 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009ca0 tx=0x7f87c402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 -- 172.21.15.8:0/1557099545 shutdown_connections 2024-02-19T16:25:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 --2- 172.21.15.8:0/1557099545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc101e60 0x7f87cc10e320 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 --2- 172.21.15.8:0/1557099545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc1014a0 0x7f87cc101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 --2- 172.21.15.8:0/1557099545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87cc107560 0x7f87cc107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 -- 172.21.15.8:0/1557099545 >> 172.21.15.8:0/1557099545 conn(0x7f87cc0788f0 msgr2=0x7f87cc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 -- 172.21.15.8:0/1557099545 shutdown_connections 2024-02-19T16:25:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.300+0000 7f87d098a640 1 -- 172.21.15.8:0/1557099545 wait complete. 2024-02-19T16:25:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 Processor -- start 2024-02-19T16:25:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 -- start start 2024-02-19T16:25:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 0x7f87cc0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc101e60 0x7f87cc073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87cc107560 0x7f87cc074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1a5e10 con 0x7f87cc1014a0 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1a5f80 con 0x7f87cc101e60 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87d098a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1a6140 con 0x7f87cc107560 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 0x7f87cc0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 0x7f87cc0737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57382/0 (socket says 172.21.15.8:57382) 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 -- 172.21.15.8:0/1886036106 learned_addr learned my addr 172.21.15.8:0/1886036106 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87cad9d640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87cc107560 0x7f87cc074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87c3fff640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc101e60 0x7f87cc073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 -- 172.21.15.8:0/1886036106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87cc107560 msgr2=0x7f87cc074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87cc107560 0x7f87cc074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 -- 172.21.15.8:0/1886036106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc101e60 msgr2=0x7f87cc073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc101e60 0x7f87cc073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.301+0000 7f87ca59c640 1 -- 172.21.15.8:0/1886036106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009490 con 0x7f87cc1014a0 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.302+0000 7f87ca59c640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 0x7f87cc0737f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00e9f0 tx=0x7f87bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.302+0000 7f87c37fe640 1 -- 172.21.15.8:0/1886036106 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc009720 con 0x7f87cc1014a0 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.302+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c4009900 con 0x7f87cc1014a0 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.302+0000 7f87c37fe640 1 -- 172.21.15.8:0/1886036106 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87bc00cd80 con 0x7f87cc1014a0 2024-02-19T16:25:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.302+0000 7f87c37fe640 1 -- 172.21.15.8:0/1886036106 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc010690 con 0x7f87cc1014a0 2024-02-19T16:25:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.302+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc1a6750 con 0x7f87cc1014a0 2024-02-19T16:25:25.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.304+0000 7f87c37fe640 1 -- 172.21.15.8:0/1886036106 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f87bc0046c0 con 0x7f87cc1014a0 2024-02-19T16:25:25.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.304+0000 7f87d3565740 1 Processor -- start 2024-02-19T16:25:25.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.304+0000 7f87d3565740 1 -- start start 2024-02-19T16:25:25.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87d3565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e86630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87d3565740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e86b70 0x55d139e8f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87d3565740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 0x55d139e91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87d3565740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139e41960 con 0x55d139e8f750 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87d3565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139df5310 con 0x55d139e86250 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87d3565740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139df2840 con 0x55d139e86b70 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c37fe640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87b80a2230 0x7f87b80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 0x55d139e91b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e86b70 0x55d139e8f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 0x55d139e91b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57396/0 (socket says 172.21.15.8:57396) 2024-02-19T16:25:25.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87cad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e86630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 -- 172.21.15.8:0/3761042188 learned_addr learned my addr 172.21.15.8:0/3761042188 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87cad9d640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87b80a2230 0x7f87b80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c37fe640 1 -- 172.21.15.8:0/1886036106 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f87bc014070 con 0x7f87cc1014a0 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 -- 172.21.15.8:0/3761042188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e86b70 msgr2=0x55d139e8f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e86b70 0x55d139e8f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 -- 172.21.15.8:0/3761042188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 msgr2=0x55d139e86630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e86630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 -- 172.21.15.8:0/3761042188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d139db0060 con 0x55d139e8f750 2024-02-19T16:25:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87cad9d640 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e86630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87cad9d640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87b80a2230 0x7f87b80a46f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87c4004af0 tx=0x7f87c4005c50 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c3fff640 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 0x55d139e91b40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e20 tx=0x7f87b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8632357cfbe4c254 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.305+0000 7f87c0ff9640 1 -- 172.21.15.8:0/3761042188 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e670 con 0x55d139e8f750 2024-02-19T16:25:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87c0ff9640 1 -- 172.21.15.8:0/3761042188 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b400ecb0 con 0x55d139e8f750 2024-02-19T16:25:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87c0ff9640 1 -- 172.21.15.8:0/3761042188 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4019c90 con 0x55d139e8f750 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 -- 172.21.15.8:0/3761042188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 msgr2=0x55d139e91b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 0x55d139e91b40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e20 tx=0x7f87b4011810 comp rx=0 tx=0).stop 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 -- 172.21.15.8:0/3761042188 shutdown_connections 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e8f750 0x55d139e91b40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e86b70 0x55d139e8f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 --2- 172.21.15.8:0/3761042188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e86630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 -- 172.21.15.8:0/3761042188 >> 172.21.15.8:0/3761042188 conn(0x55d139e80980 msgr2=0x55d139e80d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 -- 172.21.15.8:0/3761042188 shutdown_connections 2024-02-19T16:25:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.306+0000 7f87d3565740 1 -- 172.21.15.8:0/3761042188 wait complete. 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 -- start start 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 Processor -- start 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e8c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e8d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139dee020 con 0x55d139e86250 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139ac1430 con 0x55d139e86b70 2024-02-19T16:25:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139a7de00 con 0x55d139e8f750 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87d3565740 10 client.0 _ll_register_callbacks cb 0x55d139d95cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e8c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87cad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e8d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e8c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57398/0 (socket says 172.21.15.8:57398) 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 -- 172.21.15.8:0/2594182067 learned_addr learned my addr 172.21.15.8:0/2594182067 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87c3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 -- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 msgr2=0x55d139e8d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e8d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 -- 172.21.15.8:0/2594182067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 msgr2=0x55d139e8ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e8ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.307+0000 7f87ca59c640 1 -- 172.21.15.8:0/2594182067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b4009a80 con 0x55d139e86250 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87ca59c640 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e8c7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f87bc09ad50 tx=0x7f87bc09a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f617e7835e8033d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2594182067 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0a32d0 con 0x55d139e86250 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2594182067 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f87bc09e020 con 0x55d139e86250 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d139aed7e0 con 0x55d139e86250 2024-02-19T16:25:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2594182067 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87bc09ef10 con 0x55d139e86250 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d139dec6a0 con 0x55d139e86250 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2594182067 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d139aed7e0 con 0x55d139e86250 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2594182067 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f87bc01f060 con 0x55d139e86250 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 20 client.16248 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139e9fb90 0x55d139ea1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 4 client.16248 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d139b2a720 con 0x55d139e9fb90 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.308+0000 7f87c3fff640 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139e9fb90 0x55d139ea1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.309+0000 7f87b1ffb640 10 client.16248 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.309+0000 7f87c3fff640 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139e9fb90 0x55d139ea1fd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52e64cbbd00a012 server_cookie=27048ed29c69fa9 in_seq=0 out_seq=0 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2594182067 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d139b2a720 con 0x55d139e9fb90 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87b1ffb640 10 client.16248 handle_command_reply: tid=0 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 client.16248 shutdown 2024-02-19T16:25:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 msgr2=0x55d139e8c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e8c7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f87bc09ad50 tx=0x7f87bc09a720 comp rx=0 tx=0).stop 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 shutdown_connections 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139e9fb90 0x55d139ea1fd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e8d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e8ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87b1ffb640 0 client.16248 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 --2- 172.21.15.8:0/2594182067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e8c7b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 >> 172.21.15.8:0/2594182067 conn(0x55d139e80980 msgr2=0x55d139e826d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 shutdown_connections 2024-02-19T16:25:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 1 -- 172.21.15.8:0/2594182067 wait complete. 2024-02-19T16:25:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.310+0000 7f87d3565740 20 client.16248 trim_cache size 0 max 16384 2024-02-19T16:25:25.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.322+0000 7f87d3565740 1 Processor -- start 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.322+0000 7f87d3565740 1 -- start start 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.322+0000 7f87d3565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e8eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87d3565740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 0x55d139e8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87d3565740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e9aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87d3565740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139d73130 con 0x55d139e86b70 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87d3565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139e807f0 con 0x55d139e86250 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87d3565740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139e81980 con 0x55d139e8f750 2024-02-19T16:25:25.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 0x55d139e8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 0x55d139e8d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57402/0 (socket says 172.21.15.8:57402) 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 -- 172.21.15.8:0/1027414421 learned_addr learned my addr 172.21.15.8:0/1027414421 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87ca59c640 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e8eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87cad9d640 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e9aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 -- 172.21.15.8:0/1027414421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 msgr2=0x55d139e9aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e9aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 -- 172.21.15.8:0/1027414421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 msgr2=0x55d139e8eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e8eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 -- 172.21.15.8:0/1027414421 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d139d88c20 con 0x55d139e86b70 2024-02-19T16:25:25.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87cad9d640 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e9aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:25:25.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87c3fff640 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 0x55d139e8d050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009250 tx=0x7f87b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4280520ee33d277 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87b3fff640 1 -- 172.21.15.8:0/1027414421 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400eea0 con 0x55d139e86b70 2024-02-19T16:25:25.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87b3fff640 1 -- 172.21.15.8:0/1027414421 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b4019de0 con 0x55d139e86b70 2024-02-19T16:25:25.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.323+0000 7f87b3fff640 1 -- 172.21.15.8:0/1027414421 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e3b0 con 0x55d139e86b70 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- 172.21.15.8:0/1027414421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 msgr2=0x55d139e8d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 0x55d139e8d050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009250 tx=0x7f87b4000bc0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- 172.21.15.8:0/1027414421 shutdown_connections 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139e9aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86b70 0x55d139e8d050 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- 172.21.15.8:0/1027414421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86250 0x55d139e8eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- 172.21.15.8:0/1027414421 >> 172.21.15.8:0/1027414421 conn(0x55d139e80980 msgr2=0x55d139e7fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- 172.21.15.8:0/1027414421 shutdown_connections 2024-02-19T16:25:25.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- 172.21.15.8:0/1027414421 wait complete. 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- start start 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 Processor -- start 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139efa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139e93d40 con 0x55d139e86250 2024-02-19T16:25:25.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139efa9e0 con 0x55d139e86b70 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d139efaba0 con 0x55d139e8f750 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87d3565740 10 client.0 _ll_register_callbacks cb 0x7f87bc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87ca59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e93090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57406/0 (socket says 172.21.15.8:57406) 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87ca59c640 1 -- 172.21.15.8:0/2306175353 learned_addr learned my addr 172.21.15.8:0/2306175353 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87c3fff640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.324+0000 7f87cad9d640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139efa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87ca59c640 1 -- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 msgr2=0x55d139efa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87ca59c640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139efa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87ca59c640 1 -- 172.21.15.8:0/2306175353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 msgr2=0x55d139e935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87ca59c640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87ca59c640 1 -- 172.21.15.8:0/2306175353 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d139db0060 con 0x55d139e86250 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87ca59c640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e93090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f87bc099070 tx=0x7f87bc097110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f005a7554646e78e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2306175353 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc09b4c0 con 0x55d139e86250 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2306175353 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f87bc09e040 con 0x55d139e86250 2024-02-19T16:25:25.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87b400ecb0 con 0x55d139e86250 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2306175353 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87bc09ef10 con 0x55d139e86250 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 10 client.16254 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d139dae8c0 con 0x55d139e86250 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2306175353 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87bc089080 con 0x55d139e86250 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2306175353 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f87bc01f070 con 0x55d139e86250 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 10 client.16254 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:25.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 20 client.16254 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139efbe30 0x55d139efe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 4 client.16254 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.325+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d139b79890 con 0x55d139efbe30 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87c3fff640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139efbe30 0x55d139efe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87b1ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87c3fff640 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139efbe30 0x55d139efe220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3fa828f773c95e8 server_cookie=35cb99d53c9372b2 in_seq=0 out_seq=0 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87b1ffb640 1 -- 172.21.15.8:0/2306175353 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d139b79890 con 0x55d139efbe30 2024-02-19T16:25:25.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87b1ffb640 10 client.16254 handle_command_reply: tid=0 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 client.16254 shutdown 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 msgr2=0x55d139e93090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e93090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f87bc099070 tx=0x7f87bc097110 comp rx=0 tx=0).stop 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 shutdown_connections 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d139efbe30 0x55d139efe220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d139e8f750 0x55d139efa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d139e86b70 0x55d139e935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87b1ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:25.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 --2- 172.21.15.8:0/2306175353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d139e86250 0x55d139e93090 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 >> 172.21.15.8:0/2306175353 conn(0x55d139e80980 msgr2=0x55d139e7fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 shutdown_connections 2024-02-19T16:25:25.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 1 -- 172.21.15.8:0/2306175353 wait complete. 2024-02-19T16:25:25.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.326+0000 7f87d3565740 20 client.16254 trim_cache size 0 max 16384 2024-02-19T16:25:25.344 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:25:25.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87b80a2230 msgr2=0x7f87b80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87b80a2230 0x7f87b80a46f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87c4004af0 tx=0x7f87c4005c50 comp rx=0 tx=0).stop 2024-02-19T16:25:25.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 msgr2=0x7f87cc0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:25.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 0x7f87cc0737f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00e9f0 tx=0x7f87bc00edf0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 shutdown_connections 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87b80a2230 0x7f87b80a46f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87cc107560 0x7f87cc074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc101e60 0x7f87cc073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 --2- 172.21.15.8:0/1886036106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87cc1014a0 0x7f87cc0737f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 >> 172.21.15.8:0/1886036106 conn(0x7f87cc0788f0 msgr2=0x7f87cc100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 shutdown_connections 2024-02-19T16:25:25.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:25.327+0000 7f87d098a640 1 -- 172.21.15.8:0/1886036106 wait complete. 2024-02-19T16:25:25.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:25 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3919114517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:25:25.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:25 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3919114517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:25:25.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:25 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3919114517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:25:26.347 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:25:26.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:25:26.347 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:25:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.465+0000 7f8a4618a640 1 -- 172.21.15.8:0/1374400353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 msgr2=0x7f8a40100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.465+0000 7f8a4618a640 1 --2- 172.21.15.8:0/1374400353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a40100890 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009930 tx=0x7f8a3402bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 -- 172.21.15.8:0/1374400353 shutdown_connections 2024-02-19T16:25:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 --2- 172.21.15.8:0/1374400353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a40100dd0 0x7f8a4010d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 --2- 172.21.15.8:0/1374400353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a40100890 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 --2- 172.21.15.8:0/1374400353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 0x7f8a40106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 -- 172.21.15.8:0/1374400353 >> 172.21.15.8:0/1374400353 conn(0x7f8a40077550 msgr2=0x7f8a40077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 -- 172.21.15.8:0/1374400353 shutdown_connections 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 -- 172.21.15.8:0/1374400353 wait complete. 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 Processor -- start 2024-02-19T16:25:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.466+0000 7f8a4618a640 1 -- start start 2024-02-19T16:25:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a4019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a40100dd0 0x7f8a4019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 0x7f8a40197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a401980e0 con 0x7f8a40100410 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40198250 con 0x7f8a401064b0 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40198410 con 0x7f8a40100dd0 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a4019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a4019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50480/0 (socket says 172.21.15.8:50480) 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 -- 172.21.15.8:0/4289553883 learned_addr learned my addr 172.21.15.8:0/4289553883 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 0x7f8a40197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3effd640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a40100dd0 0x7f8a4019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 -- 172.21.15.8:0/4289553883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a40100dd0 msgr2=0x7f8a4019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a40100dd0 0x7f8a4019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 -- 172.21.15.8:0/4289553883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 msgr2=0x7f8a40197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 0x7f8a40197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 -- 172.21.15.8:0/4289553883 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009560 con 0x7f8a40100410 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 0x7f8a40197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a4019d890 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000e9f0 tx=0x7f8a3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3cff9640 1 -- 172.21.15.8:0/4289553883 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30009700 con 0x7f8a40100410 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3cff9640 1 -- 172.21.15.8:0/4289553883 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a3000cd80 con 0x7f8a40100410 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a401986f0 con 0x7f8a40100410 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a3cff9640 1 -- 172.21.15.8:0/4289553883 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30018920 con 0x7f8a40100410 2024-02-19T16:25:26.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.467+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a401a4930 con 0x7f8a40100410 2024-02-19T16:25:26.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.470+0000 7f8a3cff9640 1 -- 172.21.15.8:0/4289553883 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8a30018a80 con 0x7f8a40100410 2024-02-19T16:25:26.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.470+0000 7f8a48d67740 1 Processor -- start 2024-02-19T16:25:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.470+0000 7f8a48d67740 1 -- start start 2024-02-19T16:25:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64173760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173d30 0x55da6417c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 0x55da6417ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a48d67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da640e1f90 con 0x55da6417c8b0 2024-02-19T16:25:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a48d67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da640df4c0 con 0x55da64173d30 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a48d67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da640daca0 con 0x55da64173380 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 0x55da6417ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64173760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173d30 0x55da6417c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 0x55da6417ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50490/0 (socket says 172.21.15.8:50490) 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 -- 172.21.15.8:0/3322035467 learned_addr learned my addr 172.21.15.8:0/3322035467 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3cff9640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8a280a2230 0x7f8a280a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 -- 172.21.15.8:0/3322035467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 msgr2=0x55da64173760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64173760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 -- 172.21.15.8:0/3322035467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173d30 msgr2=0x55da6417c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173d30 0x55da6417c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 -- 172.21.15.8:0/3322035467 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da6409d0b0 con 0x55da6417c8b0 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8a280a2230 0x7f8a280a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3cff9640 1 -- 172.21.15.8:0/4289553883 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8a30014070 con 0x7f8a40100410 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.471+0000 7f8a3effd640 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 0x55da6417ed00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8a34036040 tx=0x7f8a3402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb0345d55bd5762 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a25ffb640 1 -- 172.21.15.8:0/3322035467 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34004800 con 0x55da6417c8b0 2024-02-19T16:25:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a25ffb640 1 -- 172.21.15.8:0/3322035467 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a34004960 con 0x55da6417c8b0 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a25ffb640 1 -- 172.21.15.8:0/3322035467 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34005120 con 0x55da6417c8b0 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8a280a2230 0x7f8a280a46f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009fd0 tx=0x7f8a38004a40 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 -- 172.21.15.8:0/3322035467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 msgr2=0x55da6417ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 0x55da6417ed00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8a34036040 tx=0x7f8a3402c5d0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 -- 172.21.15.8:0/3322035467 shutdown_connections 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da6417c8b0 0x55da6417ed00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173d30 0x55da6417c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3322035467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64173760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 -- 172.21.15.8:0/3322035467 >> 172.21.15.8:0/3322035467 conn(0x55da6416d840 msgr2=0x55da6416dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 -- 172.21.15.8:0/3322035467 shutdown_connections 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.472+0000 7f8a48d67740 1 -- 172.21.15.8:0/3322035467 wait complete. 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 -- start start 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 Processor -- start 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64179cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da6417a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6407ff40 con 0x55da64173d30 2024-02-19T16:25:26.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da63dae4b0 con 0x55da64173380 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da64060670 con 0x55da6417c8b0 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a48d67740 10 client.0 _ll_register_callbacks cb 0x55da640cdf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64179cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a3f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64179cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50502/0 (socket says 172.21.15.8:50502) 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a3effd640 1 -- 172.21.15.8:0/3174830898 learned_addr learned my addr 172.21.15.8:0/3174830898 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.473+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da6417a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3effd640 1 -- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 msgr2=0x55da6417a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3effd640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da6417a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3effd640 1 -- 172.21.15.8:0/3174830898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 msgr2=0x55da64179780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3effd640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3effd640 1 -- 172.21.15.8:0/3174830898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4010dbe0 con 0x55da64173d30 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a3effd640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64179cc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8a3402c170 tx=0x7f8a340044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f282a84ef65973 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a1effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a1effd640 1 -- 172.21.15.8:0/3174830898 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3404f070 con 0x55da64173d30 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a1effd640 1 -- 172.21.15.8:0/3174830898 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8a34005070 con 0x55da64173d30 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da640b4f40 con 0x55da64173d30 2024-02-19T16:25:26.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a1effd640 1 -- 172.21.15.8:0/3174830898 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a34004300 con 0x55da64173d30 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a48d67740 10 client.16266 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da640d9320 con 0x55da64173d30 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.474+0000 7f8a1effd640 1 -- 172.21.15.8:0/3174830898 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da640b4f40 con 0x55da64173d30 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a1effd640 1 -- 172.21.15.8:0/3174830898 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8a34052020 con 0x55da64173d30 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a48d67740 10 client.16266 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a48d67740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a48d67740 20 client.16266 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da6418c970 0x55da6418edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a48d67740 4 client.16266 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da640d1310 con 0x55da6418c970 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da6418c970 0x55da6418edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da6418c970 0x55da6418edb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e70d9b6ec09f18d server_cookie=75ece9ead230a023 in_seq=0 out_seq=0 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.475+0000 7f8a1effd640 10 client.16266 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a1effd640 1 -- 172.21.15.8:0/3174830898 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55da640d1310 con 0x55da6418c970 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a1effd640 10 client.16266 handle_command_reply: tid=0 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 client.16266 shutdown 2024-02-19T16:25:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 msgr2=0x55da64179cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64179cc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8a3402c170 tx=0x7f8a340044a0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 shutdown_connections 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da6418c970 0x55da6418edb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da6417a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64179cc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a1effd640 0 client.16266 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 --2- 172.21.15.8:0/3174830898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 >> 172.21.15.8:0/3174830898 conn(0x55da6416d840 msgr2=0x55da6416f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 shutdown_connections 2024-02-19T16:25:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 1 -- 172.21.15.8:0/3174830898 wait complete. 2024-02-19T16:25:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.476+0000 7f8a48d67740 20 client.16266 trim_cache size 0 max 16384 2024-02-19T16:25:26.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.492+0000 7f8a48d67740 1 Processor -- start 2024-02-19T16:25:26.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.492+0000 7f8a48d67740 1 -- start start 2024-02-19T16:25:26.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da6417a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da64187d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a48d67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da64175060 con 0x55da64173d30 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a48d67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da64189800 con 0x55da64173380 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a48d67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6417ac10 con 0x55da6417c8b0 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da6417a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da6417a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50516/0 (socket says 172.21.15.8:50516) 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 -- 172.21.15.8:0/566842996 learned_addr learned my addr 172.21.15.8:0/566842996 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da64187d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 -- 172.21.15.8:0/566842996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 msgr2=0x55da64187d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da64187d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 -- 172.21.15.8:0/566842996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 msgr2=0x55da64179cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 -- 172.21.15.8:0/566842996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da64075c00 con 0x55da64173d30 2024-02-19T16:25:26.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a3effd640 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da6417a210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8a34005c40 tx=0x7f8a3402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f522844585c7c25 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a257fa640 1 -- 172.21.15.8:0/566842996 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3404f070 con 0x55da64173d30 2024-02-19T16:25:26.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a257fa640 1 -- 172.21.15.8:0/566842996 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a3403fd00 con 0x55da64173d30 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.493+0000 7f8a257fa640 1 -- 172.21.15.8:0/566842996 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340052b0 con 0x55da64173d30 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 -- 172.21.15.8:0/566842996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 msgr2=0x55da6417a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da6417a210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8a34005c40 tx=0x7f8a3402c5d0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 -- 172.21.15.8:0/566842996 shutdown_connections 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da6417c8b0 0x55da64187d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da6417a210 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 --2- 172.21.15.8:0/566842996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da64173380 0x55da64179cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 -- 172.21.15.8:0/566842996 >> 172.21.15.8:0/566842996 conn(0x55da6416d840 msgr2=0x55da6417b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 -- 172.21.15.8:0/566842996 shutdown_connections 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 -- 172.21.15.8:0/566842996 wait complete. 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.494+0000 7f8a48d67740 1 -- start start 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 Processor -- start 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64183ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64184400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da6417c8b0 0x55da64180390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da641809a0 con 0x55da64173d30 2024-02-19T16:25:26.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da64180b10 con 0x55da6417c8b0 2024-02-19T16:25:26.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da64180cd0 con 0x55da64173380 2024-02-19T16:25:26.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a48d67740 10 client.0 _ll_register_callbacks cb 0x7f8a300d4060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:26.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64184400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64184400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50532/0 (socket says 172.21.15.8:50532) 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 -- 172.21.15.8:0/1870399445 learned_addr learned my addr 172.21.15.8:0/1870399445 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64183ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3ffff640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da6417c8b0 0x55da64180390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 -- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 msgr2=0x55da64183ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64183ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 -- 172.21.15.8:0/1870399445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da6417c8b0 msgr2=0x55da64180390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da6417c8b0 0x55da64180390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 -- 172.21.15.8:0/1870399445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da640d9320 con 0x55da64173d30 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a3effd640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64184400 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8a340045b0 tx=0x7f8a3403fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d4c055d5ecfd9a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a1effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a1effd640 1 -- 172.21.15.8:0/1870399445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3404f040 con 0x55da64173d30 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.495+0000 7f8a1effd640 1 -- 172.21.15.8:0/1870399445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8a34000e40 con 0x55da64173d30 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da64100900 con 0x55da64173d30 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a1effd640 1 -- 172.21.15.8:0/1870399445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3402c9f0 con 0x55da64173d30 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:26.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da6409b910 con 0x55da64173d30 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a1effd640 1 -- 172.21.15.8:0/1870399445 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da64100900 con 0x55da64173d30 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a1effd640 1 -- 172.21.15.8:0/1870399445 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8a34052040 con 0x55da64173d30 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 20 client.16272 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da641ed3c0 0x55da641ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 4 client.16272 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55da63dda860 con 0x55da641ed3c0 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.496+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da641ed3c0 0x55da641ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.497+0000 7f8a3f7fe640 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da641ed3c0 0x55da641ef7e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734c2a6b4dbb0416 server_cookie=84df43b4a8314383 in_seq=0 out_seq=0 2024-02-19T16:25:26.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.497+0000 7f8a1effd640 10 client.16272 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a1effd640 1 -- 172.21.15.8:0/1870399445 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55da63dda860 con 0x55da641ed3c0 2024-02-19T16:25:26.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a1effd640 10 client.16272 handle_command_reply: tid=0 2024-02-19T16:25:26.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 client.16272 shutdown 2024-02-19T16:25:26.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 msgr2=0x55da64184400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64184400 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8a340045b0 tx=0x7f8a3403fe70 comp rx=0 tx=0).stop 2024-02-19T16:25:26.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 shutdown_connections 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da6417c8b0 0x55da64180390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da64173d30 0x55da64184400 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55da641ed3c0 0x55da641ef7e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 --2- 172.21.15.8:0/1870399445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da64173380 0x55da64183ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 >> 172.21.15.8:0/1870399445 conn(0x55da6416d840 msgr2=0x55da6417ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a1effd640 0 client.16272 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 shutdown_connections 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 1 -- 172.21.15.8:0/1870399445 wait complete. 2024-02-19T16:25:26.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.498+0000 7f8a48d67740 20 client.16272 trim_cache size 0 max 16384 2024-02-19T16:25:26.510 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:25:26.510 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:25:26.510 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "e1441f78-6ee7-4482-b25a-abee65a2c063", 2024-02-19T16:25:26.510 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:25:26.510 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:25:26.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8a280a2230 msgr2=0x7f8a280a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8a280a2230 0x7f8a280a46f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009fd0 tx=0x7f8a38004a40 comp rx=0 tx=0).stop 2024-02-19T16:25:26.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 msgr2=0x7f8a4019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a4019d890 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000e9f0 tx=0x7f8a3000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 shutdown_connections 2024-02-19T16:25:26.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a401064b0 0x7f8a40197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8a280a2230 0x7f8a280a46f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a40100dd0 0x7f8a4019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 --2- 172.21.15.8:0/4289553883 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40100410 0x7f8a4019d890 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 >> 172.21.15.8:0/4289553883 conn(0x7f8a40077550 msgr2=0x7f8a40069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 shutdown_connections 2024-02-19T16:25:26.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.499+0000 7f8a4618a640 1 -- 172.21.15.8:0/4289553883 wait complete. 2024-02-19T16:25:26.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:25:26.514 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:25:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 -- 172.21.15.8:0/42041646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 msgr2=0x7f2264101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 --2- 172.21.15.8:0/42041646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f2264101930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f224c009900 tx=0x7f224c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 -- 172.21.15.8:0/42041646 shutdown_connections 2024-02-19T16:25:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 --2- 172.21.15.8:0/42041646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2264101e70 0x7f226410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 --2- 172.21.15.8:0/42041646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f2264101930 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 --2- 172.21.15.8:0/42041646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2264107570 0x7f2264107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 -- 172.21.15.8:0/42041646 >> 172.21.15.8:0/42041646 conn(0x7f22640fcfa0 msgr2=0x7f22640ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 -- 172.21.15.8:0/42041646 shutdown_connections 2024-02-19T16:25:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 -- 172.21.15.8:0/42041646 wait complete. 2024-02-19T16:25:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.620+0000 7f2268888640 1 Processor -- start 2024-02-19T16:25:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 -- start start 2024-02-19T16:25:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f226419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2264101e70 0x7f226419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2264107570 0x7f2264197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2264197ef0 con 0x7f22641014b0 2024-02-19T16:25:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2264198060 con 0x7f2264107570 2024-02-19T16:25:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2268888640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2264198220 con 0x7f2264101e70 2024-02-19T16:25:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f226419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f226419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50542/0 (socket says 172.21.15.8:50542) 2024-02-19T16:25:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 -- 172.21.15.8:0/3636576557 learned_addr learned my addr 172.21.15.8:0/3636576557 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f225bfff640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2264101e70 0x7f226419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f2262d9d640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2264107570 0x7f2264197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 -- 172.21.15.8:0/3636576557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2264101e70 msgr2=0x7f226419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2264101e70 0x7f226419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 -- 172.21.15.8:0/3636576557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2264107570 msgr2=0x7f2264197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2264107570 0x7f2264197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 -- 172.21.15.8:0/3636576557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c009560 con 0x7f22641014b0 2024-02-19T16:25:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.621+0000 7f226259c640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f226419d690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f225400d950 tx=0x7f225400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.622+0000 7f225b7fe640 1 -- 172.21.15.8:0/3636576557 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254009c30 con 0x7f22641014b0 2024-02-19T16:25:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.622+0000 7f225b7fe640 1 -- 172.21.15.8:0/3636576557 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2254009d90 con 0x7f22641014b0 2024-02-19T16:25:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.622+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2264198500 con 0x7f22641014b0 2024-02-19T16:25:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.622+0000 7f225b7fe640 1 -- 172.21.15.8:0/3636576557 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400f7e0 con 0x7f22641014b0 2024-02-19T16:25:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.622+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22641a46c0 con 0x7f22641014b0 2024-02-19T16:25:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.624+0000 7f225b7fe640 1 -- 172.21.15.8:0/3636576557 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f2254010ad0 con 0x7f22641014b0 2024-02-19T16:25:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.624+0000 7f226b46c740 1 Processor -- start 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 -- start start 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f225b7fe640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22500a2230 0x7f22500a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b8222b0 0x55bf0b822690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b82b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b791280 con 0x55bf0b822bd0 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b78e7b0 con 0x55bf0b8222b0 2024-02-19T16:25:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226b46c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b789f90 con 0x55bf0b82b7b0 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f225b7fe640 1 -- 172.21.15.8:0/3636576557 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f22540ce050 con 0x7f22641014b0 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f2262d9d640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22500a2230 0x7f22500a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f225bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b82b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f225bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b82b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50546/0 (socket says 172.21.15.8:50546) 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f2262d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f225bfff640 1 -- 172.21.15.8:0/379948587 learned_addr learned my addr 172.21.15.8:0/379948587 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f226259c640 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b8222b0 0x55bf0b822690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.625+0000 7f2262d9d640 1 -- 172.21.15.8:0/379948587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b8222b0 msgr2=0x55bf0b822690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2262d9d640 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b8222b0 0x55bf0b822690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2262d9d640 1 -- 172.21.15.8:0/379948587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 msgr2=0x55bf0b82b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2262d9d640 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b82b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2262d9d640 1 -- 172.21.15.8:0/379948587 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0b748570 con 0x55bf0b82b7b0 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2262d9d640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22500a2230 0x7f22500a46f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f225c005fd0 tx=0x7f225c009e60 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2262d9d640 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82dba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f225c009ba0 tx=0x7f225c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4eb88a1733ba6b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2258ff9640 1 -- 172.21.15.8:0/379948587 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00ec40 con 0x55bf0b82b7b0 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2258ff9640 1 -- 172.21.15.8:0/379948587 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f225c003e50 con 0x55bf0b82b7b0 2024-02-19T16:25:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.626+0000 7f2258ff9640 1 -- 172.21.15.8:0/379948587 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c004020 con 0x55bf0b82b7b0 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 -- 172.21.15.8:0/379948587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 msgr2=0x55bf0b82dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82dba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f225c009ba0 tx=0x7f225c01d000 comp rx=0 tx=0).stop 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 -- 172.21.15.8:0/379948587 shutdown_connections 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82dba0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b82b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 --2- 172.21.15.8:0/379948587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b8222b0 0x55bf0b822690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 -- 172.21.15.8:0/379948587 >> 172.21.15.8:0/379948587 conn(0x55bf0b81c990 msgr2=0x55bf0b81cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 -- 172.21.15.8:0/379948587 shutdown_connections 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 -- 172.21.15.8:0/379948587 wait complete. 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 -- start start 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.627+0000 7f226b46c740 1 Processor -- start 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b8222b0 0x55bf0b82b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b828600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b82b7b0 0x55bf0b828d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b45d430 con 0x55bf0b822bd0 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b419e00 con 0x55bf0b82b7b0 2024-02-19T16:25:26.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b70efd0 con 0x55bf0b8222b0 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226b46c740 10 client.0 _ll_register_callbacks cb 0x55bf0b764fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b828600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b828600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50550/0 (socket says 172.21.15.8:50550) 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 -- 172.21.15.8:0/4121468023 learned_addr learned my addr 172.21.15.8:0/4121468023 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f226259c640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b8222b0 0x55bf0b82b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f2262d9d640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b82b7b0 0x55bf0b828d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 -- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b8222b0 msgr2=0x55bf0b82b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b8222b0 0x55bf0b82b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 -- 172.21.15.8:0/4121468023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b82b7b0 msgr2=0x55bf0b828d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b82b7b0 0x55bf0b828d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f225bfff640 1 -- 172.21.15.8:0/4121468023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c01b040 con 0x55bf0b822bd0 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.628+0000 7f2262d9d640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b82b7b0 0x55bf0b828d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f225bfff640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b828600 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f224c036040 tx=0x7f224c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab38096c2b07ba09 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f2249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f2249ffb640 1 -- 172.21.15.8:0/4121468023 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c004250 con 0x55bf0b822bd0 2024-02-19T16:25:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf0b4897e0 con 0x55bf0b822bd0 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f2249ffb640 1 -- 172.21.15.8:0/4121468023 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f224c0043b0 con 0x55bf0b822bd0 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f2249ffb640 1 -- 172.21.15.8:0/4121468023 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f224c004df0 con 0x55bf0b822bd0 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 10 client.16284 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bf0b788610 con 0x55bf0b822bd0 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f2249ffb640 1 -- 172.21.15.8:0/4121468023 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bf0b4897e0 con 0x55bf0b822bd0 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f2249ffb640 1 -- 172.21.15.8:0/4121468023 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f224c03c040 con 0x55bf0b822bd0 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 10 client.16284 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:26.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 20 client.16284 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b83ba80 0x55bf0b83dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 4 client.16284 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.629+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf0b4c6720 con 0x55bf0b83ba80 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.630+0000 7f226259c640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b83ba80 0x55bf0b83dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.630+0000 7f2249ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.630+0000 7f226259c640 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b83ba80 0x55bf0b83dea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5289065df9df4c6e server_cookie=6fae245f64f51ce8 in_seq=0 out_seq=0 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f2249ffb640 1 -- 172.21.15.8:0/4121468023 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bf0b4c6720 con 0x55bf0b83ba80 2024-02-19T16:25:26.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f2249ffb640 10 client.16284 handle_command_reply: tid=0 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 client.16284 shutdown 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 msgr2=0x55bf0b828600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b828600 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f224c036040 tx=0x7f224c0048f0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 shutdown_connections 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b82b7b0 0x55bf0b828d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b822bd0 0x55bf0b828600 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b83ba80 0x55bf0b83dea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 --2- 172.21.15.8:0/4121468023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b8222b0 0x55bf0b82b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 >> 172.21.15.8:0/4121468023 conn(0x55bf0b81c990 msgr2=0x55bf0b81e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f2249ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 shutdown_connections 2024-02-19T16:25:26.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 1 -- 172.21.15.8:0/4121468023 wait complete. 2024-02-19T16:25:26.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.631+0000 7f226b46c740 20 client.16284 trim_cache size 0 max 16384 2024-02-19T16:25:26.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.649+0000 7f226b46c740 1 Processor -- start 2024-02-19T16:25:26.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 -- start start 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b8285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b817840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b81d9e0 con 0x55bf0b8222b0 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b821cf0 con 0x55bf0b822bd0 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226b46c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b816e20 con 0x55bf0b82b7b0 2024-02-19T16:25:26.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b8285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b8285a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50556/0 (socket says 172.21.15.8:50556) 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 -- 172.21.15.8:0/2986288709 learned_addr learned my addr 172.21.15.8:0/2986288709 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f225bfff640 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b817840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f2262d9d640 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 -- 172.21.15.8:0/2986288709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 msgr2=0x55bf0b82ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 -- 172.21.15.8:0/2986288709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 msgr2=0x55bf0b817840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b817840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 -- 172.21.15.8:0/2986288709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0b724ac0 con 0x55bf0b8222b0 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f226259c640 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b8285a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f22540cb9f0 tx=0x7f22540ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cd1fe2869cc5f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f224bfff640 1 -- 172.21.15.8:0/2986288709 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225409e7e0 con 0x55bf0b8222b0 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f224bfff640 1 -- 172.21.15.8:0/2986288709 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22540a2e30 con 0x55bf0b8222b0 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.650+0000 7f224bfff640 1 -- 172.21.15.8:0/2986288709 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22540cb670 con 0x55bf0b8222b0 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 -- 172.21.15.8:0/2986288709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 msgr2=0x55bf0b8285a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b8285a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f22540cb9f0 tx=0x7f22540ba040 comp rx=0 tx=0).stop 2024-02-19T16:25:26.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 -- 172.21.15.8:0/2986288709 shutdown_connections 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b82ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b817840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 --2- 172.21.15.8:0/2986288709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b8285a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 -- 172.21.15.8:0/2986288709 >> 172.21.15.8:0/2986288709 conn(0x55bf0b81c990 msgr2=0x55bf0b827e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 -- 172.21.15.8:0/2986288709 shutdown_connections 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.651+0000 7f226b46c740 1 -- 172.21.15.8:0/2986288709 wait complete. 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 -- start start 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 Processor -- start 2024-02-19T16:25:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b832d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b833260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b825400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b8259e0 con 0x55bf0b8222b0 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b825b50 con 0x55bf0b822bd0 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b825d10 con 0x55bf0b82b7b0 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226b46c740 10 client.0 _ll_register_callbacks cb 0x7f224c02c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b832d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b832d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50570/0 (socket says 172.21.15.8:50570) 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 -- 172.21.15.8:0/2329606429 learned_addr learned my addr 172.21.15.8:0/2329606429 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f225bfff640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b833260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f2262d9d640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b825400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 -- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 msgr2=0x55bf0b825400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b825400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 -- 172.21.15.8:0/2329606429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 msgr2=0x55bf0b833260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b833260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 -- 172.21.15.8:0/2329606429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22641a46c0 con 0x55bf0b8222b0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f226259c640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b832d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2254096420 tx=0x7f2254000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65c60c89c3def6a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.652+0000 7f2249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f2249ffb640 1 -- 172.21.15.8:0/2329606429 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254098a40 con 0x55bf0b8222b0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f2249ffb640 1 -- 172.21.15.8:0/2329606429 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f2254049270 con 0x55bf0b8222b0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f225c003e50 con 0x55bf0b8222b0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f2249ffb640 1 -- 172.21.15.8:0/2329606429 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22540501f0 con 0x55bf0b8222b0 2024-02-19T16:25:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 10 client.16290 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bf0b74a6a0 con 0x55bf0b8222b0 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f2249ffb640 1 -- 172.21.15.8:0/2329606429 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f225c003e50 con 0x55bf0b8222b0 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f2249ffb640 1 -- 172.21.15.8:0/2329606429 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f22540ba0b0 con 0x55bf0b8222b0 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 10 client.16290 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 20 client.16290 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b898060 0x55bf0b89a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 4 client.16290 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.653+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf0b515890 con 0x55bf0b898060 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.654+0000 7f225bfff640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b898060 0x55bf0b89a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.654+0000 7f225bfff640 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b898060 0x55bf0b89a480 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a304fa3d2b80410a server_cookie=c17f8c1d3394ae5a in_seq=0 out_seq=0 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.654+0000 7f2249ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f2249ffb640 1 -- 172.21.15.8:0/2329606429 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55bf0b515890 con 0x55bf0b898060 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f2249ffb640 10 client.16290 handle_command_reply: tid=0 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 client.16290 shutdown 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 msgr2=0x55bf0b832d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b832d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2254096420 tx=0x7f2254000bc0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 shutdown_connections 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55bf0b898060 0x55bf0b89a480 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0b82b7b0 0x55bf0b825400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf0b822bd0 0x55bf0b833260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 --2- 172.21.15.8:0/2329606429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bf0b8222b0 0x55bf0b832d20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f2249ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 >> 172.21.15.8:0/2329606429 conn(0x55bf0b81c990 msgr2=0x55bf0b81b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 shutdown_connections 2024-02-19T16:25:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 1 -- 172.21.15.8:0/2329606429 wait complete. 2024-02-19T16:25:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.655+0000 7f226b46c740 20 client.16290 trim_cache size 0 max 16384 2024-02-19T16:25:26.672 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:25:26.672 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2699 inodes in the stack)", 2024-02-19T16:25:26.672 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout: "e1441f78-6ee7-4482-b25a-abee65a2c063": { 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "e1441f78-6ee7-4482-b25a-abee65a2c063", 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:25:26.673 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:25:26.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22500a2230 msgr2=0x7f22500a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22500a2230 0x7f22500a46f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f225c005fd0 tx=0x7f225c009e60 comp rx=0 tx=0).stop 2024-02-19T16:25:26.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 msgr2=0x7f226419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f226419d690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f225400d950 tx=0x7f225400de10 comp rx=0 tx=0).stop 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 shutdown_connections 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2264107570 0x7f2264197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22500a2230 0x7f22500a46f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2264101e70 0x7f226419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 --2- 172.21.15.8:0/3636576557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22641014b0 0x7f226419d690 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 >> 172.21.15.8:0/3636576557 conn(0x7f22640fcfa0 msgr2=0x7f22640fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:26.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 shutdown_connections 2024-02-19T16:25:26.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:26.656+0000 7f2268888640 1 -- 172.21.15.8:0/3636576557 wait complete. 2024-02-19T16:25:26.676 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:26 smithi008 ceph-mon[32573]: pgmap v79: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-02-19T16:25:26.677 INFO:tasks.cephfs.filesystem:scrub status for tag:e1441f78-6ee7-4482-b25a-abee65a2c063 - {'path': '/', 'tag': 'e1441f78-6ee7-4482-b25a-abee65a2c063', 'options': 'recursive,force'} 2024-02-19T16:25:26.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:26 smithi191 ceph-mon[37203]: pgmap v79: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-02-19T16:25:26.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:26 smithi156 ceph-mon[36502]: pgmap v79: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-02-19T16:25:27.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:25:27.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:27.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:27.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:25:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:28.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:28 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:28 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:28 smithi008 ceph-mon[32573]: pgmap v80: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:28 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:28 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:28 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:28 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:28 smithi191 ceph-mon[37203]: pgmap v80: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:28 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:28 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:28.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:28 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:28 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:28 smithi156 ceph-mon[36502]: pgmap v80: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:28 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:28 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:30.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:30 smithi008 ceph-mon[32573]: pgmap v81: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-02-19T16:25:30.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:30 smithi191 ceph-mon[37203]: pgmap v81: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-02-19T16:25:30.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:30 smithi156 ceph-mon[36502]: pgmap v81: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-02-19T16:25:32.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:32 smithi008 ceph-mon[32573]: pgmap v82: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:32.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:32 smithi191 ceph-mon[37203]: pgmap v82: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:32.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:32 smithi156 ceph-mon[36502]: pgmap v82: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:34.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:34 smithi008 ceph-mon[32573]: pgmap v83: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:34 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:25:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:34 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:25:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:34 smithi191 ceph-mon[37203]: pgmap v83: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:34 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:25:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:34 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:25:34.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:34 smithi156 ceph-mon[36502]: pgmap v83: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-19T16:25:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:34 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:25:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:34 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:25:35.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:35 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:25:35.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:35 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:25:35.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:35 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:25:36.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:36 smithi008 ceph-mon[32573]: pgmap v84: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-02-19T16:25:36.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:36 smithi191 ceph-mon[37203]: pgmap v84: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-02-19T16:25:36.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:36 smithi156 ceph-mon[36502]: pgmap v84: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-02-19T16:25:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:38 smithi008 ceph-mon[32573]: pgmap v85: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-02-19T16:25:38.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:38 smithi191 ceph-mon[37203]: pgmap v85: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-02-19T16:25:38.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:38 smithi156 ceph-mon[36502]: pgmap v85: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-02-19T16:25:40.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:40 smithi008 ceph-mon[32573]: pgmap v86: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-02-19T16:25:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:40 smithi191 ceph-mon[37203]: pgmap v86: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-02-19T16:25:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:40 smithi156 ceph-mon[36502]: pgmap v86: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-02-19T16:25:42.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:42 smithi008 ceph-mon[32573]: pgmap v87: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-19T16:25:42.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:42 smithi191 ceph-mon[37203]: pgmap v87: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-19T16:25:42.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:42 smithi156 ceph-mon[36502]: pgmap v87: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-19T16:25:44.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:44 smithi008 ceph-mon[32573]: pgmap v88: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-19T16:25:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:44 smithi191 ceph-mon[37203]: pgmap v88: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-19T16:25:44.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:44 smithi156 ceph-mon[36502]: pgmap v88: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-19T16:25:46.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:46 smithi008 ceph-mon[32573]: pgmap v89: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-02-19T16:25:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:46 smithi191 ceph-mon[37203]: pgmap v89: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-02-19T16:25:46.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:46 smithi156 ceph-mon[36502]: pgmap v89: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-02-19T16:25:48.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:48 smithi008 ceph-mon[32573]: pgmap v90: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:48 smithi191 ceph-mon[37203]: pgmap v90: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:48 smithi156 ceph-mon[36502]: pgmap v90: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:50.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:50 smithi008 ceph-mon[32573]: pgmap v91: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-02-19T16:25:50.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:50 smithi191 ceph-mon[37203]: pgmap v91: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-02-19T16:25:50.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:50 smithi156 ceph-mon[36502]: pgmap v91: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-02-19T16:25:52.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:52 smithi008 ceph-mon[32573]: pgmap v92: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:52.776 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:52 smithi191 ceph-mon[37203]: pgmap v92: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:52.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:52 smithi156 ceph-mon[36502]: pgmap v92: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:54 smithi008 ceph-mon[32573]: pgmap v93: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:54 smithi191 ceph-mon[37203]: pgmap v93: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:54.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:54 smithi156 ceph-mon[36502]: pgmap v93: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-19T16:25:56.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:25:56.678 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:25:56.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:56 smithi008 ceph-mon[32573]: pgmap v94: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-02-19T16:25:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:56 smithi191 ceph-mon[37203]: pgmap v94: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-02-19T16:25:56.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/986292734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c24102410 msgr2=0x7f6c24069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/986292734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c24102410 0x7f6c24069b20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c2002bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/986292734 shutdown_connections 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/986292734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2410dd40 0x7f6c24110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/986292734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2406a0f0 0x7f6c2406a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/986292734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c24102410 0x7f6c24069b20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/986292734 >> 172.21.15.8:0/986292734 conn(0x7f6c24077190 msgr2=0x7f6c240775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.794+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/986292734 shutdown_connections 2024-02-19T16:25:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/986292734 wait complete. 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 Processor -- start 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 -- start start 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 0x7f6c2419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 0x7f6c2419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2410dd40 0x7f6c24199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419a100 con 0x7f6c2406a0f0 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419a270 con 0x7f6c24102410 2024-02-19T16:25:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c2b6a4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419a430 con 0x7f6c2410dd40 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c29440640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 0x7f6c2419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.795+0000 7f6c28c3f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 0x7f6c2419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c28c3f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 0x7f6c2419fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48176/0 (socket says 172.21.15.8:48176) 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 0x7f6c2419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60730/0 (socket says 172.21.15.8:60730) 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c28c3f640 1 -- 172.21.15.8:0/541686223 learned_addr learned my addr 172.21.15.8:0/541686223 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29c41640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2410dd40 0x7f6c24199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 -- 172.21.15.8:0/541686223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2410dd40 msgr2=0x7f6c24199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2410dd40 0x7f6c24199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 -- 172.21.15.8:0/541686223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 msgr2=0x7f6c2419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 0x7f6c2419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 -- 172.21.15.8:0/541686223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009840 con 0x7f6c2406a0f0 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c28c3f640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 0x7f6c2419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c29440640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 0x7f6c2419f900 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c20036040 tx=0x7f6c2002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c1a7fc640 1 -- 172.21.15.8:0/541686223 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004350 con 0x7f6c2406a0f0 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c1a7fc640 1 -- 172.21.15.8:0/541686223 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c200044b0 con 0x7f6c2406a0f0 2024-02-19T16:25:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c20009560 con 0x7f6c2406a0f0 2024-02-19T16:25:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c1a7fc640 1 -- 172.21.15.8:0/541686223 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20003cc0 con 0x7f6c2406a0f0 2024-02-19T16:25:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.796+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c241a67c0 con 0x7f6c2406a0f0 2024-02-19T16:25:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.798+0000 7f6c1a7fc640 1 -- 172.21.15.8:0/541686223 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f6c20005560 con 0x7f6c2406a0f0 2024-02-19T16:25:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 Processor -- start 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 -- start start 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c1a7fc640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c100a2230 0x7f6c100a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982bc2b0 0x5613982bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982c57b0 0x5613982c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139822b280 con 0x5613982bc2b0 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613982287b0 con 0x5613982c57b0 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c2e282740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561398223f90 con 0x5613982bcbd0 2024-02-19T16:25:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.799+0000 7f6c29c41640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c100a2230 0x7f6c100a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982bc2b0 0x5613982bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c28c3f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c28c3f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:49564/0 (socket says 172.21.15.8:49564) 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c1a7fc640 1 -- 172.21.15.8:0/541686223 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6c200f0340 con 0x7f6c2406a0f0 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c28c3f640 1 -- 172.21.15.8:0/656721035 learned_addr learned my addr 172.21.15.8:0/656721035 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29c41640 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982c57b0 0x5613982c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 -- 172.21.15.8:0/656721035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 msgr2=0x5613982c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 -- 172.21.15.8:0/656721035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982c57b0 msgr2=0x5613982c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982c57b0 0x5613982c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 -- 172.21.15.8:0/656721035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613981e2570 con 0x5613982bc2b0 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c28c3f640 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29c41640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c100a2230 0x7f6c100a46f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c24069660 tx=0x7f6c1c013460 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c29440640 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982bc2b0 0x5613982bc690 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6c200ed790 tx=0x7f6c200b8d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980051ba99ec76de server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c0f7fe640 1 -- 172.21.15.8:0/656721035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200f0160 con 0x5613982bc2b0 2024-02-19T16:25:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c0f7fe640 1 -- 172.21.15.8:0/656721035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c20003e20 con 0x5613982bc2b0 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.800+0000 7f6c0f7fe640 1 -- 172.21.15.8:0/656721035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200bca30 con 0x5613982bc2b0 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 -- 172.21.15.8:0/656721035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982bc2b0 msgr2=0x5613982bc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982bc2b0 0x5613982bc690 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6c200ed790 tx=0x7f6c200b8d90 comp rx=0 tx=0).stop 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 -- 172.21.15.8:0/656721035 shutdown_connections 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982c57b0 0x5613982c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 --2- 172.21.15.8:0/656721035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982bc2b0 0x5613982bc690 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 -- 172.21.15.8:0/656721035 >> 172.21.15.8:0/656721035 conn(0x5613982b6990 msgr2=0x5613982b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 -- 172.21.15.8:0/656721035 shutdown_connections 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 -- 172.21.15.8:0/656721035 wait complete. 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 -- start start 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.801+0000 7f6c2e282740 1 Processor -- start 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561397ef7430 con 0x5613982c57b0 2024-02-19T16:25:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561397eb3e00 con 0x5613982bcbd0 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981a8fd0 con 0x5613982bc2b0 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c2e282740 10 client.0 _ll_register_callbacks cb 0x5613981fefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29c41640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:49572/0 (socket says 172.21.15.8:49572) 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 -- 172.21.15.8:0/3719914237 learned_addr learned my addr 172.21.15.8:0/3719914237 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c28c3f640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 -- 172.21.15.8:0/3719914237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 msgr2=0x5613982c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 -- 172.21.15.8:0/3719914237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 msgr2=0x5613982c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29440640 1 -- 172.21.15.8:0/3719914237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c241a67c0 con 0x5613982bc2b0 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.802+0000 7f6c29c41640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c29440640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c5150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c200bc670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d56fbdf1e089d44 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3719914237 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200f0160 con 0x5613982bc2b0 2024-02-19T16:25:56.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3719914237 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6c20090e60 con 0x5613982bc2b0 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561397f237e0 con 0x5613982bc2b0 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3719914237 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c200bd130 con 0x5613982bc2b0 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 10 client.26170 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561398222610 con 0x5613982bc2b0 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3719914237 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561397f237e0 con 0x5613982bc2b0 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3719914237 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6c2003c070 con 0x5613982bc2b0 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 10 client.26170 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:56.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 20 client.26170 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5613982d52b0 0x5613982d76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 4 client.26170 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.803+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561397f60720 con 0x5613982d52b0 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.804+0000 7f6c28c3f640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5613982d52b0 0x5613982d76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.804+0000 7f6c0cff9640 10 client.26170 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.804+0000 7f6c28c3f640 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5613982d52b0 0x5613982d76d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c370376a6ace9b99 server_cookie=f5c7f38add70556d in_seq=0 out_seq=0 2024-02-19T16:25:56.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3719914237 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561397f60720 con 0x5613982d52b0 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c0cff9640 10 client.26170 handle_command_reply: tid=0 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 client.26170 shutdown 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 msgr2=0x5613982c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c5150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c200bc670 comp rx=0 tx=0).stop 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 shutdown_connections 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5613982d52b0 0x5613982d76d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3719914237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c5150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c0cff9640 0 client.26170 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:56.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 >> 172.21.15.8:0/3719914237 conn(0x5613982b6990 msgr2=0x5613982b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 shutdown_connections 2024-02-19T16:25:56.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 1 -- 172.21.15.8:0/3719914237 wait complete. 2024-02-19T16:25:56.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.805+0000 7f6c2e282740 20 client.26170 trim_cache size 0 max 16384 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.821+0000 7f6c2e282740 1 Processor -- start 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 -- start start 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561397f237e0 con 0x5613982c57b0 2024-02-19T16:25:56.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613982bbf50 con 0x5613982bcbd0 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c2e282740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613982b20f0 con 0x5613982bc2b0 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29440640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60758/0 (socket says 172.21.15.8:60758) 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 -- 172.21.15.8:0/2424541149 learned_addr learned my addr 172.21.15.8:0/2424541149 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c28c3f640 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 -- 172.21.15.8:0/2424541149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 msgr2=0x5613982c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 -- 172.21.15.8:0/2424541149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 msgr2=0x5613982c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 -- 172.21.15.8:0/2424541149 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613981beac0 con 0x5613982c57b0 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c29c41640 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c5150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c013770 tx=0x7f6c1c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e416bba191d1f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.822+0000 7f6c0effd640 1 -- 172.21.15.8:0/2424541149 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c003e00 con 0x5613982c57b0 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c0effd640 1 -- 172.21.15.8:0/2424541149 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c0130c0 con 0x5613982c57b0 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c0effd640 1 -- 172.21.15.8:0/2424541149 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c02bc90 con 0x5613982c57b0 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 -- 172.21.15.8:0/2424541149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 msgr2=0x5613982c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c5150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c013770 tx=0x7f6c1c009500 comp rx=0 tx=0).stop 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 -- 172.21.15.8:0/2424541149 shutdown_connections 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c5150 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bcbd0 0x5613982c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 --2- 172.21.15.8:0/2424541149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bc2b0 0x5613982c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 -- 172.21.15.8:0/2424541149 >> 172.21.15.8:0/2424541149 conn(0x5613982b6990 msgr2=0x5613982c1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 -- 172.21.15.8:0/2424541149 shutdown_connections 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 -- 172.21.15.8:0/2424541149 wait complete. 2024-02-19T16:25:56.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 -- start start 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.823+0000 7f6c2e282740 1 Processor -- start 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bc2b0 0x5613982bfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613982c69e0 con 0x5613982c57b0 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613982c6b50 con 0x5613982bc2b0 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613982c6d10 con 0x5613982bcbd0 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c2e282740 10 client.0 _ll_register_callbacks cb 0x7f6c200035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:56.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c28c3f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60762/0 (socket says 172.21.15.8:60762) 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 -- 172.21.15.8:0/3192849810 learned_addr learned my addr 172.21.15.8:0/3192849810 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29440640 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bc2b0 0x5613982bfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 -- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 msgr2=0x5613982c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 -- 172.21.15.8:0/3192849810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bc2b0 msgr2=0x5613982bfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:56.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bc2b0 0x5613982bfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.824+0000 7f6c29c41640 1 -- 172.21.15.8:0/3192849810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c01b040 con 0x5613982c57b0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c29c41640 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c64a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c005cc0 tx=0x7f6c1c009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f15b87199ac15ae server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3192849810 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c02b690 con 0x5613982c57b0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3192849810 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f6c1c004a40 con 0x5613982c57b0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c20003e20 con 0x5613982c57b0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3192849810 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c1c02b7f0 con 0x5613982c57b0 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 10 client.16308 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:56.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5613981e46a0 con 0x5613982c57b0 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3192849810 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c20003e20 con 0x5613982c57b0 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3192849810 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6c1c005410 con 0x5613982c57b0 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 10 client.16308 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 20 client.16308 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561398331b00 0x561398333f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 4 client.16308 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561397faf890 con 0x561398331b00 2024-02-19T16:25:56.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.825+0000 7f6c29440640 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561398331b00 0x561398333f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c29440640 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561398331b00 0x561398333f20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d5da66f296901f server_cookie=f424b1e659a82cb2 in_seq=0 out_seq=0 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c0cff9640 10 client.16308 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c0cff9640 1 -- 172.21.15.8:0/3192849810 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561397faf890 con 0x561398331b00 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c0cff9640 10 client.16308 handle_command_reply: tid=0 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 client.16308 shutdown 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 msgr2=0x5613982c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c64a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c005cc0 tx=0x7f6c1c009af0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 shutdown_connections 2024-02-19T16:25:56.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5613982c57b0 0x5613982c64a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561398331b00 0x561398333f20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5613982bcbd0 0x5613982c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c0cff9640 0 client.16308 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 --2- 172.21.15.8:0/3192849810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613982bc2b0 0x5613982bfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 >> 172.21.15.8:0/3192849810 conn(0x5613982b6990 msgr2=0x5613982b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 shutdown_connections 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 1 -- 172.21.15.8:0/3192849810 wait complete. 2024-02-19T16:25:56.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.826+0000 7f6c2e282740 20 client.16308 trim_cache size 0 max 16384 2024-02-19T16:25:56.839 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:25:56.839 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:25:56.839 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:25:56.839 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c100a2230 msgr2=0x7f6c100a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c100a2230 0x7f6c100a46f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c24069660 tx=0x7f6c1c013460 comp rx=0 tx=0).stop 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 msgr2=0x7f6c2419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 0x7f6c2419f900 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c20036040 tx=0x7f6c2002c680 comp rx=0 tx=0).stop 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 shutdown_connections 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6c100a2230 0x7f6c100a46f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2410dd40 0x7f6c24199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24102410 0x7f6c2419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 --2- 172.21.15.8:0/541686223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c2406a0f0 0x7f6c2419f900 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 >> 172.21.15.8:0/541686223 conn(0x7f6c24077190 msgr2=0x7f6c2410eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 shutdown_connections 2024-02-19T16:25:56.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.827+0000 7f6c2b6a4640 1 -- 172.21.15.8:0/541686223 wait complete. 2024-02-19T16:25:56.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:25:56.842 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:25:56.843 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:56 smithi156 ceph-mon[36502]: pgmap v94: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-02-19T16:25:56.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 -- 172.21.15.8:0/3768763232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c1003b0 msgr2=0x7f8b5c100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/3768763232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c1003b0 0x7f8b5c100810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009900 tx=0x7f8b5002bcd0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 -- 172.21.15.8:0/3768763232 shutdown_connections 2024-02-19T16:25:56.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/3768763232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c100d50 0x7f8b5c10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/3768763232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c1003b0 0x7f8b5c100810 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/3768763232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b5c106470 0x7f8b5c106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 -- 172.21.15.8:0/3768763232 >> 172.21.15.8:0/3768763232 conn(0x7f8b5c077480 msgr2=0x7f8b5c077890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:56.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 -- 172.21.15.8:0/3768763232 shutdown_connections 2024-02-19T16:25:56.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.944+0000 7f8b61c1c640 1 -- 172.21.15.8:0/3768763232 wait complete. 2024-02-19T16:25:56.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 Processor -- start 2024-02-19T16:25:56.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 -- start start 2024-02-19T16:25:56.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b5c1003b0 0x7f8b5c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 0x7f8b5c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c106470 0x7f8b5c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c19a100 con 0x7f8b5c100d50 2024-02-19T16:25:56.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c19a270 con 0x7f8b5c106470 2024-02-19T16:25:56.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b61c1c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c19a430 con 0x7f8b5c1003b0 2024-02-19T16:25:56.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b5affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 0x7f8b5c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.945+0000 7f8b5affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 0x7f8b5c19fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60770/0 (socket says 172.21.15.8:60770) 2024-02-19T16:25:56.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 -- 172.21.15.8:0/316350504 learned_addr learned my addr 172.21.15.8:0/316350504 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5b7fe640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b5c1003b0 0x7f8b5c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5bfff640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c106470 0x7f8b5c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 -- 172.21.15.8:0/316350504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b5c1003b0 msgr2=0x7f8b5c19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b5c1003b0 0x7f8b5c19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 -- 172.21.15.8:0/316350504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c106470 msgr2=0x7f8b5c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c106470 0x7f8b5c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 -- 172.21.15.8:0/316350504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50009560 con 0x7f8b5c100d50 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b5affd640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 0x7f8b5c19fe40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b5002c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5002dbe0 con 0x7f8b5c100d50 2024-02-19T16:25:56.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c19a6b0 con 0x7f8b5c100d50 2024-02-19T16:25:56.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b5002dd40 con 0x7f8b5c100d50 2024-02-19T16:25:56.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50003b20 con 0x7f8b5c100d50 2024-02-19T16:25:56.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.946+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1a69a0 con 0x7f8b5c100d50 2024-02-19T16:25:56.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.947+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c067a50 con 0x7f8b5c100d50 2024-02-19T16:25:56.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.948+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8b50003c80 con 0x7f8b5c100d50 2024-02-19T16:25:56.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.949+0000 7f8b58ff9640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b440a2230 0x7f8b440a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:56.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.949+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8b500f08d0 con 0x7f8b5c100d50 2024-02-19T16:25:56.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.949+0000 7f8b5b7fe640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b440a2230 0x7f8b440a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:56.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.949+0000 7f8b5b7fe640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b440a2230 0x7f8b440a46f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c19abf0 tx=0x7f8b4c005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:56.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:56.953+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8b500b9350 con 0x7f8b5c100d50 2024-02-19T16:25:57.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.174+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b5c110500 con 0x7f8b5c100d50 2024-02-19T16:25:57.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.174+0000 7f8b58ff9640 1 -- 172.21.15.8:0/316350504 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f8b500bca10 con 0x7f8b5c100d50 2024-02-19T16:25:57.176 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:25:57.176 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:24:42.507626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:25:57.176 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 27 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b440a2230 msgr2=0x7f8b440a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b440a2230 0x7f8b440a46f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c19abf0 tx=0x7f8b4c005f90 comp rx=0 tx=0).stop 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 msgr2=0x7f8b5c19fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 0x7f8b5c19fe40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b5002c6d0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 shutdown_connections 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c106470 0x7f8b5c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.176+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b5c100d50 0x7f8b5c19fe40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.177+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b440a2230 0x7f8b440a46f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.177+0000 7f8b61c1c640 1 --2- 172.21.15.8:0/316350504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b5c1003b0 0x7f8b5c19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.177+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 >> 172.21.15.8:0/316350504 conn(0x7f8b5c077480 msgr2=0x7f8b5c069890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.177+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 shutdown_connections 2024-02-19T16:25:57.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.177+0000 7f8b61c1c640 1 -- 172.21.15.8:0/316350504 wait complete. 2024-02-19T16:25:57.189 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:25:57.190 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:25:57.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 -- 172.21.15.8:0/3658090875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 msgr2=0x7fc3ec10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/3658090875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec10df30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e402c180 comp rx=0 tx=0).stop 2024-02-19T16:25:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 -- 172.21.15.8:0/3658090875 shutdown_connections 2024-02-19T16:25:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/3658090875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec10df30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/3658090875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3ec103050 0x7fc3ec10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/3658090875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3ec102730 0x7fc3ec102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.293+0000 7fc3f3c50640 1 -- 172.21.15.8:0/3658090875 >> 172.21.15.8:0/3658090875 conn(0x7fc3ec076770 msgr2=0x7fc3ec076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 -- 172.21.15.8:0/3658090875 shutdown_connections 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 -- 172.21.15.8:0/3658090875 wait complete. 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 Processor -- start 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 -- start start 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3ec102730 0x7fc3ec19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3ec103050 0x7fc3ec19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec197f20 con 0x7fc3ec10bb00 2024-02-19T16:25:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec198090 con 0x7fc3ec103050 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f3c50640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec198250 con 0x7fc3ec102730 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f21ed640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f21ed640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec1978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60784/0 (socket says 172.21.15.8:60784) 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f21ed640 1 -- 172.21.15.8:0/382774877 learned_addr learned my addr 172.21.15.8:0/382774877 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3ec103050 0x7fc3ec19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.294+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3ec102730 0x7fc3ec19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f21ed640 1 -- 172.21.15.8:0/382774877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3ec102730 msgr2=0x7fc3ec19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f21ed640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3ec102730 0x7fc3ec19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f21ed640 1 -- 172.21.15.8:0/382774877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3ec103050 msgr2=0x7fc3ec19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f21ed640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3ec103050 0x7fc3ec19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f21ed640 1 -- 172.21.15.8:0/382774877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8009490 con 0x7fc3ec10bb00 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f21ed640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec1978d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc3e402c790 tx=0x7fc3e40346c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3e2ffd640 1 -- 172.21.15.8:0/382774877 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4003d80 con 0x7fc3ec10bb00 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3e2ffd640 1 -- 172.21.15.8:0/382774877 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3e40348b0 con 0x7fc3ec10bb00 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e4009900 con 0x7fc3ec10bb00 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3e2ffd640 1 -- 172.21.15.8:0/382774877 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4004790 con 0x7fc3ec10bb00 2024-02-19T16:25:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.295+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec1a45e0 con 0x7fc3ec10bb00 2024-02-19T16:25:57.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.297+0000 7fc3e2ffd640 1 -- 172.21.15.8:0/382774877 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fc3e400a040 con 0x7fc3ec10bb00 2024-02-19T16:25:57.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 Processor -- start 2024-02-19T16:25:57.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 -- start start 2024-02-19T16:25:57.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bdb70 0x555dd58c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58c6750 0x555dd58c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd5878960 con 0x555dd58bdb70 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd582c310 con 0x555dd58bd250 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f6820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd5829840 con 0x555dd58c6750 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f19ec640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bdb70 0x555dd58c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3e2ffd640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc3d80a2230 0x7fc3d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f19ec640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bdb70 0x555dd58c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60790/0 (socket says 172.21.15.8:60790) 2024-02-19T16:25:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58bd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48268/0 (socket says 172.21.15.8:48268) 2024-02-19T16:25:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f11eb640 1 -- 172.21.15.8:0/1867480260 learned_addr learned my addr 172.21.15.8:0/1867480260 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58c6750 0x555dd58c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f21ed640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc3d80a2230 0x7fc3d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.298+0000 7fc3f11eb640 1 -- 172.21.15.8:0/1867480260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58c6750 msgr2=0x555dd58c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58c6750 0x555dd58c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3f11eb640 1 -- 172.21.15.8:0/1867480260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bdb70 msgr2=0x555dd58c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bdb70 0x555dd58c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3f11eb640 1 -- 172.21.15.8:0/1867480260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd57e7060 con 0x555dd58bd250 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3e2ffd640 1 -- 172.21.15.8:0/382774877 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fc3e40f0ee0 con 0x7fc3ec10bb00 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58bd630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80058c0 tx=0x7fc3e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afbe262e9a839666 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3d7fff640 1 -- 172.21.15.8:0/1867480260 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8004050 con 0x555dd58bd250 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3f21ed640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc3d80a2230 0x7fc3d80a46f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3e404a000 tx=0x7fc3e4097040 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3d7fff640 1 -- 172.21.15.8:0/1867480260 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3e800fca0 con 0x555dd58bd250 2024-02-19T16:25:57.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.299+0000 7fc3d7fff640 1 -- 172.21.15.8:0/1867480260 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8003a50 con 0x555dd58bd250 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- 172.21.15.8:0/1867480260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 msgr2=0x555dd58bd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58bd630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80058c0 tx=0x7fc3e8004730 comp rx=0 tx=0).stop 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- 172.21.15.8:0/1867480260 shutdown_connections 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58c6750 0x555dd58c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bdb70 0x555dd58c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- 172.21.15.8:0/1867480260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58bd630 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- 172.21.15.8:0/1867480260 >> 172.21.15.8:0/1867480260 conn(0x555dd58b7980 msgr2=0x555dd58b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- 172.21.15.8:0/1867480260 shutdown_connections 2024-02-19T16:25:57.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- 172.21.15.8:0/1867480260 wait complete. 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- start start 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 Processor -- start 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58c3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd58c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.300+0000 7fc3f6820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd5825020 con 0x555dd58bd250 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f6820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd54f8430 con 0x555dd58c6750 2024-02-19T16:25:57.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f6820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd54b4e00 con 0x555dd58bdb70 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f6820740 10 client.0 _ll_register_callbacks cb 0x555dd57cccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58c3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60802/0 (socket says 172.21.15.8:60802) 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 -- 172.21.15.8:0/464038124 learned_addr learned my addr 172.21.15.8:0/464038124 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd58c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 -- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 msgr2=0x555dd58c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 -- 172.21.15.8:0/464038124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 msgr2=0x555dd58c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd58c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 -- 172.21.15.8:0/464038124 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e8019040 con 0x555dd58bd250 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58c3780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800fc30 tx=0x7fc3e8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae8403a4001f4ab server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3d57fa640 1 -- 172.21.15.8:0/464038124 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e801f650 con 0x555dd58bd250 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3d57fa640 1 -- 172.21.15.8:0/464038124 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fc3e8011e40 con 0x555dd58bd250 2024-02-19T16:25:57.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dd55247e0 con 0x555dd58bd250 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3d57fa640 1 -- 172.21.15.8:0/464038124 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3e801f7b0 con 0x555dd58bd250 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f6820740 10 client.16326 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.301+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555dd58236a0 con 0x555dd58bd250 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3d57fa640 1 -- 172.21.15.8:0/464038124 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555dd55247e0 con 0x555dd58bd250 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3d57fa640 1 -- 172.21.15.8:0/464038124 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc3e801d070 con 0x555dd58bd250 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f6820740 10 client.16326 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f6820740 10 client.16326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:25:57.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f6820740 20 client.16326 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f6820740 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd58d6950 0x555dd58d8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f6820740 4 client.16326 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dd5561720 con 0x555dd58d6950 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd58d6950 0x555dd58d8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd58d6950 0x555dd58d8d70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c43739ccfaa79ae server_cookie=53d1de369f617bd4 in_seq=0 out_seq=0 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.302+0000 7fc3d57fa640 10 client.16326 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3d57fa640 1 -- 172.21.15.8:0/464038124 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555dd5561720 con 0x555dd58d6950 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3d57fa640 10 client.16326 handle_command_reply: tid=0 2024-02-19T16:25:57.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 client.16326 shutdown 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 msgr2=0x555dd58c3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58c3780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800fc30 tx=0x7fc3e8003730 comp rx=0 tx=0).stop 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 shutdown_connections 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd58d6950 0x555dd58d8d70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd58c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 --2- 172.21.15.8:0/464038124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58c3780 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3d57fa640 0 client.16326 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:57.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 >> 172.21.15.8:0/464038124 conn(0x555dd58b7980 msgr2=0x555dd58b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 shutdown_connections 2024-02-19T16:25:57.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 1 -- 172.21.15.8:0/464038124 wait complete. 2024-02-19T16:25:57.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.303+0000 7fc3f6820740 20 client.16326 trim_cache size 0 max 16384 2024-02-19T16:25:57.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 Processor -- start 2024-02-19T16:25:57.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 -- start start 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 0x555dd58d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd57aa130 con 0x555dd58c6750 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd58d6750 con 0x555dd58bd250 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f6820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd58bcee0 con 0x555dd58bdb70 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 0x555dd58d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f19ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 0x555dd58d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60812/0 (socket says 172.21.15.8:60812) 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 -- 172.21.15.8:0/668589569 learned_addr learned my addr 172.21.15.8:0/668589569 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 -- 172.21.15.8:0/668589569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 msgr2=0x555dd58c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 -- 172.21.15.8:0/668589569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 msgr2=0x555dd58c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 -- 172.21.15.8:0/668589569 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd57bfc20 con 0x555dd58c6750 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 0x555dd58d1ad0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800ed80 tx=0x7fc3e8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a092de97940eaa server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3d77fe640 1 -- 172.21.15.8:0/668589569 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800fad0 con 0x555dd58c6750 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3d77fe640 1 -- 172.21.15.8:0/668589569 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3e800eea0 con 0x555dd58c6750 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.316+0000 7fc3d77fe640 1 -- 172.21.15.8:0/668589569 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800e730 con 0x555dd58c6750 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 -- 172.21.15.8:0/668589569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 msgr2=0x555dd58d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 0x555dd58d1ad0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800ed80 tx=0x7fc3e8013040 comp rx=0 tx=0).stop 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 -- 172.21.15.8:0/668589569 shutdown_connections 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58c6750 0x555dd58d1ad0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 --2- 172.21.15.8:0/668589569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58bd250 0x555dd58c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 -- 172.21.15.8:0/668589569 >> 172.21.15.8:0/668589569 conn(0x555dd58b7980 msgr2=0x555dd58c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 -- 172.21.15.8:0/668589569 shutdown_connections 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 -- 172.21.15.8:0/668589569 wait complete. 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 -- start start 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.317+0000 7fc3f6820740 1 Processor -- start 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd592a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd592a850 con 0x555dd58bd250 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd592a9c0 con 0x555dd58c6750 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd592ab80 con 0x555dd58bdb70 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 10 client.0 _ll_register_callbacks cb 0x7fc3dc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60820/0 (socket says 172.21.15.8:60820) 2024-02-19T16:25:57.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 -- 172.21.15.8:0/2457645102 learned_addr learned my addr 172.21.15.8:0/2457645102 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd592a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 -- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 msgr2=0x555dd58caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 -- 172.21.15.8:0/2457645102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 msgr2=0x555dd592a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd592a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 -- 172.21.15.8:0/2457645102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dd58236a0 con 0x555dd58bd250 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f11eb640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58ca5a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80118c0 tx=0x7fc3e800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91dab0cb11de3d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3d57fa640 1 -- 172.21.15.8:0/2457645102 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800fad0 con 0x555dd58bd250 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3e800fca0 con 0x555dd58bd250 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.318+0000 7fc3d57fa640 1 -- 172.21.15.8:0/2457645102 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fc3e8000e40 con 0x555dd58bd250 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3d57fa640 1 -- 172.21.15.8:0/2457645102 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3e800e6e0 con 0x555dd58bd250 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 10 client.16332 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555dd57e58c0 con 0x555dd58bd250 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3d57fa640 1 -- 172.21.15.8:0/2457645102 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3e800fca0 con 0x555dd58bd250 2024-02-19T16:25:57.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3d57fa640 1 -- 172.21.15.8:0/2457645102 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc3e801d070 con 0x555dd58bd250 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 10 client.16332 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 20 client.16332 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd5932f20 0x555dd5935310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 4 client.16332 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555dd55b0890 con 0x555dd5932f20 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.319+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd5932f20 0x555dd5935310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3d57fa640 10 client.16332 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f19ec640 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd5932f20 0x555dd5935310 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56d30cf298a33541 server_cookie=17ec81ca70c87e3a in_seq=0 out_seq=0 2024-02-19T16:25:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3d57fa640 1 -- 172.21.15.8:0/2457645102 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555dd55b0890 con 0x555dd5932f20 2024-02-19T16:25:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3d57fa640 10 client.16332 handle_command_reply: tid=0 2024-02-19T16:25:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 client.16332 shutdown 2024-02-19T16:25:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 msgr2=0x555dd58ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58ca5a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80118c0 tx=0x7fc3e800ebf0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 shutdown_connections 2024-02-19T16:25:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x555dd5932f20 0x555dd5935310 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd58c6750 0x555dd592a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555dd58bdb70 0x555dd58caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 --2- 172.21.15.8:0/2457645102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555dd58bd250 0x555dd58ca5a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3d57fa640 0 client.16332 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:25:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 >> 172.21.15.8:0/2457645102 conn(0x555dd58b7980 msgr2=0x555dd58c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 shutdown_connections 2024-02-19T16:25:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 1 -- 172.21.15.8:0/2457645102 wait complete. 2024-02-19T16:25:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f6820740 20 client.16332 trim_cache size 0 max 16384 2024-02-19T16:25:57.336 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:25:57.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc3d80a2230 msgr2=0x7fc3d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc3d80a2230 0x7fc3d80a46f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3e404a000 tx=0x7fc3e4097040 comp rx=0 tx=0).stop 2024-02-19T16:25:57.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 msgr2=0x7fc3ec1978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.320+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec1978d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc3e402c790 tx=0x7fc3e40346c0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 shutdown_connections 2024-02-19T16:25:57.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3ec10bb00 0x7fc3ec1978d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3ec103050 0x7fc3ec19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc3d80a2230 0x7fc3d80a46f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 --2- 172.21.15.8:0/382774877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3ec102730 0x7fc3ec19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 >> 172.21.15.8:0/382774877 conn(0x7fc3ec076770 msgr2=0x7fc3ec0fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 shutdown_connections 2024-02-19T16:25:57.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.321+0000 7fc3f3c50640 1 -- 172.21.15.8:0/382774877 wait complete. 2024-02-19T16:25:57.340 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:25:57.340 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:25:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 -- 172.21.15.8:0/1248196290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100dd0 msgr2=0x7f125010d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 --2- 172.21.15.8:0/1248196290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100dd0 0x7f125010d2e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1248009ca0 tx=0x7f124802c100 comp rx=0 tx=0).stop 2024-02-19T16:25:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 -- 172.21.15.8:0/1248196290 shutdown_connections 2024-02-19T16:25:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 --2- 172.21.15.8:0/1248196290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100dd0 0x7f125010d2e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 --2- 172.21.15.8:0/1248196290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100410 0x7f1250100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 --2- 172.21.15.8:0/1248196290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501064b0 0x7f1250106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 -- 172.21.15.8:0/1248196290 >> 172.21.15.8:0/1248196290 conn(0x7f1250077550 msgr2=0x7f1250077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 -- 172.21.15.8:0/1248196290 shutdown_connections 2024-02-19T16:25:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.444+0000 7f12563c7640 1 -- 172.21.15.8:0/1248196290 wait complete. 2024-02-19T16:25:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 Processor -- start 2024-02-19T16:25:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 -- start start 2024-02-19T16:25:57.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100410 0x7f125019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 0x7f125019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12501064b0 0x7f12501979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1250198000 con 0x7f1250100dd0 2024-02-19T16:25:57.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1250198170 con 0x7f12501064b0 2024-02-19T16:25:57.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f12563c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1250198330 con 0x7f1250100410 2024-02-19T16:25:57.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 0x7f125019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 0x7f125019dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60844/0 (socket says 172.21.15.8:60844) 2024-02-19T16:25:57.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 -- 172.21.15.8:0/2815306428 learned_addr learned my addr 172.21.15.8:0/2815306428 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124ffff640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100410 0x7f125019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f1254964640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12501064b0 0x7f12501979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 -- 172.21.15.8:0/2815306428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100410 msgr2=0x7f125019d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100410 0x7f125019d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 -- 172.21.15.8:0/2815306428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12501064b0 msgr2=0x7f12501979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12501064b0 0x7f12501979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.445+0000 7f124f7fe640 1 -- 172.21.15.8:0/2815306428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009490 con 0x7f1250100dd0 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.446+0000 7f124f7fe640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 0x7f125019dd40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f124400e9f0 tx=0x7f124400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.446+0000 7f124d7fa640 1 -- 172.21.15.8:0/2815306428 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244009720 con 0x7f1250100dd0 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.446+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1248009900 con 0x7f1250100dd0 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.446+0000 7f124d7fa640 1 -- 172.21.15.8:0/2815306428 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124400cd80 con 0x7f1250100dd0 2024-02-19T16:25:57.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.446+0000 7f124d7fa640 1 -- 172.21.15.8:0/2815306428 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244010690 con 0x7f1250100dd0 2024-02-19T16:25:57.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.446+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12501a46c0 con 0x7f1250100dd0 2024-02-19T16:25:57.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.448+0000 7f124d7fa640 1 -- 172.21.15.8:0/2815306428 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f12440046c0 con 0x7f1250100dd0 2024-02-19T16:25:57.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.448+0000 7f1258fa0740 1 Processor -- start 2024-02-19T16:25:57.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.448+0000 7f1258fa0740 1 -- start start 2024-02-19T16:25:57.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 0x556f64c73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 0x556f64c7c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1258fa0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64c2e960 con 0x556f64c73b70 2024-02-19T16:25:57.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1258fa0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64be2310 con 0x556f64c7c750 2024-02-19T16:25:57.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1258fa0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64bdf840 con 0x556f64c73250 2024-02-19T16:25:57.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f124d7fa640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f12380a2230 0x7f12380a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 0x556f64c7c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 0x556f64c73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 0x556f64c73630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:49668/0 (socket says 172.21.15.8:49668) 2024-02-19T16:25:57.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 0x556f64c7c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60856/0 (socket says 172.21.15.8:60856) 2024-02-19T16:25:57.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 -- 172.21.15.8:0/420550285 learned_addr learned my addr 172.21.15.8:0/420550285 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f124f7fe640 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f12380a2230 0x7f12380a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f124d7fa640 1 -- 172.21.15.8:0/2815306428 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f1244014070 con 0x7f1250100dd0 2024-02-19T16:25:57.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 -- 172.21.15.8:0/420550285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 msgr2=0x556f64c7eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 -- 172.21.15.8:0/420550285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 msgr2=0x556f64c7c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 0x556f64c7c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 -- 172.21.15.8:0/420550285 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f64b9d060 con 0x556f64c73250 2024-02-19T16:25:57.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f124ffff640 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 0x556f64c7c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:57.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.449+0000 7f1254964640 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 0x556f64c73630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1248009430 tx=0x7f1248032ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a42b24ed8a7bc server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1254964640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f12380a2230 0x7f12380a46f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1248037040 tx=0x7f124803c000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f12367fc640 1 -- 172.21.15.8:0/420550285 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248048070 con 0x556f64c73250 2024-02-19T16:25:57.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f12367fc640 1 -- 172.21.15.8:0/420550285 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1248032e00 con 0x556f64c73250 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f12367fc640 1 -- 172.21.15.8:0/420550285 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248004240 con 0x556f64c73250 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 -- 172.21.15.8:0/420550285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 msgr2=0x556f64c73630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 0x556f64c73630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1248009430 tx=0x7f1248032ad0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 -- 172.21.15.8:0/420550285 shutdown_connections 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73b70 0x556f64c7c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 --2- 172.21.15.8:0/420550285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73250 0x556f64c73630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 -- 172.21.15.8:0/420550285 >> 172.21.15.8:0/420550285 conn(0x556f64c6d980 msgr2=0x556f64c6dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 -- 172.21.15.8:0/420550285 shutdown_connections 2024-02-19T16:25:57.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.450+0000 7f1258fa0740 1 -- 172.21.15.8:0/420550285 wait complete. 2024-02-19T16:25:57.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 -- start start 2024-02-19T16:25:57.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 Processor -- start 2024-02-19T16:25:57.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64bdb020 con 0x556f64c73250 2024-02-19T16:25:57.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f648ae430 con 0x556f64c7c750 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6486ae00 con 0x556f64c73b70 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1258fa0740 10 client.0 _ll_register_callbacks cb 0x556f64b82cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c797d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60868/0 (socket says 172.21.15.8:60868) 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f124f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f1254964640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.451+0000 7f124ffff640 1 -- 172.21.15.8:0/2330937612 learned_addr learned my addr 172.21.15.8:0/2330937612 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124ffff640 1 -- 172.21.15.8:0/2330937612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 msgr2=0x556f64c79d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124ffff640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c79d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124ffff640 1 -- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 msgr2=0x556f64c7a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124ffff640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124ffff640 1 -- 172.21.15.8:0/2330937612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125010dbe0 con 0x556f64c73250 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1254964640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124f7fe640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c79d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f124ffff640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c797d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1240013bc0 tx=0x7f124000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214d5cafb6cb3eb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f122f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f122f7fe640 1 -- 172.21.15.8:0/2330937612 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124001e600 con 0x556f64c73250 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f64917720 con 0x556f64c73250 2024-02-19T16:25:57.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f122f7fe640 1 -- 172.21.15.8:0/2330937612 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f124001e760 con 0x556f64c73250 2024-02-19T16:25:57.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f122f7fe640 1 -- 172.21.15.8:0/2330937612 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f124001e920 con 0x556f64c73250 2024-02-19T16:25:57.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1258fa0740 10 client.16344 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:57.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556f64bd96a0 con 0x556f64c73250 2024-02-19T16:25:57.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f122f7fe640 1 -- 172.21.15.8:0/2330937612 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556f64917720 con 0x556f64c73250 2024-02-19T16:25:57.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f122f7fe640 1 -- 172.21.15.8:0/2330937612 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f124000d340 con 0x556f64c73250 2024-02-19T16:25:57.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1258fa0740 10 client.16344 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:57.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1258fa0740 10 client.16344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:25:57.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.452+0000 7f1258fa0740 20 client.16344 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:57.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.453+0000 7f1258fa0740 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64c8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.453+0000 7f1258fa0740 4 client.16344 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:57.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.453+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f64966890 con 0x556f64c8cb80 2024-02-19T16:25:57.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.453+0000 7f124f7fe640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64c8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.453+0000 7f122f7fe640 10 client.16344 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:57.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.453+0000 7f124f7fe640 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64c8efa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4ac23e6f024f000 server_cookie=8d7a1f96e379c350 in_seq=0 out_seq=0 2024-02-19T16:25:57.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f122f7fe640 1 -- 172.21.15.8:0/2330937612 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556f64966890 con 0x556f64c8cb80 2024-02-19T16:25:57.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f122f7fe640 10 client.16344 handle_command_reply: tid=0 2024-02-19T16:25:57.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 client.16344 shutdown 2024-02-19T16:25:57.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 msgr2=0x556f64c797d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c797d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1240013bc0 tx=0x7f124000e780 comp rx=0 tx=0).stop 2024-02-19T16:25:57.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 shutdown_connections 2024-02-19T16:25:57.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64c8efa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64c7a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f122f7fe640 0 client.16344 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:57.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c79d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 --2- 172.21.15.8:0/2330937612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c797d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 >> 172.21.15.8:0/2330937612 conn(0x556f64c6d980 msgr2=0x556f64c6f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 shutdown_connections 2024-02-19T16:25:57.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 1 -- 172.21.15.8:0/2330937612 wait complete. 2024-02-19T16:25:57.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.454+0000 7f1258fa0740 20 client.16344 trim_cache size 0 max 16384 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 Processor -- start 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 -- start start 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c7bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c73b70 0x556f64c7a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c7c750 0x556f64c87a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64b60130 con 0x556f64c73250 2024-02-19T16:25:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64c7ab20 con 0x556f64c73b70 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.466+0000 7f1258fa0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64c8c900 con 0x556f64c7c750 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c7bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c7bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60882/0 (socket says 172.21.15.8:60882) 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 -- 172.21.15.8:0/1665863482 learned_addr learned my addr 172.21.15.8:0/1665863482 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f1254964640 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c7c750 0x556f64c87a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124f7fe640 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c73b70 0x556f64c7a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 -- 172.21.15.8:0/1665863482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c7c750 msgr2=0x556f64c87a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c7c750 0x556f64c87a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 -- 172.21.15.8:0/1665863482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c73b70 msgr2=0x556f64c7a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c73b70 0x556f64c7a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 -- 172.21.15.8:0/1665863482 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f64b75c20 con 0x556f64c73250 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f124ffff640 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c7bf00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f124000f010 tx=0x7f12400093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cc6bafd235ae1b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f1235ffb640 1 -- 172.21.15.8:0/1665863482 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240005d30 con 0x556f64c73250 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f1235ffb640 1 -- 172.21.15.8:0/1665863482 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1240017070 con 0x556f64c73250 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.467+0000 7f1235ffb640 1 -- 172.21.15.8:0/1665863482 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400035f0 con 0x556f64c73250 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- 172.21.15.8:0/1665863482 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 msgr2=0x556f64c7bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c7bf00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f124000f010 tx=0x7f12400093f0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- 172.21.15.8:0/1665863482 shutdown_connections 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c7c750 0x556f64c87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c73b70 0x556f64c7a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1665863482 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c7bf00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- 172.21.15.8:0/1665863482 >> 172.21.15.8:0/1665863482 conn(0x556f64c6d980 msgr2=0x556f64c6f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- 172.21.15.8:0/1665863482 shutdown_connections 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- 172.21.15.8:0/1665863482 wait complete. 2024-02-19T16:25:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- start start 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 Processor -- start 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c76500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c76a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64ce7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64ce7970 con 0x556f64c73250 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64ce7ab0 con 0x556f64c7c750 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.468+0000 7f1258fa0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f64ce7bf0 con 0x556f64c73b70 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f1258fa0740 10 client.0 _ll_register_callbacks cb 0x7f12400099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c76500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c76500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60886/0 (socket says 172.21.15.8:60886) 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 -- 172.21.15.8:0/1010924229 learned_addr learned my addr 172.21.15.8:0/1010924229 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124f7fe640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c76a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f1254964640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64ce7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 -- 172.21.15.8:0/1010924229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 msgr2=0x556f64c76a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c76a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 -- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 msgr2=0x556f64ce7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64ce7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 -- 172.21.15.8:0/1010924229 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f64bd96a0 con 0x556f64c73250 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124f7fe640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c76a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f1254964640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64ce7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f124ffff640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c76500 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f124000d7c0 tx=0x7f124000d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28335b08666e567e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f122f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f122f7fe640 1 -- 172.21.15.8:0/1010924229 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124000d340 con 0x556f64c73250 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f122f7fe640 1 -- 172.21.15.8:0/1010924229 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f1240000e40 con 0x556f64c73250 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f648da7e0 con 0x556f64c73250 2024-02-19T16:25:57.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f122f7fe640 1 -- 172.21.15.8:0/1010924229 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f124000d4a0 con 0x556f64c73250 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f1258fa0740 10 client.16350 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556f64b9b8c0 con 0x556f64c73250 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.469+0000 7f122f7fe640 1 -- 172.21.15.8:0/1010924229 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556f648da7e0 con 0x556f64c73250 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f122f7fe640 1 -- 172.21.15.8:0/1010924229 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1240026040 con 0x556f64c73250 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 10 client.16350 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 10 client.16350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 20 client.16350 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64ceaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 4 client.16350 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556f64886700 con 0x556f64c8cb80 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f124f7fe640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64ceaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f122f7fe640 10 client.16350 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f124f7fe640 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64ceaf40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=205c55ad1a88ef80 server_cookie=2c18b34ee2e5f25e in_seq=0 out_seq=0 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f122f7fe640 1 -- 172.21.15.8:0/1010924229 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556f64886700 con 0x556f64c8cb80 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f122f7fe640 10 client.16350 handle_command_reply: tid=0 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 client.16350 shutdown 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 msgr2=0x556f64c76500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c76500 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f124000d7c0 tx=0x7f124000d7f0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 shutdown_connections 2024-02-19T16:25:57.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556f64c8cb80 0x556f64ceaf40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.470+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f64c7c750 0x556f64ce7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f122f7fe640 0 client.16350 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:25:57.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556f64c73b70 0x556f64c76a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f1258fa0740 1 --2- 172.21.15.8:0/1010924229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556f64c73250 0x556f64c76500 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 >> 172.21.15.8:0/1010924229 conn(0x556f64c6d980 msgr2=0x556f64c6edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 shutdown_connections 2024-02-19T16:25:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f1258fa0740 1 -- 172.21.15.8:0/1010924229 wait complete. 2024-02-19T16:25:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f1258fa0740 20 client.16350 trim_cache size 0 max 16384 2024-02-19T16:25:57.498 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:25:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f12380a2230 msgr2=0x7f12380a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f12380a2230 0x7f12380a46f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1248037040 tx=0x7f124803c000 comp rx=0 tx=0).stop 2024-02-19T16:25:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 msgr2=0x7f125019dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 0x7f125019dd40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f124400e9f0 tx=0x7f124400edf0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 shutdown_connections 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12501064b0 0x7f12501979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250100dd0 0x7f125019dd40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f12380a2230 0x7f12380a46f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 --2- 172.21.15.8:0/2815306428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100410 0x7f125019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 >> 172.21.15.8:0/2815306428 conn(0x7f1250077550 msgr2=0x7f1250069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 shutdown_connections 2024-02-19T16:25:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.471+0000 7f12563c7640 1 -- 172.21.15.8:0/2815306428 wait complete. 2024-02-19T16:25:57.501 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:25:57.501 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:25:57.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 -- 172.21.15.8:0/3660669954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca4101e60 msgr2=0x7f8ca410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3660669954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca4101e60 0x7f8ca410e320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009ca0 tx=0x7f8c9c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 -- 172.21.15.8:0/3660669954 shutdown_connections 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3660669954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca4101e60 0x7f8ca410e320 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3660669954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ca41014a0 0x7f8ca4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3660669954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca4107560 0x7f8ca4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 -- 172.21.15.8:0/3660669954 >> 172.21.15.8:0/3660669954 conn(0x7f8ca40788f0 msgr2=0x7f8ca4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 -- 172.21.15.8:0/3660669954 shutdown_connections 2024-02-19T16:25:57.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.605+0000 7f8ca9377640 1 -- 172.21.15.8:0/3660669954 wait complete. 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 Processor -- start 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 -- start start 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 0x7f8ca419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ca4101e60 0x7f8ca419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca4107560 0x7f8ca4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4197ee0 con 0x7f8ca41014a0 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4198050 con 0x7f8ca4101e60 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca9377640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4198210 con 0x7f8ca4107560 2024-02-19T16:25:57.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 0x7f8ca419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 0x7f8ca419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60906/0 (socket says 172.21.15.8:60906) 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 -- 172.21.15.8:0/3190718851 learned_addr learned my addr 172.21.15.8:0/3190718851 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ca4101e60 0x7f8ca419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca4107560 0x7f8ca4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 -- 172.21.15.8:0/3190718851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca4107560 msgr2=0x7f8ca4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca4107560 0x7f8ca4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 -- 172.21.15.8:0/3190718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ca4101e60 msgr2=0x7f8ca419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ca4101e60 0x7f8ca419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.606+0000 7f8ca2ffd640 1 -- 172.21.15.8:0/3190718851 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c009900 con 0x7f8ca41014a0 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.607+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 0x7f8ca419d680 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400e9f0 tx=0x7f8c9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.607+0000 7f8ca0ff9640 1 -- 172.21.15.8:0/3190718851 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94009700 con 0x7f8ca41014a0 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.607+0000 7f8ca0ff9640 1 -- 172.21.15.8:0/3190718851 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c9400cd80 con 0x7f8ca41014a0 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.607+0000 7f8ca0ff9640 1 -- 172.21.15.8:0/3190718851 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94018920 con 0x7f8ca41014a0 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.607+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca41984f0 con 0x7f8ca41014a0 2024-02-19T16:25:57.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.607+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca41a46b0 con 0x7f8ca41014a0 2024-02-19T16:25:57.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.609+0000 7f8ca0ff9640 1 -- 172.21.15.8:0/3190718851 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8c94018a80 con 0x7f8ca41014a0 2024-02-19T16:25:57.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.609+0000 7f8cac02e740 1 Processor -- start 2024-02-19T16:25:57.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 -- start start 2024-02-19T16:25:57.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca0ff9640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8c900a2230 0x7f8c900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 0x555e27c29630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29b70 0x555e27c32210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c34b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27be4960 con 0x555e27c29250 2024-02-19T16:25:57.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27b98310 con 0x555e27c29b70 2024-02-19T16:25:57.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8cac02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27b95840 con 0x555e27c32750 2024-02-19T16:25:57.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8c900a2230 0x7f8c900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 0x555e27c29630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 0x555e27c29630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60914/0 (socket says 172.21.15.8:60914) 2024-02-19T16:25:57.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca2ffd640 1 -- 172.21.15.8:0/1239422148 learned_addr learned my addr 172.21.15.8:0/1239422148 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29b70 0x555e27c32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c34b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca0ff9640 1 -- 172.21.15.8:0/3190718851 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8c94014070 con 0x7f8ca41014a0 2024-02-19T16:25:57.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1239422148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 msgr2=0x555e27c34b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c34b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1239422148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 msgr2=0x555e27c29630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 0x555e27c29630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1239422148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e27b53060 con 0x555e27c29b70 2024-02-19T16:25:57.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.610+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 0x555e27c29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8c900a2230 0x7f8c900a46f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c004af0 tx=0x7f8c9c005fa0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29b70 0x555e27c32210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4198b40 tx=0x7f8c8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3d4d086826fff2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8c997fa640 1 -- 172.21.15.8:0/1239422148 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c004050 con 0x555e27c29b70 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8c997fa640 1 -- 172.21.15.8:0/1239422148 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c0041b0 con 0x555e27c29b70 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8c997fa640 1 -- 172.21.15.8:0/1239422148 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c019710 con 0x555e27c29b70 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8cac02e740 1 -- 172.21.15.8:0/1239422148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29b70 msgr2=0x555e27c32210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.611+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29b70 0x555e27c32210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4198b40 tx=0x7f8c8c011810 comp rx=0 tx=0).stop 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- 172.21.15.8:0/1239422148 shutdown_connections 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c34b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29b70 0x555e27c32210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1239422148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29250 0x555e27c29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- 172.21.15.8:0/1239422148 >> 172.21.15.8:0/1239422148 conn(0x555e27c23980 msgr2=0x555e27c23d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- 172.21.15.8:0/1239422148 shutdown_connections 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- 172.21.15.8:0/1239422148 wait complete. 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- start start 2024-02-19T16:25:57.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 Processor -- start 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c30180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27b91020 con 0x555e27c29b70 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27864430 con 0x555e27c29250 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27820e00 con 0x555e27c32750 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8cac02e740 10 client.0 _ll_register_callbacks cb 0x555e27b38cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:57.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c2fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60924/0 (socket says 172.21.15.8:60924) 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1400870098 learned_addr learned my addr 172.21.15.8:0/1400870098 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.612+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c30180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 msgr2=0x555e27c30180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c30180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1400870098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 msgr2=0x555e27c2f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1400870098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c009850 con 0x555e27c29b70 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c2fc40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c000c00 tx=0x7f8c8c004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388b0a5cfe7f532e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1400870098 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c005050 con 0x555e27c29b70 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1400870098 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8c8c0051b0 con 0x555e27c29b70 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e278907e0 con 0x555e27c29b70 2024-02-19T16:25:57.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1400870098 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c8c005370 con 0x555e27c29b70 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8cac02e740 10 client.16362 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555e27b8f6a0 con 0x555e27c29b70 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.613+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1400870098 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555e278907e0 con 0x555e27c29b70 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1400870098 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8c8c021040 con 0x555e27c29b70 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8cac02e740 10 client.16362 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8cac02e740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8cac02e740 20 client.16362 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c42aa0 0x555e27c44ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8cac02e740 4 client.16362 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e278cd720 con 0x555e27c42aa0 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c42aa0 0x555e27c44ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c42aa0 0x555e27c44ec0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef4bd90cfc3c629 server_cookie=2d33ca3e917764f in_seq=0 out_seq=0 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.614+0000 7f8c8a7fc640 10 client.16362 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1400870098 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e278cd720 con 0x555e27c42aa0 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8c8a7fc640 10 client.16362 handle_command_reply: tid=0 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 client.16362 shutdown 2024-02-19T16:25:57.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 msgr2=0x555e27c2fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c2fc40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c000c00 tx=0x7f8c8c004050 comp rx=0 tx=0).stop 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 shutdown_connections 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c42aa0 0x555e27c44ec0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c30180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c2fc40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1400870098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8c8a7fc640 0 client.16362 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:57.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 >> 172.21.15.8:0/1400870098 conn(0x555e27c23980 msgr2=0x555e27c256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 shutdown_connections 2024-02-19T16:25:57.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 1 -- 172.21.15.8:0/1400870098 wait complete. 2024-02-19T16:25:57.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.615+0000 7f8cac02e740 20 client.16362 trim_cache size 0 max 16384 2024-02-19T16:25:57.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 Processor -- start 2024-02-19T16:25:57.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 -- start start 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c25800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27b16130 con 0x555e27c29b70 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27c24980 con 0x555e27c29250 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8cac02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27c231f0 con 0x555e27c32750 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60928/0 (socket says 172.21.15.8:60928) 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 -- 172.21.15.8:0/422248017 learned_addr learned my addr 172.21.15.8:0/422248017 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c25800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 -- 172.21.15.8:0/422248017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 msgr2=0x555e27c25800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c25800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 -- 172.21.15.8:0/422248017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 msgr2=0x555e27c2ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.628+0000 7f8c9bfff640 1 -- 172.21.15.8:0/422248017 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e27b2bc20 con 0x555e27c29b70 2024-02-19T16:25:57.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c252c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009380 tx=0x7f8c8c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60996d766f95d3e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8c98ff9640 1 -- 172.21.15.8:0/422248017 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c003dd0 con 0x555e27c29b70 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8c98ff9640 1 -- 172.21.15.8:0/422248017 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c019420 con 0x555e27c29b70 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8c98ff9640 1 -- 172.21.15.8:0/422248017 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c004890 con 0x555e27c29b70 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 -- 172.21.15.8:0/422248017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 msgr2=0x555e27c252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c252c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009380 tx=0x7f8c8c009310 comp rx=0 tx=0).stop 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 -- 172.21.15.8:0/422248017 shutdown_connections 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c32750 0x555e27c25800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c252c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 --2- 172.21.15.8:0/422248017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c29250 0x555e27c2ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.629+0000 7f8cac02e740 1 -- 172.21.15.8:0/422248017 >> 172.21.15.8:0/422248017 conn(0x555e27c23980 msgr2=0x555e27c24bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 -- 172.21.15.8:0/422248017 shutdown_connections 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 -- 172.21.15.8:0/422248017 wait complete. 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 -- start start 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 Processor -- start 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c29250 0x555e27c36110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c36650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c32750 0x555e27c9d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27c36dc0 con 0x555e27c29b70 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27c9d8d0 con 0x555e27c32750 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e27c9da90 con 0x555e27c29250 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8cac02e740 10 client.0 _ll_register_callbacks cb 0x7f8c94010e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c36650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c36650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60944/0 (socket says 172.21.15.8:60944) 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1416087345 learned_addr learned my addr 172.21.15.8:0/1416087345 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:57.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c29250 0x555e27c36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8ca37fe640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c32750 0x555e27c9d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c29250 msgr2=0x555e27c36110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c29250 0x555e27c36110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1416087345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c32750 msgr2=0x555e27c9d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.630+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c32750 0x555e27c9d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c9bfff640 1 -- 172.21.15.8:0/1416087345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e27b8f6a0 con 0x555e27c29b70 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c9bfff640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c36650 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c005e00 tx=0x7f8c8c0195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677fc01eca3ed6b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:57.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1416087345 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c005270 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1416087345 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8c8c000e40 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c8c0041b0 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1416087345 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c8c003dd0 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 10 client.16368 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555e27b518c0 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1416087345 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c0041b0 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1416087345 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8c8c021070 con 0x555e27c29b70 2024-02-19T16:25:57.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 10 client.16368 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 20 client.16368 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c9ee90 0x555e27ca12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 4 client.16368 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.631+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e2791c890 con 0x555e27c9ee90 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c9ee90 0x555e27ca12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8ca2ffd640 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c9ee90 0x555e27ca12b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6930bef65b378c server_cookie=dc835862743566a7 in_seq=0 out_seq=0 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8c8a7fc640 10 client.16368 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:57.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8c8a7fc640 1 -- 172.21.15.8:0/1416087345 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e2791c890 con 0x555e27c9ee90 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8c8a7fc640 10 client.16368 handle_command_reply: tid=0 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 client.16368 shutdown 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 msgr2=0x555e27c36650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c36650 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c005e00 tx=0x7f8c8c0195d0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 shutdown_connections 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e27c32750 0x555e27c9d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555e27c29b70 0x555e27c36650 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x555e27c9ee90 0x555e27ca12b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 --2- 172.21.15.8:0/1416087345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e27c29250 0x555e27c36110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8c8a7fc640 0 client.16368 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:57.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 >> 172.21.15.8:0/1416087345 conn(0x555e27c23980 msgr2=0x555e27c24b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 shutdown_connections 2024-02-19T16:25:57.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 1 -- 172.21.15.8:0/1416087345 wait complete. 2024-02-19T16:25:57.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.632+0000 7f8cac02e740 20 client.16368 trim_cache size 0 max 16384 2024-02-19T16:25:57.650 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:25:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8c900a2230 msgr2=0x7f8c900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8c900a2230 0x7f8c900a46f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c004af0 tx=0x7f8c9c005fa0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 msgr2=0x7f8ca419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 0x7f8ca419d680 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400e9f0 tx=0x7f8c9400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 shutdown_connections 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8c900a2230 0x7f8c900a46f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca4107560 0x7f8ca4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ca4101e60 0x7f8ca419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 --2- 172.21.15.8:0/3190718851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca41014a0 0x7f8ca419d680 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 >> 172.21.15.8:0/3190718851 conn(0x7f8ca40788f0 msgr2=0x7f8ca41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:57.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 shutdown_connections 2024-02-19T16:25:57.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:57.633+0000 7f8ca9377640 1 -- 172.21.15.8:0/3190718851 wait complete. 2024-02-19T16:25:57.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:57 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/316350504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:25:57.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:57 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/316350504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:25:57.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:57 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/316350504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:25:58.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:25:58.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:25:58.654 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:25:58.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:58 smithi008 ceph-mon[32573]: pgmap v95: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 -- 172.21.15.8:0/4288652327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c107580 msgr2=0x7f302c107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 --2- 172.21.15.8:0/4288652327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c107580 0x7f302c107960 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3028009900 tx=0x7f302802bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 -- 172.21.15.8:0/4288652327 shutdown_connections 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 --2- 172.21.15.8:0/4288652327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c1003b0 0x7f302c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 --2- 172.21.15.8:0/4288652327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c0ff9f0 0x7f302c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 --2- 172.21.15.8:0/4288652327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c107580 0x7f302c107960 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 -- 172.21.15.8:0/4288652327 >> 172.21.15.8:0/4288652327 conn(0x7f302c0769b0 msgr2=0x7f302c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.772+0000 7f3033a9a640 1 -- 172.21.15.8:0/4288652327 shutdown_connections 2024-02-19T16:25:58.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 -- 172.21.15.8:0/4288652327 wait complete. 2024-02-19T16:25:58.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 Processor -- start 2024-02-19T16:25:58.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 -- start start 2024-02-19T16:25:58.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 0x7f302c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c1003b0 0x7f302c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c107580 0x7f302c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c197f30 con 0x7f302c0ff9f0 2024-02-19T16:25:58.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1980a0 con 0x7f302c1003b0 2024-02-19T16:25:58.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3033a9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c198260 con 0x7f302c107580 2024-02-19T16:25:58.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 0x7f302c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 0x7f302c19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60952/0 (socket says 172.21.15.8:60952) 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3031035640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c1003b0 0x7f302c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3031836640 1 -- 172.21.15.8:0/125623570 learned_addr learned my addr 172.21.15.8:0/125623570 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.773+0000 7f3032037640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c107580 0x7f302c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3031836640 1 -- 172.21.15.8:0/125623570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c107580 msgr2=0x7f302c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3031836640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c107580 0x7f302c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3031836640 1 -- 172.21.15.8:0/125623570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c1003b0 msgr2=0x7f302c19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3031836640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c1003b0 0x7f302c19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3031836640 1 -- 172.21.15.8:0/125623570 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3028009560 con 0x7f302c0ff9f0 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3032037640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c107580 0x7f302c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3031836640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 0x7f302c19d730 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3028005940 tx=0x7f302802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3022ffd640 1 -- 172.21.15.8:0/125623570 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280042d0 con 0x7f302c0ff9f0 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c1984e0 con 0x7f302c0ff9f0 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3022ffd640 1 -- 172.21.15.8:0/125623570 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3028004430 con 0x7f302c0ff9f0 2024-02-19T16:25:58.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3022ffd640 1 -- 172.21.15.8:0/125623570 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028003c50 con 0x7f302c0ff9f0 2024-02-19T16:25:58.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.774+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c100ce0 con 0x7f302c0ff9f0 2024-02-19T16:25:58.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:58 smithi191 ceph-mon[37203]: pgmap v95: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:25:58.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.776+0000 7f3022ffd640 1 -- 172.21.15.8:0/125623570 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f30280054f0 con 0x7f302c0ff9f0 2024-02-19T16:25:58.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.777+0000 7f3036673740 1 Processor -- start 2024-02-19T16:25:58.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.777+0000 7f3036673740 1 -- start start 2024-02-19T16:25:58.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3036673740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3afeb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3036673740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 0x560f3aff42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3036673740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3aff6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3036673740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3af59f90 con 0x560f3afebd30 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3036673740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3af574c0 con 0x560f3afeb380 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3036673740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3af52ca0 con 0x560f3aff48b0 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3022ffd640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f30180a2230 0x7f30180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 0x560f3aff42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031035640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3aff6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3032037640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3afeb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 0x560f3aff42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60960/0 (socket says 172.21.15.8:60960) 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031035640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3aff6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:49760/0 (socket says 172.21.15.8:49760) 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 -- 172.21.15.8:0/4113580313 learned_addr learned my addr 172.21.15.8:0/4113580313 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3032037640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f30180a2230 0x7f30180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3022ffd640 1 -- 172.21.15.8:0/125623570 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f30280f03f0 con 0x7f302c0ff9f0 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 -- 172.21.15.8:0/4113580313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 msgr2=0x560f3aff6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3aff6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 -- 172.21.15.8:0/4113580313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 msgr2=0x560f3afeb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3afeb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 -- 172.21.15.8:0/4113580313 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f3af150b0 con 0x560f3afebd30 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031035640 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3aff6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3032037640 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3afeb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.778+0000 7f3031836640 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 0x560f3aff42e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3028005ec0 tx=0x7f30280036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ad60e0c6b2838c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.779+0000 7f3032037640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f30180a2230 0x7f30180a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30240099e0 tx=0x7f3024009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.779+0000 7f3017fff640 1 -- 172.21.15.8:0/4113580313 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280bd240 con 0x560f3afebd30 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.779+0000 7f3017fff640 1 -- 172.21.15.8:0/4113580313 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30280bcb40 con 0x560f3afebd30 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.779+0000 7f3017fff640 1 -- 172.21.15.8:0/4113580313 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280eda10 con 0x560f3afebd30 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.779+0000 7f3036673740 1 -- 172.21.15.8:0/4113580313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 msgr2=0x560f3aff42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.779+0000 7f3036673740 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 0x560f3aff42e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3028005ec0 tx=0x7f30280036f0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 -- 172.21.15.8:0/4113580313 shutdown_connections 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3aff6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afebd30 0x560f3aff42e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 --2- 172.21.15.8:0/4113580313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3afeb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 -- 172.21.15.8:0/4113580313 >> 172.21.15.8:0/4113580313 conn(0x560f3afe5840 msgr2=0x560f3afe5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 -- 172.21.15.8:0/4113580313 shutdown_connections 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 -- 172.21.15.8:0/4113580313 wait complete. 2024-02-19T16:25:58.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 -- start start 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 Processor -- start 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3aff1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.780+0000 7f3036673740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3036673740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 0x560f3aff22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3036673740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3aef7f40 con 0x560f3aff48b0 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3036673740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3ac264b0 con 0x560f3afeb380 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3036673740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3aed8670 con 0x560f3afebd30 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3036673740 10 client.0 _ll_register_callbacks cb 0x560f3af45f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:58.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 0x560f3aff22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 0x560f3aff22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60974/0 (socket says 172.21.15.8:60974) 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3031035640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 -- 172.21.15.8:0/3104039014 learned_addr learned my addr 172.21.15.8:0/3104039014 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3031836640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3aff1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 -- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 msgr2=0x560f3aff1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 -- 172.21.15.8:0/3104039014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 msgr2=0x560f3aff1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3aff1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 -- 172.21.15.8:0/3104039014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30280bde70 con 0x560f3aff48b0 2024-02-19T16:25:58.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3031836640 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3aff1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3032037640 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 0x560f3aff22e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3024005ca0 tx=0x7f3024004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccb08a6781e7549 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f30157fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:58.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f30157fa640 1 -- 172.21.15.8:0/3104039014 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024026070 con 0x560f3aff48b0 2024-02-19T16:25:58.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f30157fa640 1 -- 172.21.15.8:0/3104039014 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f302401e410 con 0x560f3aff48b0 2024-02-19T16:25:58.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.781+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f3af2cf40 con 0x560f3aff48b0 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f30157fa640 1 -- 172.21.15.8:0/3104039014 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3024015070 con 0x560f3aff48b0 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 10 client.16380 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560f3af51320 con 0x560f3aff48b0 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f30157fa640 1 -- 172.21.15.8:0/3104039014 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560f3af2cf40 con 0x560f3aff48b0 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f30157fa640 1 -- 172.21.15.8:0/3104039014 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3024020930 con 0x560f3aff48b0 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 10 client.16380 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:58.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 20 client.16380 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b004d00 0x560f3b007120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 4 client.16380 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f3af49310 con 0x560f3b004d00 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.782+0000 7f3031836640 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b004d00 0x560f3b007120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.783+0000 7f30157fa640 10 client.16380 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.783+0000 7f3031836640 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b004d00 0x560f3b007120 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b2466fb59c39 server_cookie=5356e99690a8e516 in_seq=0 out_seq=0 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f30157fa640 1 -- 172.21.15.8:0/3104039014 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560f3af49310 con 0x560f3b004d00 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f30157fa640 10 client.16380 handle_command_reply: tid=0 2024-02-19T16:25:58.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 client.16380 shutdown 2024-02-19T16:25:58.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 msgr2=0x560f3aff22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 0x560f3aff22e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3024005ca0 tx=0x7f3024004a40 comp rx=0 tx=0).stop 2024-02-19T16:25:58.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 shutdown_connections 2024-02-19T16:25:58.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3aff48b0 0x560f3aff22e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b004d00 0x560f3b007120 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 --2- 172.21.15.8:0/3104039014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afeb380 0x560f3aff1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f30157fa640 0 client.16380 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 >> 172.21.15.8:0/3104039014 conn(0x560f3afe5840 msgr2=0x560f3afe77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 shutdown_connections 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 1 -- 172.21.15.8:0/3104039014 wait complete. 2024-02-19T16:25:58.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.784+0000 7f3036673740 20 client.16380 trim_cache size 0 max 16384 2024-02-19T16:25:58.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.800+0000 7f3036673740 1 Processor -- start 2024-02-19T16:25:58.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.800+0000 7f3036673740 1 -- start start 2024-02-19T16:25:58.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.800+0000 7f3036673740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3036673740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afebd30 0x560f3aff2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3036673740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3afff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3036673740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3afed2f0 con 0x560f3afeb380 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3036673740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3b0033d0 con 0x560f3afebd30 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3036673740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3afefdb0 con 0x560f3aff48b0 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60986/0 (socket says 172.21.15.8:60986) 2024-02-19T16:25:58.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 -- 172.21.15.8:0/2030109552 learned_addr learned my addr 172.21.15.8:0/2030109552 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3032037640 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3afff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031035640 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afebd30 0x560f3aff2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 -- 172.21.15.8:0/2030109552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 msgr2=0x560f3afff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3afff920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 -- 172.21.15.8:0/2030109552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afebd30 msgr2=0x560f3aff2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afebd30 0x560f3aff2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 -- 172.21.15.8:0/2030109552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f3aeedc00 con 0x560f3afeb380 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031035640 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afebd30 0x560f3aff2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:25:58.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f3031836640 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff3f90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f30280b8fa0 tx=0x7f30280b8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b4db9a32a8969d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f30177fe640 1 -- 172.21.15.8:0/2030109552 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280d9070 con 0x560f3afeb380 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f30177fe640 1 -- 172.21.15.8:0/2030109552 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30280bcb40 con 0x560f3afeb380 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.801+0000 7f30177fe640 1 -- 172.21.15.8:0/2030109552 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280bae40 con 0x560f3afeb380 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 -- 172.21.15.8:0/2030109552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 msgr2=0x560f3aff3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff3f90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f30280b8fa0 tx=0x7f30280b8f10 comp rx=0 tx=0).stop 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 -- 172.21.15.8:0/2030109552 shutdown_connections 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3aff48b0 0x560f3afff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3afebd30 0x560f3aff2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 --2- 172.21.15.8:0/2030109552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff3f90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 -- 172.21.15.8:0/2030109552 >> 172.21.15.8:0/2030109552 conn(0x560f3afe5840 msgr2=0x560f3afe48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 -- 172.21.15.8:0/2030109552 shutdown_connections 2024-02-19T16:25:58.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 -- 172.21.15.8:0/2030109552 wait complete. 2024-02-19T16:25:58.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 -- start start 2024-02-19T16:25:58.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.802+0000 7f3036673740 1 Processor -- start 2024-02-19T16:25:58.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3aff48b0 0x560f3b05f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3aff8cb0 con 0x560f3afeb380 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3b05fb00 con 0x560f3aff48b0 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3b05fcc0 con 0x560f3afebd30 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3036673740 10 client.0 _ll_register_callbacks cb 0x7f3024003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031836640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60998/0 (socket says 172.21.15.8:60998) 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031836640 1 -- 172.21.15.8:0/1184766839 learned_addr learned my addr 172.21.15.8:0/1184766839 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3032037640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3aff48b0 0x560f3b05f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 -- 172.21.15.8:0/1184766839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3aff48b0 msgr2=0x560f3b05f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3aff48b0 0x560f3b05f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 -- 172.21.15.8:0/1184766839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 msgr2=0x560f3aff7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 -- 172.21.15.8:0/1184766839 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f3af150b0 con 0x560f3afebd30 2024-02-19T16:25:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031836640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f3031035640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff8510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x560f3b007390 tx=0x7f301c010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eae7e8389303a69 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f30157fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f30157fa640 1 -- 172.21.15.8:0/1184766839 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c017040 con 0x560f3afebd30 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.803+0000 7f30157fa640 1 -- 172.21.15.8:0/1184766839 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f301c004460 con 0x560f3afebd30 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f3af78900 con 0x560f3afebd30 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f30157fa640 1 -- 172.21.15.8:0/1184766839 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f301c00f890 con 0x560f3afebd30 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 10 client.26254 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560f3af13910 con 0x560f3afebd30 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f30157fa640 1 -- 172.21.15.8:0/1184766839 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560f3af78900 con 0x560f3afebd30 2024-02-19T16:25:58.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f30157fa640 1 -- 172.21.15.8:0/1184766839 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f301c01c3d0 con 0x560f3afebd30 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 10 client.26254 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 20 client.26254 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b061610 0x560f3b063a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 4 client.26254 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560f3ac52860 con 0x560f3b061610 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.804+0000 7f3031836640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b061610 0x560f3b063a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.805+0000 7f3031836640 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b061610 0x560f3b063a30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f089f54c207a397 server_cookie=97b5adc5d272f465 in_seq=0 out_seq=0 2024-02-19T16:25:58.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.805+0000 7f30157fa640 10 client.26254 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f30157fa640 1 -- 172.21.15.8:0/1184766839 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560f3ac52860 con 0x560f3b061610 2024-02-19T16:25:58.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f30157fa640 10 client.26254 handle_command_reply: tid=0 2024-02-19T16:25:58.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 client.26254 shutdown 2024-02-19T16:25:58.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 msgr2=0x560f3aff8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff8510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x560f3b007390 tx=0x7f301c010c80 comp rx=0 tx=0).stop 2024-02-19T16:25:58.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 shutdown_connections 2024-02-19T16:25:58.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3aff48b0 0x560f3b05f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x560f3b061610 0x560f3b063a30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f3afebd30 0x560f3aff8510 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 --2- 172.21.15.8:0/1184766839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f3afeb380 0x560f3aff7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f30157fa640 0 client.26254 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 >> 172.21.15.8:0/1184766839 conn(0x560f3afe5840 msgr2=0x560f3afe6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 shutdown_connections 2024-02-19T16:25:58.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 1 -- 172.21.15.8:0/1184766839 wait complete. 2024-02-19T16:25:58.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.806+0000 7f3036673740 20 client.26254 trim_cache size 0 max 16384 2024-02-19T16:25:58.821 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:25:58.821 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:25:58.821 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "8d4765e3-c198-458e-b7aa-299d515a1083", 2024-02-19T16:25:58.821 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:25:58.821 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f30180a2230 msgr2=0x7f30180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f30180a2230 0x7f30180a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30240099e0 tx=0x7f3024009500 comp rx=0 tx=0).stop 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 msgr2=0x7f302c19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 0x7f302c19d730 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3028005940 tx=0x7f302802c670 comp rx=0 tx=0).stop 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 shutdown_connections 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f30180a2230 0x7f30180a46f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f302c107580 0x7f302c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c1003b0 0x7f302c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 --2- 172.21.15.8:0/125623570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f302c0ff9f0 0x7f302c19d730 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 >> 172.21.15.8:0/125623570 conn(0x7f302c0769b0 msgr2=0x7f302c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 shutdown_connections 2024-02-19T16:25:58.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.807+0000 7f3033a9a640 1 -- 172.21.15.8:0/125623570 wait complete. 2024-02-19T16:25:58.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:25:58.825 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:25:58.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:58 smithi156 ceph-mon[36502]: pgmap v95: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:25:58.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 -- 172.21.15.8:0/4204033029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c100fa0 msgr2=0x7f8c0c104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/4204033029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c100fa0 0x7f8c0c104fe0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009930 tx=0x7f8c0002bc30 comp rx=0 tx=0).stop 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 -- 172.21.15.8:0/4204033029 shutdown_connections 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/4204033029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/4204033029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c100fa0 0x7f8c0c104fe0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/4204033029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c0c1005f0 0x7f8c0c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 -- 172.21.15.8:0/4204033029 >> 172.21.15.8:0/4204033029 conn(0x7f8c0c076770 msgr2=0x7f8c0c076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 -- 172.21.15.8:0/4204033029 shutdown_connections 2024-02-19T16:25:58.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.931+0000 7f8c10d3e640 1 -- 172.21.15.8:0/4204033029 wait complete. 2024-02-19T16:25:58.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 Processor -- start 2024-02-19T16:25:58.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 -- start start 2024-02-19T16:25:58.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c0c1005f0 0x7f8c0c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c0c197950 0x7f8c0c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c198340 con 0x7f8c0c1056c0 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c1984b0 con 0x7f8c0c197950 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c10d3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c198670 con 0x7f8c0c1005f0 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c19dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32784/0 (socket says 172.21.15.8:32784) 2024-02-19T16:25:58.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 -- 172.21.15.8:0/754333467 learned_addr learned my addr 172.21.15.8:0/754333467 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c0c1005f0 0x7f8c0c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c0c197950 0x7f8c0c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 -- 172.21.15.8:0/754333467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c0c1005f0 msgr2=0x7f8c0c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c0c1005f0 0x7f8c0c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 -- 172.21.15.8:0/754333467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c0c197950 msgr2=0x7f8c0c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.932+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c0c197950 0x7f8c0c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8c09d9b640 1 -- 172.21.15.8:0/754333467 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00009560 con 0x7f8c0c1056c0 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c19dce0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8c0002c170 tx=0x7f8c00004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8bfb7fe640 1 -- 172.21.15.8:0/754333467 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0002c5d0 con 0x7f8c0c1056c0 2024-02-19T16:25:58.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c1a4660 con 0x7f8c0c1056c0 2024-02-19T16:25:58.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8bfb7fe640 1 -- 172.21.15.8:0/754333467 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c000054e0 con 0x7f8c0c1056c0 2024-02-19T16:25:58.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8bfb7fe640 1 -- 172.21.15.8:0/754333467 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00003ab0 con 0x7f8c0c1056c0 2024-02-19T16:25:58.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.933+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c1a4b60 con 0x7f8c0c1056c0 2024-02-19T16:25:58.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.935+0000 7f8bfb7fe640 1 -- 172.21.15.8:0/754333467 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8c00003c10 con 0x7f8c0c1056c0 2024-02-19T16:25:58.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 Processor -- start 2024-02-19T16:25:58.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8bfb7fe640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8bf40a2230 0x7f8bf40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8bfb7fe640 1 -- 172.21.15.8:0/754333467 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8c000f0840 con 0x7f8c0c1056c0 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8bf40a2230 0x7f8bf40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 -- start start 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d1642280 con 0x5583d16e0bc0 2024-02-19T16:25:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d163f7b0 con 0x5583d16d2f70 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c139f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d163af90 con 0x5583d16d3890 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.936+0000 7f8c09d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c09d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:49818/0 (socket says 172.21.15.8:49818) 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c09d9b640 1 -- 172.21.15.8:0/17881690 learned_addr learned my addr 172.21.15.8:0/17881690 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8bf40a2230 0x7f8bf40a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005fd0 tx=0x7f8c04009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 -- 172.21.15.8:0/17881690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 msgr2=0x5583d16e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 -- 172.21.15.8:0/17881690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 msgr2=0x5583d16e2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 -- 172.21.15.8:0/17881690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d15f9570 con 0x5583d16d2f70 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16d3350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009bf0 tx=0x7f8bfc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44215d6b44740e69 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8bf8ff9640 1 -- 172.21.15.8:0/17881690 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc004050 con 0x5583d16d2f70 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8bf8ff9640 1 -- 172.21.15.8:0/17881690 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bfc005500 con 0x5583d16d2f70 2024-02-19T16:25:58.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.937+0000 7f8bf8ff9640 1 -- 172.21.15.8:0/17881690 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc004d90 con 0x5583d16d2f70 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 -- 172.21.15.8:0/17881690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 msgr2=0x5583d16d3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16d3350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009bf0 tx=0x7f8bfc011810 comp rx=0 tx=0).stop 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 -- 172.21.15.8:0/17881690 shutdown_connections 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 --2- 172.21.15.8:0/17881690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16d3350 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 -- 172.21.15.8:0/17881690 >> 172.21.15.8:0/17881690 conn(0x5583d16cd990 msgr2=0x5583d16cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 -- 172.21.15.8:0/17881690 shutdown_connections 2024-02-19T16:25:58.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.938+0000 7f8c139f2740 1 -- 172.21.15.8:0/17881690 wait complete. 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 -- start start 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 Processor -- start 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16dd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d130e430 con 0x5583d16e0bc0 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d12cae00 con 0x5583d16d2f70 2024-02-19T16:25:58.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d15bffd0 con 0x5583d16d3890 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c139f2740 10 client.0 _ll_register_callbacks cb 0x5583d1615fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c09d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16ddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32792/0 (socket says 172.21.15.8:32792) 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 -- 172.21.15.8:0/715957745 learned_addr learned my addr 172.21.15.8:0/715957745 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 -- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 msgr2=0x5583d16dd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16dd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 -- 172.21.15.8:0/715957745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 msgr2=0x5583d16e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0ad9d640 1 -- 172.21.15.8:0/715957745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc009850 con 0x5583d16e0bc0 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.939+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16ddf20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8c040043d0 tx=0x7f8c04004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302d257107365a61 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/715957745 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400eb30 con 0x5583d16e0bc0 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/715957745 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8c0400ec90 con 0x5583d16e0bc0 2024-02-19T16:25:58.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583d133a7e0 con 0x5583d16e0bc0 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/715957745 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0400ee50 con 0x5583d16e0bc0 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c139f2740 10 client.16398 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5583d1639610 con 0x5583d16e0bc0 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/715957745 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583d133a7e0 con 0x5583d16e0bc0 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/715957745 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8c0401f5c0 con 0x5583d16e0bc0 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c139f2740 10 client.16398 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c139f2740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:58.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.940+0000 7f8c139f2740 20 client.16398 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:58.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.941+0000 7f8c139f2740 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d16f0e80 0x5583d16f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.941+0000 7f8c139f2740 4 client.16398 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:25:58.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.941+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583d1377720 con 0x5583d16f0e80 2024-02-19T16:25:58.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.941+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d16f0e80 0x5583d16f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.941+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d16f0e80 0x5583d16f32a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf18c195083a98fd server_cookie=e77026ba4a401f4c in_seq=0 out_seq=0 2024-02-19T16:25:58.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.941+0000 7f8bf1ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/715957745 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5583d1377720 con 0x5583d16f0e80 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8bf1ffb640 10 client.16398 handle_command_reply: tid=0 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 client.16398 shutdown 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 msgr2=0x5583d16ddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16ddf20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8c040043d0 tx=0x7f8c04004ab0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 shutdown_connections 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16ddf20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d16f0e80 0x5583d16f32a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16dd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 --2- 172.21.15.8:0/715957745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8bf1ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 >> 172.21.15.8:0/715957745 conn(0x5583d16cd990 msgr2=0x5583d16cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 shutdown_connections 2024-02-19T16:25:58.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 1 -- 172.21.15.8:0/715957745 wait complete. 2024-02-19T16:25:58.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.943+0000 7f8c139f2740 20 client.16398 trim_cache size 0 max 16384 2024-02-19T16:25:58.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.961+0000 7f8c139f2740 1 Processor -- start 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.961+0000 7f8c139f2740 1 -- start start 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 0x5583d16e0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d3890 0x5583d16e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16e0bc0 0x5583d16ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c139f2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d16ef580 con 0x5583d16d2f70 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c139f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d16c92b0 con 0x5583d16d3890 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c139f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d16f3900 con 0x5583d16e0bc0 2024-02-19T16:25:58.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 0x5583d16e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 0x5583d16e0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32804/0 (socket says 172.21.15.8:32804) 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 -- 172.21.15.8:0/691357619 learned_addr learned my addr 172.21.15.8:0/691357619 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d3890 0x5583d16e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16e0bc0 0x5583d16ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 -- 172.21.15.8:0/691357619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16e0bc0 msgr2=0x5583d16ebf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16e0bc0 0x5583d16ebf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 -- 172.21.15.8:0/691357619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d3890 msgr2=0x5583d16e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d3890 0x5583d16e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 -- 172.21.15.8:0/691357619 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d15d5ac0 con 0x5583d16d2f70 2024-02-19T16:25:58.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 0x5583d16e0040 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f28e5cb8f6a42ff server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.962+0000 7f8bf3fff640 1 -- 172.21.15.8:0/691357619 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc032030 con 0x5583d16d2f70 2024-02-19T16:25:58.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8bf3fff640 1 -- 172.21.15.8:0/691357619 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bfc011d40 con 0x5583d16d2f70 2024-02-19T16:25:58.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8bf3fff640 1 -- 172.21.15.8:0/691357619 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc02a040 con 0x5583d16d2f70 2024-02-19T16:25:58.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 -- 172.21.15.8:0/691357619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 msgr2=0x5583d16e0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 0x5583d16e0040 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc004a30 comp rx=0 tx=0).stop 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 -- 172.21.15.8:0/691357619 shutdown_connections 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16e0bc0 0x5583d16ebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d3890 0x5583d16e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 --2- 172.21.15.8:0/691357619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16d2f70 0x5583d16e0040 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 -- 172.21.15.8:0/691357619 >> 172.21.15.8:0/691357619 conn(0x5583d16cd990 msgr2=0x5583d16cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 -- 172.21.15.8:0/691357619 shutdown_connections 2024-02-19T16:25:58.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.963+0000 7f8c139f2740 1 -- 172.21.15.8:0/691357619 wait complete. 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 -- start start 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 Processor -- start 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d16e25b0 con 0x5583d16e0bc0 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d16e6ff0 con 0x5583d16d2f70 2024-02-19T16:25:58.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d16e71b0 con 0x5583d16d3890 2024-02-19T16:25:58.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c139f2740 10 client.0 _ll_register_callbacks cb 0x7f8c04004050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:25:58.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48520/0 (socket says 172.21.15.8:48520) 2024-02-19T16:25:58.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c0a59c640 1 -- 172.21.15.8:0/3508165454 learned_addr learned my addr 172.21.15.8:0/3508165454 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:25:58.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.964+0000 7f8c0ad9d640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c0a59c640 1 -- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 msgr2=0x5583d16e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c0a59c640 1 -- 172.21.15.8:0/3508165454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 msgr2=0x5583d16e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c0a59c640 1 -- 172.21.15.8:0/3508165454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583d15f9570 con 0x5583d16d2f70 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c0a59c640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e17c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc011850 tx=0x7f8bfc011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c2258b2d56fee3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/3508165454 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc02a040 con 0x5583d16d2f70 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583d13c6890 con 0x5583d16d2f70 2024-02-19T16:25:58.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/3508165454 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8bfc033070 con 0x5583d16d2f70 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/3508165454 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc02e2d0 con 0x5583d16d2f70 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c139f2740 10 client.26177 fetch_fsmap learned FSMap version 27 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.965+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5583d15fb6a0 con 0x5583d16d2f70 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/3508165454 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583d13c6890 con 0x5583d16d2f70 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/3508165454 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8bfc03e020 con 0x5583d16d2f70 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c139f2740 10 client.26177 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c139f2740 10 client.26177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:25:58.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c139f2740 20 client.26177 populate_metadata read hostname 'smithi008' 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c139f2740 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d174d370 0x5583d174f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c139f2740 4 client.26177 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5583d13294e0 con 0x5583d174d370 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.966+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d174d370 0x5583d174f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8bf1ffb640 10 client.26177 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c09d9b640 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d174d370 0x5583d174f790 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3af90db67f3b4 server_cookie=39de3a41720d1a17 in_seq=0 out_seq=0 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8bf1ffb640 1 -- 172.21.15.8:0/3508165454 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5583d13294e0 con 0x5583d174d370 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8bf1ffb640 10 client.26177 handle_command_reply: tid=0 2024-02-19T16:25:58.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 client.26177 shutdown 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 msgr2=0x5583d16e17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e17c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc011850 tx=0x7f8bfc011d10 comp rx=0 tx=0).stop 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 shutdown_connections 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5583d16e0bc0 0x5583d16e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5583d174d370 0x5583d174f790 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583d16d3890 0x5583d16e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 --2- 172.21.15.8:0/3508165454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5583d16d2f70 0x5583d16e17c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 >> 172.21.15.8:0/3508165454 conn(0x5583d16cd990 msgr2=0x5583d16ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8bf1ffb640 0 client.26177 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:25:58.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 shutdown_connections 2024-02-19T16:25:58.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.967+0000 7f8c139f2740 1 -- 172.21.15.8:0/3508165454 wait complete. 2024-02-19T16:25:58.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c139f2740 20 client.26177 trim_cache size 0 max 16384 2024-02-19T16:25:58.979 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:25:58.979 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2732 inodes in the stack)", 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: "8d4765e3-c198-458e-b7aa-299d515a1083": { 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "8d4765e3-c198-458e-b7aa-299d515a1083", 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:25:58.980 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8bf40a2230 msgr2=0x7f8bf40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8bf40a2230 0x7f8bf40a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005fd0 tx=0x7f8c04009500 comp rx=0 tx=0).stop 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 msgr2=0x7f8c0c19dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c19dce0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8c0002c170 tx=0x7f8c00004970 comp rx=0 tx=0).stop 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 shutdown_connections 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c0c197950 0x7f8c0c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c0c1056c0 0x7f8c0c19dce0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8bf40a2230 0x7f8bf40a46f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 --2- 172.21.15.8:0/754333467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c0c1005f0 0x7f8c0c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:25:58.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 >> 172.21.15.8:0/754333467 conn(0x7f8c0c076770 msgr2=0x7f8c0c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:25:58.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 shutdown_connections 2024-02-19T16:25:58.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:25:58.968+0000 7f8c10d3e640 1 -- 172.21.15.8:0/754333467 wait complete. 2024-02-19T16:25:58.984 INFO:tasks.cephfs.filesystem:scrub status for tag:8d4765e3-c198-458e-b7aa-299d515a1083 - {'path': '/', 'tag': '8d4765e3-c198-458e-b7aa-299d515a1083', 'options': 'recursive,force'} 2024-02-19T16:25:59.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:59 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:25:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:59 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:59 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:59 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:25:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:25:59 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:59 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:25:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:59 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:59 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:59 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:25:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:25:59 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:59.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:59 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:25:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:59 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:25:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:59 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:59 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:25:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:25:59 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: pgmap v96: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:26:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:26:00.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:26:00.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:26:00.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:26:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: pgmap v96: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:26:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:26:00.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:26:00.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: pgmap v96: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:00.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:26:00.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:26:00.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:26:00.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:26:02.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:02 smithi008 ceph-mon[32573]: pgmap v97: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:26:02.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:02 smithi191 ceph-mon[37203]: pgmap v97: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:26:02.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:02 smithi156 ceph-mon[36502]: pgmap v97: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:26:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:04 smithi008 ceph-mon[32573]: pgmap v98: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:26:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:04 smithi191 ceph-mon[37203]: pgmap v98: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:26:04.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:04 smithi156 ceph-mon[36502]: pgmap v98: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T16:26:05.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:05 smithi156 ceph-mon[36502]: pgmap v99: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:26:06.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:05 smithi008 ceph-mon[32573]: pgmap v99: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:26:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:05 smithi191 ceph-mon[37203]: pgmap v99: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:26:08.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:07 smithi008 ceph-mon[32573]: pgmap v100: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:26:08.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:07 smithi191 ceph-mon[37203]: pgmap v100: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:26:08.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:07 smithi156 ceph-mon[36502]: pgmap v100: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:26:10.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:09 smithi008 ceph-mon[32573]: pgmap v101: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 73 KiB/s wr, 23 op/s 2024-02-19T16:26:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:09 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:26:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:09 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:26:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:09 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:26:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:09 smithi191 ceph-mon[37203]: pgmap v101: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 73 KiB/s wr, 23 op/s 2024-02-19T16:26:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:09 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:26:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:09 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:26:10.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:09 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:26:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:09 smithi156 ceph-mon[36502]: pgmap v101: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 73 KiB/s wr, 23 op/s 2024-02-19T16:26:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:09 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:26:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:09 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:26:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:09 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:26:12.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:11 smithi008 ceph-mon[32573]: pgmap v102: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-02-19T16:26:12.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:11 smithi191 ceph-mon[37203]: pgmap v102: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-02-19T16:26:12.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:11 smithi156 ceph-mon[36502]: pgmap v102: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-02-19T16:26:14.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:13 smithi008 ceph-mon[32573]: pgmap v103: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-02-19T16:26:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:13 smithi191 ceph-mon[37203]: pgmap v103: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-02-19T16:26:14.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:13 smithi156 ceph-mon[36502]: pgmap v103: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-02-19T16:26:16.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:15 smithi008 ceph-mon[32573]: pgmap v104: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 79 KiB/s wr, 24 op/s 2024-02-19T16:26:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:15 smithi191 ceph-mon[37203]: pgmap v104: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 79 KiB/s wr, 24 op/s 2024-02-19T16:26:16.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:15 smithi156 ceph-mon[36502]: pgmap v104: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 79 KiB/s wr, 24 op/s 2024-02-19T16:26:18.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:17 smithi008 ceph-mon[32573]: pgmap v105: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 6 op/s 2024-02-19T16:26:18.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:17 smithi191 ceph-mon[37203]: pgmap v105: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 6 op/s 2024-02-19T16:26:18.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:17 smithi156 ceph-mon[36502]: pgmap v105: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 6 op/s 2024-02-19T16:26:20.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:19 smithi008 ceph-mon[32573]: pgmap v106: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2024-02-19T16:26:20.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:19 smithi191 ceph-mon[37203]: pgmap v106: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2024-02-19T16:26:20.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:19 smithi156 ceph-mon[36502]: pgmap v106: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2024-02-19T16:26:22.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:21 smithi008 ceph-mon[32573]: pgmap v107: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 6 op/s 2024-02-19T16:26:22.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:21 smithi191 ceph-mon[37203]: pgmap v107: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 6 op/s 2024-02-19T16:26:22.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:21 smithi156 ceph-mon[36502]: pgmap v107: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 6 op/s 2024-02-19T16:26:24.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:23 smithi008 ceph-mon[32573]: pgmap v108: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 6 op/s 2024-02-19T16:26:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:23 smithi191 ceph-mon[37203]: pgmap v108: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 6 op/s 2024-02-19T16:26:24.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:23 smithi156 ceph-mon[36502]: pgmap v108: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 6 op/s 2024-02-19T16:26:26.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:25 smithi008 ceph-mon[32573]: pgmap v109: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-02-19T16:26:26.026 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:25 smithi191 ceph-mon[37203]: pgmap v109: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-02-19T16:26:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:25 smithi156 ceph-mon[36502]: pgmap v109: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-02-19T16:26:28.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:27 smithi008 ceph-mon[32573]: pgmap v110: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-02-19T16:26:28.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:27 smithi191 ceph-mon[37203]: pgmap v110: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-02-19T16:26:28.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:27 smithi156 ceph-mon[36502]: pgmap v110: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-02-19T16:26:28.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:26:28.985 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:26:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 -- 172.21.15.8:0/15833070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd006a4e0 msgr2=0x7f3dd010e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/15833070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd006a4e0 0x7f3dd010e310 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009cd0 tx=0x7f3dc802c180 comp rx=0 tx=0).stop 2024-02-19T16:26:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 -- 172.21.15.8:0/15833070 shutdown_connections 2024-02-19T16:26:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/15833070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd006a4e0 0x7f3dd010e310 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/15833070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 0x7f3dd0069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/15833070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dd0107600 0x7f3dd01079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 -- 172.21.15.8:0/15833070 >> 172.21.15.8:0/15833070 conn(0x7f3dd0077190 msgr2=0x7f3dd00775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 -- 172.21.15.8:0/15833070 shutdown_connections 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.101+0000 7f3dd76f4640 1 -- 172.21.15.8:0/15833070 wait complete. 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 Processor -- start 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 -- start start 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 0x7f3dd019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dd006a4e0 0x7f3dd019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 0x7f3dd0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd019a0f0 con 0x7f3dd0107600 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd019a260 con 0x7f3dd006a4e0 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd76f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd019a420 con 0x7f3dd0069b20 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd5c91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 0x7f3dd0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd5490640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 0x7f3dd019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd5c91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 0x7f3dd0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46842/0 (socket says 172.21.15.8:46842) 2024-02-19T16:26:29.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd5c91640 1 -- 172.21.15.8:0/864648422 learned_addr learned my addr 172.21.15.8:0/864648422 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.102+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dd006a4e0 0x7f3dd019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5c91640 1 -- 172.21.15.8:0/864648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 msgr2=0x7f3dd019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 0x7f3dd019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5c91640 1 -- 172.21.15.8:0/864648422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dd006a4e0 msgr2=0x7f3dd019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dd006a4e0 0x7f3dd019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5c91640 1 -- 172.21.15.8:0/864648422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8009900 con 0x7f3dd0107600 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5490640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 0x7f3dd019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 0x7f3dd0199aa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009ca0 tx=0x7f3dc80346c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dc67fc640 1 -- 172.21.15.8:0/864648422 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003d80 con 0x7f3dd0107600 2024-02-19T16:26:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dc67fc640 1 -- 172.21.15.8:0/864648422 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dc80348b0 con 0x7f3dd0107600 2024-02-19T16:26:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd019a6a0 con 0x7f3dd0107600 2024-02-19T16:26:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dc67fc640 1 -- 172.21.15.8:0/864648422 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004790 con 0x7f3dd0107600 2024-02-19T16:26:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.103+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd01a6990 con 0x7f3dd0107600 2024-02-19T16:26:29.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.105+0000 7f3dc67fc640 1 -- 172.21.15.8:0/864648422 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f3dc800a040 con 0x7f3dd0107600 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 Processor -- start 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 -- start start 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dc67fc640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3dbc0a2230 0x7f3dbc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 0x563c91392690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c9139b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c9139dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c91301280 con 0x563c913922b0 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c912fe7b0 con 0x563c91392bd0 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c912f9f90 con 0x563c9139b7b0 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 0x563c91392690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3dbc0a2230 0x7f3dbc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dc67fc640 1 -- 172.21.15.8:0/864648422 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3dc80f0ba0 con 0x7f3dd0107600 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 0x563c91392690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46854/0 (socket says 172.21.15.8:46854) 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 -- 172.21.15.8:0/2285941030 learned_addr learned my addr 172.21.15.8:0/2285941030 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c9139dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c9139b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 -- 172.21.15.8:0/2285941030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 msgr2=0x563c9139dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c9139dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 -- 172.21.15.8:0/2285941030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 msgr2=0x563c9139b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c9139b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.106+0000 7f3dd5490640 1 -- 172.21.15.8:0/2285941030 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c912b8570 con 0x563c913922b0 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c9139dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dd5490640 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 0x563c91392690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0114f0 tx=0x7f3dcc0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7889d3d95f414d42 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3dbc0a2230 0x7f3dbc0a46f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009710 tx=0x7f3dc0009af0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dbb7fe640 1 -- 172.21.15.8:0/2285941030 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011030 con 0x563c913922b0 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dbb7fe640 1 -- 172.21.15.8:0/2285941030 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc011190 con 0x563c913922b0 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dbb7fe640 1 -- 172.21.15.8:0/2285941030 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00f5c0 con 0x563c913922b0 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2285941030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 msgr2=0x563c91392690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.107+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 0x563c91392690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0114f0 tx=0x7f3dcc0119b0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2285941030 shutdown_connections 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c9139dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c9139b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2285941030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c913922b0 0x563c91392690 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2285941030 >> 172.21.15.8:0/2285941030 conn(0x563c9138c990 msgr2=0x563c9138cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2285941030 shutdown_connections 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2285941030 wait complete. 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- start start 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 Processor -- start 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c9139b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c91398570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c9139b7b0 0x563c91398cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90fcd430 con 0x563c9139b7b0 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90f89e00 con 0x563c91392bd0 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.108+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9127efd0 con 0x563c913922b0 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dda3aa740 10 client.0 _ll_register_callbacks cb 0x563c912d4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd5c91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c9139b7b0 0x563c91398cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd5490640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c9139b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd5c91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c9139b7b0 0x563c91398cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46870/0 (socket says 172.21.15.8:46870) 2024-02-19T16:26:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd5490640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c9139b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:42332/0 (socket says 172.21.15.8:42332) 2024-02-19T16:26:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd5490640 1 -- 172.21.15.8:0/4253704412 learned_addr learned my addr 172.21.15.8:0/4253704412 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c91398570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 msgr2=0x563c9139b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c9139b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/4253704412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c9139b7b0 msgr2=0x563c91398cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c9139b7b0 0x563c91398cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/4253704412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc009cc0 con 0x563c91392bd0 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd5490640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c9139b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c91398570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00fe6d0 tx=0x7f3dc0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d098592741ac6260 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3db8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3db8ff9640 1 -- 172.21.15.8:0/4253704412 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0005020 con 0x563c91392bd0 2024-02-19T16:26:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c90ff97e0 con 0x563c91392bd0 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3db8ff9640 1 -- 172.21.15.8:0/4253704412 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3dc0005180 con 0x563c91392bd0 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3db8ff9640 1 -- 172.21.15.8:0/4253704412 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc0005340 con 0x563c91392bd0 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.109+0000 7f3dda3aa740 10 client.26189 fetch_fsmap learned FSMap version 27 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c912f8610 con 0x563c91392bd0 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3db8ff9640 1 -- 172.21.15.8:0/4253704412 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563c90ff97e0 con 0x563c91392bd0 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3db8ff9640 1 -- 172.21.15.8:0/4253704412 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3dc0027030 con 0x563c91392bd0 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 10 client.26189 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:26:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 20 client.26189 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c913ab950 0x563c913add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 4 client.26189 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c91036720 con 0x563c913ab950 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dd5490640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c913ab950 0x563c913add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3db8ff9640 10 client.26189 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.110+0000 7f3dd5490640 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c913ab950 0x563c913add90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d65ec0c0f409e72b server_cookie=2e0e28400eb1150a in_seq=0 out_seq=0 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3db8ff9640 1 -- 172.21.15.8:0/4253704412 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563c91036720 con 0x563c913ab950 2024-02-19T16:26:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3db8ff9640 10 client.26189 handle_command_reply: tid=0 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 client.26189 shutdown 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 msgr2=0x563c91398570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c91398570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00fe6d0 tx=0x7f3dc0004730 comp rx=0 tx=0).stop 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 shutdown_connections 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c9139b7b0 0x563c91398cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c91392bd0 0x563c91398570 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c913ab950 0x563c913add90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/4253704412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c9139b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3db8ff9640 0 client.26189 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 >> 172.21.15.8:0/4253704412 conn(0x563c9138c990 msgr2=0x563c9138e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 shutdown_connections 2024-02-19T16:26:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.111+0000 7f3dda3aa740 1 -- 172.21.15.8:0/4253704412 wait complete. 2024-02-19T16:26:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.112+0000 7f3dda3aa740 20 client.26189 trim_cache size 0 max 16384 2024-02-19T16:26:29.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 Processor -- start 2024-02-19T16:26:29.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 -- start start 2024-02-19T16:26:29.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c91398510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c9139b7b0 0x563c9139b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c91391cf0 con 0x563c91392bd0 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9138d560 con 0x563c9139b7b0 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c91386e20 con 0x563c913922b0 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46878/0 (socket says 172.21.15.8:46878) 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd5490640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c91398510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/1041268429 learned_addr learned my addr 172.21.15.8:0/1041268429 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c9139b7b0 0x563c9139b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/1041268429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 msgr2=0x563c91398510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c91398510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/1041268429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c9139b7b0 msgr2=0x563c9139b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c9139b7b0 0x563c9139b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/1041268429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c91294ac0 con 0x563c91392bd0 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd5490640 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c91398510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.128+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c9139b7b0 0x563c9139b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139aca0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000f7b0 tx=0x7f3dc0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ad0eecd2ad032d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dbaffd640 1 -- 172.21.15.8:0/1041268429 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0009780 con 0x563c91392bd0 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dbaffd640 1 -- 172.21.15.8:0/1041268429 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dc00035f0 con 0x563c91392bd0 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dbaffd640 1 -- 172.21.15.8:0/1041268429 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0003bd0 con 0x563c91392bd0 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 -- 172.21.15.8:0/1041268429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 msgr2=0x563c9139aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139aca0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000f7b0 tx=0x7f3dc0009280 comp rx=0 tx=0).stop 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 -- 172.21.15.8:0/1041268429 shutdown_connections 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c9139b7b0 0x563c9139b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139aca0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/1041268429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c913922b0 0x563c91398510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 -- 172.21.15.8:0/1041268429 >> 172.21.15.8:0/1041268429 conn(0x563c9138c990 msgr2=0x563c91397e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 -- 172.21.15.8:0/1041268429 shutdown_connections 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.129+0000 7f3dda3aa740 1 -- 172.21.15.8:0/1041268429 wait complete. 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 -- start start 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 Processor -- start 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c913922b0 0x563c9139c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c913a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9139d3d0 con 0x563c91392bd0 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c913a1c80 con 0x563c913922b0 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c913a1e40 con 0x563c9139b7b0 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dda3aa740 10 client.0 _ll_register_callbacks cb 0x7f3dc0013350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46882/0 (socket says 172.21.15.8:46882) 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/2700041922 learned_addr learned my addr 172.21.15.8:0/2700041922 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd5c91640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c913a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd5490640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c913922b0 0x563c9139c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 msgr2=0x563c913a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c913a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/2700041922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c913922b0 msgr2=0x563c9139c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.130+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c913922b0 0x563c9139c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dd4c8f640 1 -- 172.21.15.8:0/2700041922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c912b8570 con 0x563c91392bd0 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dd4c8f640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139cc70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0003910 tx=0x7f3dc0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e630da7d8a04191 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3db8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.141 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3db8ff9640 1 -- 172.21.15.8:0/2700041922 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0003ba0 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3db8ff9640 1 -- 172.21.15.8:0/2700041922 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3dc0010430 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dcc011190 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3db8ff9640 1 -- 172.21.15.8:0/2700041922 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc0009780 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 10 client.16422 fetch_fsmap learned FSMap version 27 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3dd019a6a0 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3db8ff9640 1 -- 172.21.15.8:0/2700041922 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc011190 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3db8ff9640 1 -- 172.21.15.8:0/2700041922 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3dc0027040 con 0x563c91392bd0 2024-02-19T16:26:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 10 client.16422 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 20 client.16422 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c91407d10 0x563c9140a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 4 client.16422 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.131+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563c91085890 con 0x563c91407d10 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dd5490640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c91407d10 0x563c9140a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dd5490640 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c91407d10 0x563c9140a130 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c1be5e466cbd47 server_cookie=564f7fb621566bbd in_seq=0 out_seq=0 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3db8ff9640 10 client.16422 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3db8ff9640 1 -- 172.21.15.8:0/2700041922 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563c91085890 con 0x563c91407d10 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3db8ff9640 10 client.16422 handle_command_reply: tid=0 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 client.16422 shutdown 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 msgr2=0x563c9139cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139cc70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0003910 tx=0x7f3dc0009e60 comp rx=0 tx=0).stop 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 shutdown_connections 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563c91407d10 0x563c9140a130 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563c9139b7b0 0x563c913a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91392bd0 0x563c9139cc70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3db8ff9640 0 client.16422 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 --2- 172.21.15.8:0/2700041922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c913922b0 0x563c9139c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 >> 172.21.15.8:0/2700041922 conn(0x563c9138c990 msgr2=0x563c9138b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 shutdown_connections 2024-02-19T16:26:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 1 -- 172.21.15.8:0/2700041922 wait complete. 2024-02-19T16:26:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.132+0000 7f3dda3aa740 20 client.16422 trim_cache size 0 max 16384 2024-02-19T16:26:29.146 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:26:29.146 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:26:29.146 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:26:29.146 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:26:29.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3dbc0a2230 msgr2=0x7f3dbc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3dbc0a2230 0x7f3dbc0a46f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009710 tx=0x7f3dc0009af0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 msgr2=0x7f3dd0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 0x7f3dd0199aa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009ca0 tx=0x7f3dc80346c0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 shutdown_connections 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dd0107600 0x7f3dd0199aa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dd006a4e0 0x7f3dd019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.148 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3dbc0a2230 0x7f3dbc0a46f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 --2- 172.21.15.8:0/864648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dd0069b20 0x7f3dd019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 >> 172.21.15.8:0/864648422 conn(0x7f3dd0077190 msgr2=0x7f3dd010c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 shutdown_connections 2024-02-19T16:26:29.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.133+0000 7f3dd76f4640 1 -- 172.21.15.8:0/864648422 wait complete. 2024-02-19T16:26:29.149 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:26:29.149 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:26:29.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 -- 172.21.15.8:0/1077878515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580069df0 msgr2=0x7ff58006a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 --2- 172.21.15.8:0/1077878515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580069df0 0x7ff58006a1d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009900 tx=0x7ff57c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:26:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 -- 172.21.15.8:0/1077878515 shutdown_connections 2024-02-19T16:26:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 --2- 172.21.15.8:0/1077878515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580106880 0x7ff58010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 --2- 172.21.15.8:0/1077878515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff58006a7a0 0x7ff580106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 --2- 172.21.15.8:0/1077878515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580069df0 0x7ff58006a1d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 -- 172.21.15.8:0/1077878515 >> 172.21.15.8:0/1077878515 conn(0x7ff580077490 msgr2=0x7ff5800778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 -- 172.21.15.8:0/1077878515 shutdown_connections 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.251+0000 7ff587d1e640 1 -- 172.21.15.8:0/1077878515 wait complete. 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 Processor -- start 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 -- start start 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 0x7ff58019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580106880 0x7ff58019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580197900 0x7ff580197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5801982f0 con 0x7ff580069df0 2024-02-19T16:26:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff580198460 con 0x7ff580106880 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff587d1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff580198620 con 0x7ff580197900 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff585aba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 0x7ff58019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff585aba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 0x7ff58019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46900/0 (socket says 172.21.15.8:46900) 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff585aba640 1 -- 172.21.15.8:0/171899681 learned_addr learned my addr 172.21.15.8:0/171899681 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff5852b9640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580106880 0x7ff58019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.252+0000 7ff5862bb640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580197900 0x7ff580197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff585aba640 1 -- 172.21.15.8:0/171899681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580197900 msgr2=0x7ff580197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff585aba640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580197900 0x7ff580197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff585aba640 1 -- 172.21.15.8:0/171899681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580106880 msgr2=0x7ff58019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff585aba640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580106880 0x7ff58019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff585aba640 1 -- 172.21.15.8:0/171899681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff570009490 con 0x7ff580069df0 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff585aba640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 0x7ff58019d700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005940 tx=0x7ff57c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004350 con 0x7ff580069df0 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff578009840 con 0x7ff580069df0 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff57c0044b0 con 0x7ff580069df0 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003c20 con 0x7ff580069df0 2024-02-19T16:26:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.253+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c009560 con 0x7ff580069df0 2024-02-19T16:26:29.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.254+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff580068730 con 0x7ff580069df0 2024-02-19T16:26:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.255+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff57c0054f0 con 0x7ff580069df0 2024-02-19T16:26:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.255+0000 7ff576ffd640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff56c0a2230 0x7ff56c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.256+0000 7ff5852b9640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff56c0a2230 0x7ff56c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.256+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7ff57c0f0990 con 0x7ff580069df0 2024-02-19T16:26:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.256+0000 7ff5852b9640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff56c0a2230 0x7ff56c0a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5800ffe60 tx=0x7ff570005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.259+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff57c0b9380 con 0x7ff580069df0 2024-02-19T16:26:29.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.478+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff580198de0 con 0x7ff580069df0 2024-02-19T16:26:29.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.478+0000 7ff576ffd640 1 -- 172.21.15.8:0/171899681 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11347 (secure 0 0 0) 0x7ff57c0bca40 con 0x7ff580069df0 2024-02-19T16:26:29.480 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:26:29.480 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:24:42.507626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:26:29.480 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 27 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff56c0a2230 msgr2=0x7ff56c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff56c0a2230 0x7ff56c0a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5800ffe60 tx=0x7ff570005f90 comp rx=0 tx=0).stop 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 msgr2=0x7ff58019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 0x7ff58019d700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005940 tx=0x7ff57c02c670 comp rx=0 tx=0).stop 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 shutdown_connections 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff56c0a2230 0x7ff56c0a46f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff580197900 0x7ff580197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff580106880 0x7ff58019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 --2- 172.21.15.8:0/171899681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff580069df0 0x7ff58019d700 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.480+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 >> 172.21.15.8:0/171899681 conn(0x7ff580077490 msgr2=0x7ff5800fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.481+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 shutdown_connections 2024-02-19T16:26:29.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.481+0000 7ff587d1e640 1 -- 172.21.15.8:0/171899681 wait complete. 2024-02-19T16:26:29.493 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:26:29.493 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:26:29.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 -- 172.21.15.8:0/166769205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e4107580 msgr2=0x7f98e4107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 --2- 172.21.15.8:0/166769205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e4107580 0x7f98e4107960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402bc30 comp rx=0 tx=0).stop 2024-02-19T16:26:29.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 -- 172.21.15.8:0/166769205 shutdown_connections 2024-02-19T16:26:29.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 --2- 172.21.15.8:0/166769205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e41003b0 0x7f98e410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 --2- 172.21.15.8:0/166769205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e40ff9f0 0x7f98e40ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 --2- 172.21.15.8:0/166769205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e4107580 0x7f98e4107960 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 -- 172.21.15.8:0/166769205 >> 172.21.15.8:0/166769205 conn(0x7f98e40769b0 msgr2=0x7f98e40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 -- 172.21.15.8:0/166769205 shutdown_connections 2024-02-19T16:26:29.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 -- 172.21.15.8:0/166769205 wait complete. 2024-02-19T16:26:29.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.596+0000 7f98ea476640 1 Processor -- start 2024-02-19T16:26:29.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 -- start start 2024-02-19T16:26:29.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 0x7f98e419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e41003b0 0x7f98e419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4107580 0x7f98e41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4197f40 con 0x7f98e40ff9f0 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41980b0 con 0x7f98e41003b0 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98ea476640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4198270 con 0x7f98e4107580 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 0x7f98e419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 0x7f98e419d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46920/0 (socket says 172.21.15.8:46920) 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 -- 172.21.15.8:0/687584164 learned_addr learned my addr 172.21.15.8:0/687584164 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e37fe640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e41003b0 0x7f98e419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e8a13640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4107580 0x7f98e41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 -- 172.21.15.8:0/687584164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4107580 msgr2=0x7f98e41978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4107580 0x7f98e41978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 -- 172.21.15.8:0/687584164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e41003b0 msgr2=0x7f98e419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e41003b0 0x7f98e419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 -- 172.21.15.8:0/687584164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009560 con 0x7f98e40ff9f0 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e37fe640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e41003b0 0x7f98e419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.597+0000 7f98e3fff640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 0x7f98e419d6e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005940 tx=0x7f98d402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.598+0000 7f98e17fa640 1 -- 172.21.15.8:0/687584164 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40042d0 con 0x7f98e40ff9f0 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.598+0000 7f98e17fa640 1 -- 172.21.15.8:0/687584164 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98d4004430 con 0x7f98e40ff9f0 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.598+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e41984f0 con 0x7f98e40ff9f0 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.598+0000 7f98e17fa640 1 -- 172.21.15.8:0/687584164 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003c50 con 0x7f98e40ff9f0 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.598+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e41a4710 con 0x7f98e40ff9f0 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e17fa640 1 -- 172.21.15.8:0/687584164 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f98d40054f0 con 0x7f98e40ff9f0 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 Processor -- start 2024-02-19T16:26:29.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 -- start start 2024-02-19T16:26:29.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9250 0x55ea98fa9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fb2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea98fb4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98f64960 con 0x55ea98fb2750 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98f18310 con 0x55ea98fa9250 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98ed051740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98f15840 con 0x55ea98fa9b70 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e17fa640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f98cc0a2230 0x7f98cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea98fb4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fb2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea98fb4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46936/0 (socket says 172.21.15.8:46936) 2024-02-19T16:26:29.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e8a13640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9250 0x55ea98fa9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.600+0000 7f98e37fe640 1 -- 172.21.15.8:0/328724215 learned_addr learned my addr 172.21.15.8:0/328724215 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e8a13640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f98cc0a2230 0x7f98cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e17fa640 1 -- 172.21.15.8:0/687584164 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f98d40f04e0 con 0x7f98e40ff9f0 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e3fff640 1 -- 172.21.15.8:0/328724215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9250 msgr2=0x55ea98fa9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e3fff640 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9250 0x55ea98fa9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e3fff640 1 -- 172.21.15.8:0/328724215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 msgr2=0x55ea98fb4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e3fff640 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea98fb4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e3fff640 1 -- 172.21.15.8:0/328724215 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea98ed3060 con 0x55ea98fa9b70 2024-02-19T16:26:29.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e8a13640 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9250 0x55ea98fa9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e8a13640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f98cc0a2230 0x7f98cc0a46f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005c40 tx=0x7f98dc009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98e3fff640 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fb2210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98d40ed930 tx=0x7f98d408fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa8894a84fa89d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98ca7fc640 1 -- 172.21.15.8:0/328724215 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bd2d0 con 0x55ea98fa9b70 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98ca7fc640 1 -- 172.21.15.8:0/328724215 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98d40bcbd0 con 0x55ea98fa9b70 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.601+0000 7f98ca7fc640 1 -- 172.21.15.8:0/328724215 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bce30 con 0x55ea98fa9b70 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 -- 172.21.15.8:0/328724215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 msgr2=0x55ea98fb2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fb2210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98d40ed930 tx=0x7f98d408fc40 comp rx=0 tx=0).stop 2024-02-19T16:26:29.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 -- 172.21.15.8:0/328724215 shutdown_connections 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea98fb4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fb2210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 --2- 172.21.15.8:0/328724215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9250 0x55ea98fa9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 -- 172.21.15.8:0/328724215 >> 172.21.15.8:0/328724215 conn(0x55ea98fa3980 msgr2=0x55ea98fa3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 -- 172.21.15.8:0/328724215 shutdown_connections 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 -- 172.21.15.8:0/328724215 wait complete. 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 -- start start 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.602+0000 7f98ed051740 1 Processor -- start 2024-02-19T16:26:29.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98faf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 0x55ea98fafc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fb0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98f11020 con 0x55ea98fa9b70 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98be4430 con 0x55ea98fb2750 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98ba0e00 con 0x55ea98fa9250 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 10 client.0 _ll_register_callbacks cb 0x55ea98eb8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 0x55ea98fafc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 0x55ea98fafc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46944/0 (socket says 172.21.15.8:46944) 2024-02-19T16:26:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 -- 172.21.15.8:0/1310733326 learned_addr learned my addr 172.21.15.8:0/1310733326 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e3fff640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98faf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e8a13640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fb0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 -- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 msgr2=0x55ea98faf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98faf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 -- 172.21.15.8:0/1310733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 msgr2=0x55ea98fb0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fb0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 -- 172.21.15.8:0/1310733326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e41a4710 con 0x55ea98fa9b70 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e8a13640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fb0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98e37fe640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 0x55ea98fafc30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f98d8012d20 tx=0x7f98d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5829b62ad4480a76 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98c37fe640 1 -- 172.21.15.8:0/1310733326 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8017070 con 0x55ea98fa9b70 2024-02-19T16:26:29.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98c37fe640 1 -- 172.21.15.8:0/1310733326 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f98d8012110 con 0x55ea98fa9b70 2024-02-19T16:26:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.603+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea98c107e0 con 0x55ea98fa9b70 2024-02-19T16:26:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98c37fe640 1 -- 172.21.15.8:0/1310733326 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98d80122d0 con 0x55ea98fa9b70 2024-02-19T16:26:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 10 client.16440 fetch_fsmap learned FSMap version 27 2024-02-19T16:26:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea98f0f6a0 con 0x55ea98fa9b70 2024-02-19T16:26:29.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98c37fe640 1 -- 172.21.15.8:0/1310733326 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ea98c107e0 con 0x55ea98fa9b70 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98c37fe640 1 -- 172.21.15.8:0/1310733326 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f98d8010430 con 0x55ea98fa9b70 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 10 client.16440 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 10 client.16440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 20 client.16440 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea98fc2860 0x55ea98fc4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 4 client.16440 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea98c4d720 con 0x55ea98fc2860 2024-02-19T16:26:29.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.604+0000 7f98e3fff640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea98fc2860 0x55ea98fc4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98c37fe640 10 client.16440 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98e3fff640 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea98fc2860 0x55ea98fc4c80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcd7aeeb74dde178 server_cookie=68df1ed7fd492641 in_seq=0 out_seq=0 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98c37fe640 1 -- 172.21.15.8:0/1310733326 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ea98c4d720 con 0x55ea98fc2860 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98c37fe640 10 client.16440 handle_command_reply: tid=0 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 client.16440 shutdown 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 msgr2=0x55ea98fafc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 0x55ea98fafc30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f98d8012d20 tx=0x7f98d800f790 comp rx=0 tx=0).stop 2024-02-19T16:26:29.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 shutdown_connections 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea98fc2860 0x55ea98fc4c80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fb0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9b70 0x55ea98fafc30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98c37fe640 0 client.16440 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 --2- 172.21.15.8:0/1310733326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98faf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.605+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 >> 172.21.15.8:0/1310733326 conn(0x55ea98fa3980 msgr2=0x55ea98fa56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.606+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 shutdown_connections 2024-02-19T16:26:29.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.606+0000 7f98ed051740 1 -- 172.21.15.8:0/1310733326 wait complete. 2024-02-19T16:26:29.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.606+0000 7f98ed051740 20 client.16440 trim_cache size 0 max 16384 2024-02-19T16:26:29.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 Processor -- start 2024-02-19T16:26:29.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 -- start start 2024-02-19T16:26:29.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 0x55ea98faff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fbd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fbdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98e96130 con 0x55ea98fa9250 2024-02-19T16:26:29.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98faa640 con 0x55ea98fb2750 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98ed051740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98f9f5c0 con 0x55ea98fa9b70 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 0x55ea98faff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 0x55ea98faff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46960/0 (socket says 172.21.15.8:46960) 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98e3fff640 1 -- 172.21.15.8:0/3324185088 learned_addr learned my addr 172.21.15.8:0/3324185088 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.618+0000 7f98e37fe640 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fbd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e8a13640 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fbdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e3fff640 1 -- 172.21.15.8:0/3324185088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 msgr2=0x55ea98fbd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e3fff640 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fbd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e3fff640 1 -- 172.21.15.8:0/3324185088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 msgr2=0x55ea98fbdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e3fff640 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fbdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e3fff640 1 -- 172.21.15.8:0/3324185088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea98eabc20 con 0x55ea98fa9250 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e37fe640 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fbd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98e3fff640 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 0x55ea98faff90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f98d402c4e0 tx=0x7f98d40e74d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686ee93ff16cdfde server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98c9ffb640 1 -- 172.21.15.8:0/3324185088 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40c5370 con 0x55ea98fa9250 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98c9ffb640 1 -- 172.21.15.8:0/3324185088 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98d40a5040 con 0x55ea98fa9250 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.619+0000 7f98c9ffb640 1 -- 172.21.15.8:0/3324185088 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bafd0 con 0x55ea98fa9250 2024-02-19T16:26:29.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- 172.21.15.8:0/3324185088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 msgr2=0x55ea98faff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 0x55ea98faff90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f98d402c4e0 tx=0x7f98d40e74d0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- 172.21.15.8:0/3324185088 shutdown_connections 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fb2750 0x55ea98fbdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9b70 0x55ea98fbd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- 172.21.15.8:0/3324185088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fa9250 0x55ea98faff90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- 172.21.15.8:0/3324185088 >> 172.21.15.8:0/3324185088 conn(0x55ea98fa3980 msgr2=0x55ea98fa4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- 172.21.15.8:0/3324185088 shutdown_connections 2024-02-19T16:26:29.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- 172.21.15.8:0/3324185088 wait complete. 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- start start 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 Processor -- start 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98fac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9b70 0x55ea98fac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea9901d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea98fad060 con 0x55ea98fb2750 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9901d650 con 0x55ea98fa9b70 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9901d7c0 con 0x55ea98fa9250 2024-02-19T16:26:29.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.620+0000 7f98ed051740 10 client.0 _ll_register_callbacks cb 0x7f98d8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea9901d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98fac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea9901d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46964/0 (socket says 172.21.15.8:46964) 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 -- 172.21.15.8:0/782714262 learned_addr learned my addr 172.21.15.8:0/782714262 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e37fe640 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9b70 0x55ea98fac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 -- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 msgr2=0x55ea98fac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98fac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 -- 172.21.15.8:0/782714262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9b70 msgr2=0x55ea98fac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9b70 0x55ea98fac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 -- 172.21.15.8:0/782714262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea98f0f6a0 con 0x55ea98fb2750 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98e8a13640 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea9901d110 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f98dc010010 tx=0x7f98dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bc2c5abb17bf5b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98c37fe640 1 -- 172.21.15.8:0/782714262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc025070 con 0x55ea98fb2750 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98c37fe640 1 -- 172.21.15.8:0/782714262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f98dc004500 con 0x55ea98fb2750 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98d40bcbd0 con 0x55ea98fb2750 2024-02-19T16:26:29.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98c37fe640 1 -- 172.21.15.8:0/782714262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98dc004700 con 0x55ea98fb2750 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98ed051740 10 client.16446 fetch_fsmap learned FSMap version 27 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.621+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea98ed3060 con 0x55ea98fb2750 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98c37fe640 1 -- 172.21.15.8:0/782714262 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98d40bcbd0 con 0x55ea98fb2750 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98c37fe640 1 -- 172.21.15.8:0/782714262 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 27) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f98dc02b540 con 0x55ea98fb2750 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 10 client.16446 fetch_fsmap finished waiting for FSMap version 27 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 10 client.16446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 20 client.16446 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea9901f050 0x55ea99021470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 4 client.16446 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea98c9c890 con 0x55ea9901f050 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98e3fff640 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea9901f050 0x55ea99021470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98c37fe640 10 client.16446 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98e3fff640 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea9901f050 0x55ea99021470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5db5b7aa9384ff27 server_cookie=1b4b48f5ea1956c2 in_seq=0 out_seq=0 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98c37fe640 1 -- 172.21.15.8:0/782714262 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea98c9c890 con 0x55ea9901f050 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98c37fe640 10 client.16446 handle_command_reply: tid=0 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.622+0000 7f98ed051740 1 client.16446 shutdown 2024-02-19T16:26:29.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 msgr2=0x55ea9901d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea9901d110 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f98dc010010 tx=0x7f98dc004a40 comp rx=0 tx=0).stop 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 shutdown_connections 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55ea9901f050 0x55ea99021470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ea98fb2750 0x55ea9901d110 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea98fa9b70 0x55ea98fac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98c37fe640 0 client.16446 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 --2- 172.21.15.8:0/782714262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea98fa9250 0x55ea98fac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 >> 172.21.15.8:0/782714262 conn(0x55ea98fa3980 msgr2=0x55ea98fa5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 shutdown_connections 2024-02-19T16:26:29.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 1 -- 172.21.15.8:0/782714262 wait complete. 2024-02-19T16:26:29.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ed051740 20 client.16446 trim_cache size 0 max 16384 2024-02-19T16:26:29.641 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:26:29.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f98cc0a2230 msgr2=0x7f98cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f98cc0a2230 0x7f98cc0a46f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005c40 tx=0x7f98dc009500 comp rx=0 tx=0).stop 2024-02-19T16:26:29.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 msgr2=0x7f98e419d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 0x7f98e419d6e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005940 tx=0x7f98d402c670 comp rx=0 tx=0).stop 2024-02-19T16:26:29.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 shutdown_connections 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f98cc0a2230 0x7f98cc0a46f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4107580 0x7f98e41978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e41003b0 0x7f98e419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 --2- 172.21.15.8:0/687584164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98e40ff9f0 0x7f98e419d6e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 >> 172.21.15.8:0/687584164 conn(0x7f98e40769b0 msgr2=0x7f98e40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 shutdown_connections 2024-02-19T16:26:29.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.623+0000 7f98ea476640 1 -- 172.21.15.8:0/687584164 wait complete. 2024-02-19T16:26:29.644 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:26:29.644 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:26:29.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 -- 172.21.15.8:0/1833732346 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781056c0 msgr2=0x7fd87810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 --2- 172.21.15.8:0/1833732346 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781056c0 0x7fd87810d290 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd868009930 tx=0x7fd86802bc30 comp rx=0 tx=0).stop 2024-02-19T16:26:29.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 -- 172.21.15.8:0/1833732346 shutdown_connections 2024-02-19T16:26:29.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 --2- 172.21.15.8:0/1833732346 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781056c0 0x7fd87810d290 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 --2- 172.21.15.8:0/1833732346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd878100fa0 0x7fd878104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 --2- 172.21.15.8:0/1833732346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8781005f0 0x7fd8781009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 -- 172.21.15.8:0/1833732346 >> 172.21.15.8:0/1833732346 conn(0x7fd8780774c0 msgr2=0x7fd8780778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 -- 172.21.15.8:0/1833732346 shutdown_connections 2024-02-19T16:26:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.750+0000 7fd87cc07640 1 -- 172.21.15.8:0/1833732346 wait complete. 2024-02-19T16:26:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 Processor -- start 2024-02-19T16:26:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 -- start start 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8781005f0 0x7fd878198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8781056c0 0x7fd878198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 0x7fd8781a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781a4060 con 0x7fd8781993e0 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781a41d0 con 0x7fd8781056c0 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87cc07640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781a4390 con 0x7fd8781005f0 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 0x7fd8781a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 0x7fd8781a3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46978/0 (socket says 172.21.15.8:46978) 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 -- 172.21.15.8:0/4155708595 learned_addr learned my addr 172.21.15.8:0/4155708595 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd87659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8781005f0 0x7fd878198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd875d9b640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8781056c0 0x7fd878198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 -- 172.21.15.8:0/4155708595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8781005f0 msgr2=0x7fd878198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8781005f0 0x7fd878198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 -- 172.21.15.8:0/4155708595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8781056c0 msgr2=0x7fd878198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8781056c0 0x7fd878198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.751+0000 7fd876d9d640 1 -- 172.21.15.8:0/4155708595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009560 con 0x7fd8781993e0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.752+0000 7fd876d9d640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 0x7fd8781a3990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd868005940 tx=0x7fd86802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.752+0000 7fd86f7fe640 1 -- 172.21.15.8:0/4155708595 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680042d0 con 0x7fd8781993e0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.752+0000 7fd86f7fe640 1 -- 172.21.15.8:0/4155708595 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd868004430 con 0x7fd8781993e0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.752+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8781a4610 con 0x7fd8781993e0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.752+0000 7fd86f7fe640 1 -- 172.21.15.8:0/4155708595 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868003b50 con 0x7fd8781993e0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.752+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8781a4b20 con 0x7fd8781993e0 2024-02-19T16:26:29.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.754+0000 7fd86f7fe640 1 -- 172.21.15.8:0/4155708595 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fd868005460 con 0x7fd8781993e0 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.754+0000 7fd87f7e4740 1 Processor -- start 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.754+0000 7fd87f7e4740 1 -- start start 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd86f7fe640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8640a2230 0x7fd8640a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2d9b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2d56960 con 0x55d6e2d9b250 2024-02-19T16:26:29.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2d0a310 con 0x55d6e2d9bb70 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2d07840 con 0x55d6e2da4750 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd875d9b640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8640a2230 0x7fd8640a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd86f7fe640 1 -- 172.21.15.8:0/4155708595 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd8680f04b0 con 0x7fd8781993e0 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2d9b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2d9b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46980/0 (socket says 172.21.15.8:46980) 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 -- 172.21.15.8:0/952548608 learned_addr learned my addr 172.21.15.8:0/952548608 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd875d9b640 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd876d9d640 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 -- 172.21.15.8:0/952548608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 msgr2=0x55d6e2da6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 -- 172.21.15.8:0/952548608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 msgr2=0x55d6e2da4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 -- 172.21.15.8:0/952548608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e2cc5060 con 0x55d6e2d9b250 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd875d9b640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8640a2230 0x7fd8640a46f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd87806a670 tx=0x7fd860009a30 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd87659c640 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2d9b630 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd8700114f0 tx=0x7fd8700119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7960140fc00bc15f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.755+0000 7fd86cff9640 1 -- 172.21.15.8:0/952548608 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870004d20 con 0x55d6e2d9b250 2024-02-19T16:26:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd86cff9640 1 -- 172.21.15.8:0/952548608 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd870004e80 con 0x55d6e2d9b250 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd86cff9640 1 -- 172.21.15.8:0/952548608 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870005280 con 0x55d6e2d9b250 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 -- 172.21.15.8:0/952548608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 msgr2=0x55d6e2d9b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2d9b630 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd8700114f0 tx=0x7fd8700119b0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 -- 172.21.15.8:0/952548608 shutdown_connections 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/952548608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2d9b630 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 -- 172.21.15.8:0/952548608 >> 172.21.15.8:0/952548608 conn(0x55d6e2d95980 msgr2=0x55d6e2d95d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 -- 172.21.15.8:0/952548608 shutdown_connections 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.756+0000 7fd87f7e4740 1 -- 172.21.15.8:0/952548608 wait complete. 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 -- start start 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 Processor -- start 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2da1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2d03020 con 0x55d6e2d9b250 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e29d6430 con 0x55d6e2d9bb70 2024-02-19T16:26:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2992e00 con 0x55d6e2da4750 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87f7e4740 10 client.0 _ll_register_callbacks cb 0x55d6e2caacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2da1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2da1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46984/0 (socket says 172.21.15.8:46984) 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 -- 172.21.15.8:0/2739750827 learned_addr learned my addr 172.21.15.8:0/2739750827 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd875d9b640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd876d9d640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 -- 172.21.15.8:0/2739750827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 msgr2=0x55d6e2da21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 -- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 msgr2=0x55d6e2da1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd87659c640 1 -- 172.21.15.8:0/2739750827 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd870009c80 con 0x55d6e2d9b250 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.757+0000 7fd876d9d640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87659c640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2da1740 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd870004640 tx=0x7fd870003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d29f3caa5e32250 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd85dffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd85dffb640 1 -- 172.21.15.8:0/2739750827 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870004460 con 0x55d6e2d9b250 2024-02-19T16:26:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd85dffb640 1 -- 172.21.15.8:0/2739750827 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd87000f4e0 con 0x55d6e2d9b250 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6e2a3f720 con 0x55d6e2d9b250 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd85dffb640 1 -- 172.21.15.8:0/2739750827 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd87000f6a0 con 0x55d6e2d9b250 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 10 client.16458 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d6e2d016a0 con 0x55d6e2d9b250 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd85dffb640 1 -- 172.21.15.8:0/2739750827 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d6e2a3f720 con 0x55d6e2d9b250 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd85dffb640 1 -- 172.21.15.8:0/2739750827 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd870021040 con 0x55d6e2d9b250 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 10 client.16458 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 10 client.16458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 20 client.16458 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2db4bf0 0x55d6e2db7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 4 client.16458 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.758+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6e2a8e890 con 0x55d6e2db4bf0 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.759+0000 7fd875d9b640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2db4bf0 0x55d6e2db7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.759+0000 7fd875d9b640 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2db4bf0 0x55d6e2db7010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3c306412295c6f5 server_cookie=8eb8c281ccf465d in_seq=0 out_seq=0 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.759+0000 7fd85dffb640 10 client.16458 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd85dffb640 1 -- 172.21.15.8:0/2739750827 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d6e2a8e890 con 0x55d6e2db4bf0 2024-02-19T16:26:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd85dffb640 10 client.16458 handle_command_reply: tid=0 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 client.16458 shutdown 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 msgr2=0x55d6e2da1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2da1740 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd870004640 tx=0x7fd870003ce0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 shutdown_connections 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2db4bf0 0x55d6e2db7010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2da21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd85dffb640 0 client.16458 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:26:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2739750827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9b250 0x55d6e2da1740 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 >> 172.21.15.8:0/2739750827 conn(0x55d6e2d95980 msgr2=0x55d6e2d976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 shutdown_connections 2024-02-19T16:26:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2739750827 wait complete. 2024-02-19T16:26:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.760+0000 7fd87f7e4740 20 client.16458 trim_cache size 0 max 16384 2024-02-19T16:26:29.776 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:29 smithi008 ceph-mon[32573]: pgmap v111: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-02-19T16:26:29.776 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:29 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/171899681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:26:29.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 Processor -- start 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 -- start start 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2d9b250 0x55d6e2da3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2da4750 0x55d6e2dafad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2c88130 con 0x55d6e2d9bb70 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2d9a6a0 con 0x55d6e2da4750 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2d910a0 con 0x55d6e2d9b250 2024-02-19T16:26:29.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46998/0 (socket says 172.21.15.8:46998) 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 -- 172.21.15.8:0/172242127 learned_addr learned my addr 172.21.15.8:0/172242127 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd87659c640 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2d9b250 0x55d6e2da3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd876d9d640 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2da4750 0x55d6e2dafad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 -- 172.21.15.8:0/172242127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2d9b250 msgr2=0x55d6e2da3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2d9b250 0x55d6e2da3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 -- 172.21.15.8:0/172242127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2da4750 msgr2=0x55d6e2dafad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2da4750 0x55d6e2dafad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 -- 172.21.15.8:0/172242127 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e2c9dc20 con 0x55d6e2d9bb70 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.773+0000 7fd875d9b640 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1fe0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55d6e2d9ad20 tx=0x7fd860005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9db55ab7661a84 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd85ffff640 1 -- 172.21.15.8:0/172242127 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860033070 con 0x55d6e2d9bb70 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd85ffff640 1 -- 172.21.15.8:0/172242127 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd860028040 con 0x55d6e2d9bb70 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd85ffff640 1 -- 172.21.15.8:0/172242127 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860026040 con 0x55d6e2d9bb70 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 -- 172.21.15.8:0/172242127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 msgr2=0x55d6e2da1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1fe0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55d6e2d9ad20 tx=0x7fd860005f90 comp rx=0 tx=0).stop 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 -- 172.21.15.8:0/172242127 shutdown_connections 2024-02-19T16:26:29.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2da4750 0x55d6e2dafad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da1fe0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/172242127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2d9b250 0x55d6e2da3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 -- 172.21.15.8:0/172242127 >> 172.21.15.8:0/172242127 conn(0x55d6e2d95980 msgr2=0x55d6e2d94b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 -- 172.21.15.8:0/172242127 shutdown_connections 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.774+0000 7fd87f7e4740 1 -- 172.21.15.8:0/172242127 wait complete. 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 -- start start 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 Processor -- start 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9b250 0x55d6e2da8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2e094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2e09a20 con 0x55d6e2d9bb70 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2e09b90 con 0x55d6e2d9b250 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2e09d50 con 0x55d6e2da4750 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87f7e4740 10 client.0 _ll_register_callbacks cb 0x7fd860013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47002/0 (socket says 172.21.15.8:47002) 2024-02-19T16:26:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9b250 0x55d6e2da8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 -- 172.21.15.8:0/2767931020 learned_addr learned my addr 172.21.15.8:0/2767931020 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd876d9d640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2e094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 -- 172.21.15.8:0/2767931020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 msgr2=0x55d6e2e094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2e094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 -- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9b250 msgr2=0x55d6e2da8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9b250 0x55d6e2da8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 -- 172.21.15.8:0/2767931020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86001b040 con 0x55d6e2d9bb70 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd87659c640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9b250 0x55d6e2da8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd875d9b640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da8870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd860031040 tx=0x7fd86002ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b9aac1a20fc80c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.775+0000 7fd85dffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd85dffb640 1 -- 172.21.15.8:0/2767931020 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860033040 con 0x55d6e2d9bb70 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd85dffb640 1 -- 172.21.15.8:0/2767931020 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd860013e40 con 0x55d6e2d9bb70 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6e2a027e0 con 0x55d6e2d9bb70 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd85dffb640 1 -- 172.21.15.8:0/2767931020 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd860026040 con 0x55d6e2d9bb70 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 10 client.16464 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d6e2cc38c0 con 0x55d6e2d9bb70 2024-02-19T16:26:29.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd85dffb640 1 -- 172.21.15.8:0/2767931020 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d6e2a027e0 con 0x55d6e2d9bb70 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd85dffb640 1 -- 172.21.15.8:0/2767931020 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd860038410 con 0x55d6e2d9bb70 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 10 client.16464 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 10 client.16464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 20 client.16464 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2e116a0 0x55d6e2e13ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 4 client.16464 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d6e29ae700 con 0x55d6e2e116a0 2024-02-19T16:26:29.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.776+0000 7fd87659c640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2e116a0 0x55d6e2e13ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87659c640 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2e116a0 0x55d6e2e13ac0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfd404ed71d1c71c server_cookie=34f397b7f51153f in_seq=0 out_seq=0 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd85dffb640 10 client.16464 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd85dffb640 1 -- 172.21.15.8:0/2767931020 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d6e29ae700 con 0x55d6e2e116a0 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd85dffb640 10 client.16464 handle_command_reply: tid=0 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 client.16464 shutdown 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 msgr2=0x55d6e2da8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da8870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd860031040 tx=0x7fd86002ec20 comp rx=0 tx=0).stop 2024-02-19T16:26:29.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 shutdown_connections 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55d6e2e116a0 0x55d6e2e13ac0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6e2da4750 0x55d6e2e094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d6e2d9bb70 0x55d6e2da8870 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd85dffb640 0 client.16464 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 --2- 172.21.15.8:0/2767931020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6e2d9b250 0x55d6e2da8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 >> 172.21.15.8:0/2767931020 conn(0x55d6e2d95980 msgr2=0x55d6e2dbb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 shutdown_connections 2024-02-19T16:26:29.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 1 -- 172.21.15.8:0/2767931020 wait complete. 2024-02-19T16:26:29.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.777+0000 7fd87f7e4740 20 client.16464 trim_cache size 0 max 16384 2024-02-19T16:26:29.795 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8640a2230 msgr2=0x7fd8640a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8640a2230 0x7fd8640a46f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd87806a670 tx=0x7fd860009a30 comp rx=0 tx=0).stop 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 msgr2=0x7fd8781a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 0x7fd8781a3990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd868005940 tx=0x7fd86802c5d0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 shutdown_connections 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8781993e0 0x7fd8781a3990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8781056c0 0x7fd878198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8640a2230 0x7fd8640a46f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 --2- 172.21.15.8:0/4155708595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8781005f0 0x7fd878198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 >> 172.21.15.8:0/4155708595 conn(0x7fd8780774c0 msgr2=0x7fd8780fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 shutdown_connections 2024-02-19T16:26:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.778+0000 7fd87cc07640 1 -- 172.21.15.8:0/4155708595 wait complete. 2024-02-19T16:26:29.798 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:26:29.799 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:26:29.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 -- 172.21.15.8:0/2977820461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd010d620 msgr2=0x7fccd01151f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 --2- 172.21.15.8:0/2977820461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd010d620 0x7fccd01151f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009be0 tx=0x7fccc802bef0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 -- 172.21.15.8:0/2977820461 shutdown_connections 2024-02-19T16:26:29.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 --2- 172.21.15.8:0/2977820461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd010d620 0x7fccd01151f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 --2- 172.21.15.8:0/2977820461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd01099a0 0x7fccd010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 --2- 172.21.15.8:0/2977820461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd0108ff0 0x7fccd01093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 -- 172.21.15.8:0/2977820461 >> 172.21.15.8:0/2977820461 conn(0x7fccd006ca30 msgr2=0x7fccd006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.902+0000 7fccd724e640 1 -- 172.21.15.8:0/2977820461 shutdown_connections 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 -- 172.21.15.8:0/2977820461 wait complete. 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 Processor -- start 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 -- start start 2024-02-19T16:26:29.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0108ff0 0x7fccd019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 0x7fccd019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd01a29d0 0x7fccd01a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd01a33c0 con 0x7fccd010d620 2024-02-19T16:26:29.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd01a3530 con 0x7fccd0108ff0 2024-02-19T16:26:29.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd724e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd01a36a0 con 0x7fccd01a29d0 2024-02-19T16:26:29.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fcccffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 0x7fccd019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd4fea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0108ff0 0x7fccd019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fcccffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 0x7fccd019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47018/0 (socket says 172.21.15.8:47018) 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fcccffff640 1 -- 172.21.15.8:0/1205186803 learned_addr learned my addr 172.21.15.8:0/1205186803 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.903+0000 7fccd57eb640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd01a29d0 0x7fccd01a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccffff640 1 -- 172.21.15.8:0/1205186803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd01a29d0 msgr2=0x7fccd01a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccffff640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd01a29d0 0x7fccd01a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccffff640 1 -- 172.21.15.8:0/1205186803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0108ff0 msgr2=0x7fccd019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccffff640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0108ff0 0x7fccd019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccffff640 1 -- 172.21.15.8:0/1205186803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8009840 con 0x7fccd010d620 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fccd4fea640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0108ff0 0x7fccd019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccffff640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 0x7fccd019e6a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e9c0 tx=0x7fccbc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccdffb640 1 -- 172.21.15.8:0/1205186803 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc009700 con 0x7fccd010d620 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd01aac80 con 0x7fccd010d620 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccdffb640 1 -- 172.21.15.8:0/1205186803 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccbc005500 con 0x7fccd010d620 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fcccdffb640 1 -- 172.21.15.8:0/1205186803 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc010430 con 0x7fccd010d620 2024-02-19T16:26:29.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.904+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01ab180 con 0x7fccd010d620 2024-02-19T16:26:29.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.906+0000 7fcccdffb640 1 -- 172.21.15.8:0/1205186803 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fccbc00ccc0 con 0x7fccd010d620 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 Processor -- start 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 -- start start 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fcccdffb640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fccc00a2230 0x7fccc00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd57eb640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fccc00a2230 0x7fccc00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b7b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844b2d960 con 0x55d844b7b750 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844ae1310 con 0x55d844b72250 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fcccdffb640 1 -- 172.21.15.8:0/1205186803 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fccbc014070 con 0x7fccd010d620 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844ade840 con 0x55d844b72b70 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd57eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fcccffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b7b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd4fea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd57eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b7db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47026/0 (socket says 172.21.15.8:47026) 2024-02-19T16:26:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd57eb640 1 -- 172.21.15.8:0/2258171897 learned_addr learned my addr 172.21.15.8:0/2258171897 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd4fea640 1 -- 172.21.15.8:0/2258171897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 msgr2=0x55d844b7b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd4fea640 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b7b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd4fea640 1 -- 172.21.15.8:0/2258171897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 msgr2=0x55d844b7db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd4fea640 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b7db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fccd4fea640 1 -- 172.21.15.8:0/2258171897 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d844a9c060 con 0x55d844b72250 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.907+0000 7fcccffff640 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccd57eb640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fccc00a2230 0x7fccc00a46f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccc8036040 tx=0x7fccc802cb60 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccd4fea640 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b72630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccc400e5d0 tx=0x7fccc400a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf958edb8984dd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccbaffd640 1 -- 172.21.15.8:0/2258171897 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4014040 con 0x55d844b72250 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccbaffd640 1 -- 172.21.15.8:0/2258171897 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccc400dd90 con 0x55d844b72250 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccbaffd640 1 -- 172.21.15.8:0/2258171897 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4003b90 con 0x55d844b72250 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2258171897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 msgr2=0x55d844b72630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.908+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b72630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccc400e5d0 tx=0x7fccc400a370 comp rx=0 tx=0).stop 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2258171897 shutdown_connections 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b7db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2258171897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b72630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2258171897 >> 172.21.15.8:0/2258171897 conn(0x55d844b6c980 msgr2=0x55d844b6cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2258171897 shutdown_connections 2024-02-19T16:26:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2258171897 wait complete. 2024-02-19T16:26:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- start start 2024-02-19T16:26:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 Processor -- start 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b78c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b79150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844ada020 con 0x55d844b7b750 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8447ad430 con 0x55d844b72250 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844769e00 con 0x55d844b72b70 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd9e2e740 10 client.0 _ll_register_callbacks cb 0x55d844a81cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd57eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b79150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd57eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b79150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47030/0 (socket says 172.21.15.8:47030) 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd57eb640 1 -- 172.21.15.8:0/1154569717 learned_addr learned my addr 172.21.15.8:0/1154569717 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fccd4fea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.909+0000 7fcccffff640 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b78c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd57eb640 1 -- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 msgr2=0x55d844b78c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd57eb640 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b78c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd57eb640 1 -- 172.21.15.8:0/1154569717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 msgr2=0x55d844b786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd57eb640 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd57eb640 1 -- 172.21.15.8:0/1154569717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc400ad50 con 0x55d844b7b750 2024-02-19T16:26:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd57eb640 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b79150 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006010 tx=0x7fccc8045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a0ea2fcc7fdb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccaffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccaffff640 1 -- 172.21.15.8:0/1154569717 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8004690 con 0x55d844b7b750 2024-02-19T16:26:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccaffff640 1 -- 172.21.15.8:0/1154569717 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fccc80047f0 con 0x55d844b7b750 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8447d97e0 con 0x55d844b7b750 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccaffff640 1 -- 172.21.15.8:0/1154569717 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc804d480 con 0x55d844b7b750 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd9e2e740 10 client.16476 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d844ad86a0 con 0x55d844b7b750 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.910+0000 7fccaffff640 1 -- 172.21.15.8:0/1154569717 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d8447d97e0 con 0x55d844b7b750 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccaffff640 1 -- 172.21.15.8:0/1154569717 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccc8041640 con 0x55d844b7b750 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd9e2e740 10 client.16476 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:29.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd9e2e740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd9e2e740 20 client.16476 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844b8ba80 0x55d844b8dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd9e2e740 4 client.16476 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d844816720 con 0x55d844b8ba80 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd4fea640 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844b8ba80 0x55d844b8dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccaffff640 10 client.16476 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.911+0000 7fccd4fea640 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844b8ba80 0x55d844b8dea0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b033cd7571229be server_cookie=60f25cf50ba1e982 in_seq=0 out_seq=0 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccaffff640 1 -- 172.21.15.8:0/1154569717 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d844816720 con 0x55d844b8ba80 2024-02-19T16:26:29.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccaffff640 10 client.16476 handle_command_reply: tid=0 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 client.16476 shutdown 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 msgr2=0x55d844b79150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b79150 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006010 tx=0x7fccc8045040 comp rx=0 tx=0).stop 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 shutdown_connections 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b7b750 0x55d844b79150 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844b8ba80 0x55d844b8dea0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b72b70 0x55d844b78c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccaffff640 0 client.16476 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/1154569717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 >> 172.21.15.8:0/1154569717 conn(0x55d844b6c980 msgr2=0x55d844b6e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 shutdown_connections 2024-02-19T16:26:29.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 1 -- 172.21.15.8:0/1154569717 wait complete. 2024-02-19T16:26:29.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.912+0000 7fccd9e2e740 20 client.16476 trim_cache size 0 max 16384 2024-02-19T16:26:29.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 Processor -- start 2024-02-19T16:26:29.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 -- start start 2024-02-19T16:26:29.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 0x55d844b78f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844b86ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844a5f130 con 0x55d844b72b70 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844b682b0 con 0x55d844b72250 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844b71ee0 con 0x55d844b7b750 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 0x55d844b78f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 0x55d844b78f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47034/0 (socket says 172.21.15.8:47034) 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 -- 172.21.15.8:0/3219513258 learned_addr learned my addr 172.21.15.8:0/3219513258 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd4fea640 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccd57eb640 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844b86ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 -- 172.21.15.8:0/3219513258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 msgr2=0x55d844b86ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844b86ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 -- 172.21.15.8:0/3219513258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 msgr2=0x55d844b7ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b7ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 -- 172.21.15.8:0/3219513258 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d844a74c20 con 0x55d844b72b70 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fcccffff640 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 0x55d844b78f70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c6a0 tx=0x7fccbc005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933cbfdb25bee8e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccba7fc640 1 -- 172.21.15.8:0/3219513258 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc09e040 con 0x55d844b72b70 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.925+0000 7fccba7fc640 1 -- 172.21.15.8:0/3219513258 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccbc09eee0 con 0x55d844b72b70 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccba7fc640 1 -- 172.21.15.8:0/3219513258 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0ce320 con 0x55d844b72b70 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 -- 172.21.15.8:0/3219513258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 msgr2=0x55d844b78f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 0x55d844b78f70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c6a0 tx=0x7fccbc005670 comp rx=0 tx=0).stop 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 -- 172.21.15.8:0/3219513258 shutdown_connections 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844b86ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72b70 0x55d844b78f70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/3219513258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72250 0x55d844b7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 -- 172.21.15.8:0/3219513258 >> 172.21.15.8:0/3219513258 conn(0x55d844b6c980 msgr2=0x55d844b6bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 -- 172.21.15.8:0/3219513258 shutdown_connections 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 -- 172.21.15.8:0/3219513258 wait complete. 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 -- start start 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.926+0000 7fccd9e2e740 1 Processor -- start 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 0x55d844b7f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72b70 0x55d844b7f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844be4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844be4cc0 con 0x55d844b72250 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844be4e00 con 0x55d844b72b70 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d844be4fc0 con 0x55d844b7b750 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 10 client.0 _ll_register_callbacks cb 0x7fccc8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 0x55d844b7f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 0x55d844b7f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47040/0 (socket says 172.21.15.8:47040) 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 -- 172.21.15.8:0/2910817815 learned_addr learned my addr 172.21.15.8:0/2910817815 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd57eb640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844be4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fcccffff640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72b70 0x55d844b7f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 -- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 msgr2=0x55d844be4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844be4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 -- 172.21.15.8:0/2910817815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72b70 msgr2=0x55d844b7f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72b70 0x55d844b7f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 -- 172.21.15.8:0/2910817815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01aac80 con 0x55d844b72250 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd4fea640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 0x55d844b7f2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fccc400ef20 tx=0x7fccc4003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd58f397e0621f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccaffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccaffff640 1 -- 172.21.15.8:0/2910817815 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4014070 con 0x55d844b72250 2024-02-19T16:26:29.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccaffff640 1 -- 172.21.15.8:0/2910817815 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fccc4003cb0 con 0x55d844b72250 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.927+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc400dd90 con 0x55d844b72250 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccaffff640 1 -- 172.21.15.8:0/2910817815 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc4028040 con 0x55d844b72250 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 10 client.16482 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d844a9a8c0 con 0x55d844b72250 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccaffff640 1 -- 172.21.15.8:0/2910817815 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccc400dd90 con 0x55d844b72250 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccaffff640 1 -- 172.21.15.8:0/2910817815 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccc402b050 con 0x55d844b72250 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 10 client.16482 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:29.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 20 client.16482 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844bec860 0x55d844beec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 4 client.16482 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d844865890 con 0x55d844bec860 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fcccffff640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844bec860 0x55d844beec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccaffff640 10 client.16482 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.928+0000 7fcccffff640 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844bec860 0x55d844beec80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe01f8536ea85952 server_cookie=d2147aea34d4ad9e in_seq=0 out_seq=0 2024-02-19T16:26:29.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccaffff640 1 -- 172.21.15.8:0/2910817815 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d844865890 con 0x55d844bec860 2024-02-19T16:26:29.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccaffff640 10 client.16482 handle_command_reply: tid=0 2024-02-19T16:26:29.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 client.16482 shutdown 2024-02-19T16:26:29.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 msgr2=0x55d844b7f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 0x55d844b7f2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fccc400ef20 tx=0x7fccc4003770 comp rx=0 tx=0).stop 2024-02-19T16:26:29.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 shutdown_connections 2024-02-19T16:26:29.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d844bec860 0x55d844beec80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d844b7b750 0x55d844be4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d844b72b70 0x55d844b7f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccaffff640 0 client.16482 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 --2- 172.21.15.8:0/2910817815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d844b72250 0x55d844b7f2c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 >> 172.21.15.8:0/2910817815 conn(0x55d844b6c980 msgr2=0x55d844b6db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 shutdown_connections 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 1 -- 172.21.15.8:0/2910817815 wait complete. 2024-02-19T16:26:29.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.929+0000 7fccd9e2e740 20 client.16482 trim_cache size 0 max 16384 2024-02-19T16:26:29.946 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:26:29.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fccc00a2230 msgr2=0x7fccc00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fccc00a2230 0x7fccc00a46f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccc8036040 tx=0x7fccc802cb60 comp rx=0 tx=0).stop 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 msgr2=0x7fccd019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 0x7fccd019e6a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e9c0 tx=0x7fccbc00edc0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 shutdown_connections 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fccc00a2230 0x7fccc00a46f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd01a29d0 0x7fccd01a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccd010d620 0x7fccd019e6a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 --2- 172.21.15.8:0/1205186803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0108ff0 0x7fccd019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:29.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 >> 172.21.15.8:0/1205186803 conn(0x7fccd006ca30 msgr2=0x7fccd0071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:29.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 shutdown_connections 2024-02-19T16:26:29.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:29.930+0000 7fccd724e640 1 -- 172.21.15.8:0/1205186803 wait complete. 2024-02-19T16:26:30.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:29 smithi191 ceph-mon[37203]: pgmap v111: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-02-19T16:26:30.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:29 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/171899681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:26:30.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:29 smithi156 ceph-mon[36502]: pgmap v111: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-02-19T16:26:30.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:29 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/171899681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:26:30.950 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:26:30.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:26:30.951 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:26:30.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:30 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:26:31.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:30 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:26:31.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 -- 172.21.15.8:0/2578777763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 msgr2=0x7f7094100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 --2- 172.21.15.8:0/2578777763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f7094100920 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7088009930 tx=0x7f708802bc30 comp rx=0 tx=0).stop 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 -- 172.21.15.8:0/2578777763 shutdown_connections 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 --2- 172.21.15.8:0/2578777763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7094105610 0x7f709410d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 --2- 172.21.15.8:0/2578777763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7094100ef0 0x7f7094104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 --2- 172.21.15.8:0/2578777763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f7094100920 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 -- 172.21.15.8:0/2578777763 >> 172.21.15.8:0/2578777763 conn(0x7f70940fbda0 msgr2=0x7f70940fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 -- 172.21.15.8:0/2578777763 shutdown_connections 2024-02-19T16:26:31.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.068+0000 7f7099ba0640 1 -- 172.21.15.8:0/2578777763 wait complete. 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 Processor -- start 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 -- start start 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f709419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7094105610 0x7f709419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7094197910 0x7f7094197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7094198300 con 0x7f7094100540 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7094198470 con 0x7f7094197910 2024-02-19T16:26:31.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7099ba0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7094198630 con 0x7f7094105610 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f709419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f709419d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47052/0 (socket says 172.21.15.8:47052) 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 -- 172.21.15.8:0/315513028 learned_addr learned my addr 172.21.15.8:0/315513028 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f709939f640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7094197910 0x7f7094197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7093fff640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7094105610 0x7f709419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 -- 172.21.15.8:0/315513028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7094105610 msgr2=0x7f709419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7094105610 0x7f709419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 -- 172.21.15.8:0/315513028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7094197910 msgr2=0x7f7094197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7094197910 0x7f7094197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.069+0000 7f7098b9e640 1 -- 172.21.15.8:0/315513028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7088009560 con 0x7f7094100540 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.070+0000 7f7098b9e640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f709419d760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7088009230 tx=0x7f708802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.070+0000 7f7091ffb640 1 -- 172.21.15.8:0/315513028 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088004350 con 0x7f7094100540 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.070+0000 7f7091ffb640 1 -- 172.21.15.8:0/315513028 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70880044b0 con 0x7f7094100540 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.070+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70941a4620 con 0x7f7094100540 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.070+0000 7f7091ffb640 1 -- 172.21.15.8:0/315513028 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088003d00 con 0x7f7094100540 2024-02-19T16:26:31.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.070+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70941a4ad0 con 0x7f7094100540 2024-02-19T16:26:31.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:30 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:26:31.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.072+0000 7f7091ffb640 1 -- 172.21.15.8:0/315513028 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f7088004df0 con 0x7f7094100540 2024-02-19T16:26:31.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 Processor -- start 2024-02-19T16:26:31.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 -- start start 2024-02-19T16:26:31.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5430 0x55f67ede5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5de0 0x55f67edee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f7091ffb640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f70840a2230 0x7f70840a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 0x55f67edef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ed53f90 con 0x55f67edeeee0 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ed514c0 con 0x55f67ede5430 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709d9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ed4cca0 con 0x55f67ede5de0 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f7093fff640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f70840a2230 0x7f70840a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 0x55f67edef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f7091ffb640 1 -- 172.21.15.8:0/315513028 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f70880f0320 con 0x7f7094100540 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 0x55f67edef340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47066/0 (socket says 172.21.15.8:47066) 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f7098b9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5de0 0x55f67edee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 -- 172.21.15.8:0/4229320905 learned_addr learned my addr 172.21.15.8:0/4229320905 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5430 0x55f67ede5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 -- 172.21.15.8:0/4229320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5de0 msgr2=0x55f67edee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5de0 0x55f67edee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 -- 172.21.15.8:0/4229320905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5430 msgr2=0x55f67ede5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5430 0x55f67ede5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f709939f640 1 -- 172.21.15.8:0/4229320905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f67ed0f0b0 con 0x55f67edeeee0 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.073+0000 7f7098b9e640 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5de0 0x55f67edee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f7093fff640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f70840a2230 0x7f70840a46f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70940fc2c0 tx=0x7f7080005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709939f640 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 0x55f67edef340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f708c005f40 tx=0x7f708c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5cd6da3ec8ace7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f707effd640 1 -- 172.21.15.8:0/4229320905 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c011130 con 0x55f67edeeee0 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f707effd640 1 -- 172.21.15.8:0/4229320905 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f708c011290 con 0x55f67edeeee0 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f707effd640 1 -- 172.21.15.8:0/4229320905 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00fc00 con 0x55f67edeeee0 2024-02-19T16:26:31.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709d9d8740 1 -- 172.21.15.8:0/4229320905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 msgr2=0x55f67edef340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 0x55f67edef340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f708c005f40 tx=0x7f708c005bd0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709d9d8740 1 -- 172.21.15.8:0/4229320905 shutdown_connections 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67edeeee0 0x55f67edef340 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5de0 0x55f67edee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.074+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4229320905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5430 0x55f67ede5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- 172.21.15.8:0/4229320905 >> 172.21.15.8:0/4229320905 conn(0x55f67eddfa10 msgr2=0x55f67eddfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- 172.21.15.8:0/4229320905 shutdown_connections 2024-02-19T16:26:31.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- 172.21.15.8:0/4229320905 wait complete. 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- start start 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 Processor -- start 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 0x55f67edf29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5de0 0x55f67edf2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67edeeee0 0x55f67edf3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ecf1f40 con 0x55f67ede5430 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ea204b0 con 0x55f67ede5de0 2024-02-19T16:26:31.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ecd2670 con 0x55f67edeeee0 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.075+0000 7f709d9d8740 10 client.0 _ll_register_callbacks cb 0x55f67ed3ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 0x55f67edf29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 0x55f67edf29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47068/0 (socket says 172.21.15.8:47068) 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 -- 172.21.15.8:0/2055173473 learned_addr learned my addr 172.21.15.8:0/2055173473 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7093fff640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5de0 0x55f67edf2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f709939f640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67edeeee0 0x55f67edf3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 -- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67edeeee0 msgr2=0x55f67edf3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67edeeee0 0x55f67edf3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 -- 172.21.15.8:0/2055173473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5de0 msgr2=0x55f67edf2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5de0 0x55f67edf2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 -- 172.21.15.8:0/2055173473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c019040 con 0x55f67ede5430 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7098b9e640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 0x55f67edf29e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7088009430 tx=0x7f7088009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60f7d59eda2018b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7073fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7073fff640 1 -- 172.21.15.8:0/2055173473 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880bca10 con 0x55f67ede5430 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7073fff640 1 -- 172.21.15.8:0/2055173473 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f70880c0b60 con 0x55f67ede5430 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f67ed26f40 con 0x55f67ede5430 2024-02-19T16:26:31.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f7073fff640 1 -- 172.21.15.8:0/2055173473 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70880c51b0 con 0x55f67ede5430 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f709d9d8740 10 client.16494 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.076+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f67ed4b320 con 0x55f67ede5430 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f7073fff640 1 -- 172.21.15.8:0/2055173473 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70880d4030 con 0x55f67ede5430 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f7073fff640 1 -- 172.21.15.8:0/2055173473 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f708803c070 con 0x55f67ede5430 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f709d9d8740 10 client.16494 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f709d9d8740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f709d9d8740 20 client.16494 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f709d9d8740 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee03220 0x55f67ee05660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f709d9d8740 4 client.16494 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f67ed43310 con 0x55f67ee03220 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f7093fff640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee03220 0x55f67ee05660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f7073fff640 10 client.16494 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.077+0000 7f7093fff640 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee03220 0x55f67ee05660 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9660542c42ed8028 server_cookie=85f2c0aef0c79487 in_seq=0 out_seq=0 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.078+0000 7f7073fff640 1 -- 172.21.15.8:0/2055173473 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f67ed43310 con 0x55f67ee03220 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.078+0000 7f7073fff640 10 client.16494 handle_command_reply: tid=0 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.078+0000 7f709d9d8740 1 client.16494 shutdown 2024-02-19T16:26:31.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 msgr2=0x55f67edf29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 0x55f67edf29e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7088009430 tx=0x7f7088009260 comp rx=0 tx=0).stop 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 shutdown_connections 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee03220 0x55f67ee05660 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67edeeee0 0x55f67edf3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67ede5de0 0x55f67edf2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 --2- 172.21.15.8:0/2055173473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5430 0x55f67edf29e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f7073fff640 0 client.16494 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 >> 172.21.15.8:0/2055173473 conn(0x55f67eddfa10 msgr2=0x55f67ede1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 shutdown_connections 2024-02-19T16:26:31.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 1 -- 172.21.15.8:0/2055173473 wait complete. 2024-02-19T16:26:31.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.079+0000 7f709d9d8740 20 client.16494 trim_cache size 0 max 16384 2024-02-19T16:26:31.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 Processor -- start 2024-02-19T16:26:31.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 -- start start 2024-02-19T16:26:31.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67eded340 con 0x55f67ede5de0 2024-02-19T16:26:31.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67edeed10 con 0x55f67edeeee0 2024-02-19T16:26:31.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709d9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ede4860 con 0x55f67ede5430 2024-02-19T16:26:31.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47078/0 (socket says 172.21.15.8:47078) 2024-02-19T16:26:31.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 -- 172.21.15.8:0/4222026847 learned_addr learned my addr 172.21.15.8:0/4222026847 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7098b9e640 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f709939f640 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 -- 172.21.15.8:0/4222026847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 msgr2=0x55f67edf50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 -- 172.21.15.8:0/4222026847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 msgr2=0x55f67edf02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.095+0000 7f7093fff640 1 -- 172.21.15.8:0/4222026847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f67ece7c00 con 0x55f67ede5de0 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f7093fff640 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf3280 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7080009710 tx=0x7f7080014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcffc56ec3fa5244 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f707e7fc640 1 -- 172.21.15.8:0/4222026847 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080025070 con 0x55f67ede5de0 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f707e7fc640 1 -- 172.21.15.8:0/4222026847 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7080005500 con 0x55f67ede5de0 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f707e7fc640 1 -- 172.21.15.8:0/4222026847 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708001e420 con 0x55f67ede5de0 2024-02-19T16:26:31.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 -- 172.21.15.8:0/4222026847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 msgr2=0x55f67edf3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf3280 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7080009710 tx=0x7f7080014000 comp rx=0 tx=0).stop 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 -- 172.21.15.8:0/4222026847 shutdown_connections 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf3280 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 --2- 172.21.15.8:0/4222026847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 -- 172.21.15.8:0/4222026847 >> 172.21.15.8:0/4222026847 conn(0x55f67eddfa10 msgr2=0x55f67edde400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 -- 172.21.15.8:0/4222026847 shutdown_connections 2024-02-19T16:26:31.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.096+0000 7f709d9d8740 1 -- 172.21.15.8:0/4222026847 wait complete. 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 -- start start 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 Processor -- start 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ee5eb00 con 0x55f67ede5de0 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ee5ec70 con 0x55f67edeeee0 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ee5ee30 con 0x55f67ede5430 2024-02-19T16:26:31.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709d9d8740 10 client.0 _ll_register_callbacks cb 0x7f70880bc7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f7093fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f7098b9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f7093fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47088/0 (socket says 172.21.15.8:47088) 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f7093fff640 1 -- 172.21.15.8:0/1619799672 learned_addr learned my addr 172.21.15.8:0/1619799672 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.097+0000 7f709939f640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7093fff640 1 -- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 msgr2=0x55f67edf7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7093fff640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7093fff640 1 -- 172.21.15.8:0/1619799672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 msgr2=0x55f67edf8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7093fff640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7093fff640 1 -- 172.21.15.8:0/1619799672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7080017070 con 0x55f67ede5de0 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7098b9e640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7093fff640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf8440 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7080009460 tx=0x7f7080027400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b712d3868db1c15b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7073fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7073fff640 1 -- 172.21.15.8:0/1619799672 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080025070 con 0x55f67ede5de0 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f67ed72900 con 0x55f67ede5de0 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7073fff640 1 -- 172.21.15.8:0/1619799672 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f7080004e60 con 0x55f67ede5de0 2024-02-19T16:26:31.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7073fff640 1 -- 172.21.15.8:0/1619799672 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7080005020 con 0x55f67ede5de0 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f709d9d8740 10 client.16500 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f67ed0d910 con 0x55f67ede5de0 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7073fff640 1 -- 172.21.15.8:0/1619799672 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f708003c020 con 0x55f67ede5de0 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f7073fff640 1 -- 172.21.15.8:0/1619799672 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f708001a040 con 0x55f67ede5de0 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f709d9d8740 10 client.16500 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f709d9d8740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.098+0000 7f709d9d8740 20 client.16500 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.099+0000 7f709d9d8740 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee60750 0x55f67ee62b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.099+0000 7f709d9d8740 4 client.16500 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:26:31.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.099+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f67ea4c860 con 0x55f67ee60750 2024-02-19T16:26:31.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.099+0000 7f7098b9e640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee60750 0x55f67ee62b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.099+0000 7f7098b9e640 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee60750 0x55f67ee62b40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f657e6ce309aa8c server_cookie=f90d0bbe2acbb22b in_seq=0 out_seq=0 2024-02-19T16:26:31.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.099+0000 7f7073fff640 10 client.16500 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f7073fff640 1 -- 172.21.15.8:0/1619799672 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f67ea4c860 con 0x55f67ee60750 2024-02-19T16:26:31.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f7073fff640 10 client.16500 handle_command_reply: tid=0 2024-02-19T16:26:31.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 client.16500 shutdown 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 msgr2=0x55f67edf8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf8440 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7080009460 tx=0x7f7080027400 comp rx=0 tx=0).stop 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 shutdown_connections 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f67edeeee0 0x55f67edf8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f67ede5de0 0x55f67edf8440 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f67ee60750 0x55f67ee62b40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 --2- 172.21.15.8:0/1619799672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f67ede5430 0x55f67edf7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 >> 172.21.15.8:0/1619799672 conn(0x55f67eddfa10 msgr2=0x55f67eddec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f7073fff640 0 client.16500 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 shutdown_connections 2024-02-19T16:26:31.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 1 -- 172.21.15.8:0/1619799672 wait complete. 2024-02-19T16:26:31.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.100+0000 7f709d9d8740 20 client.16500 trim_cache size 0 max 16384 2024-02-19T16:26:31.112 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:26:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:26:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "27e08184-6757-42a5-bd66-b0928660ea9a", 2024-02-19T16:26:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:26:31.113 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:26:31.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f70840a2230 msgr2=0x7f70840a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f70840a2230 0x7f70840a46f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70940fc2c0 tx=0x7f7080005f90 comp rx=0 tx=0).stop 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 msgr2=0x7f709419d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f709419d760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7088009230 tx=0x7f708802c650 comp rx=0 tx=0).stop 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 shutdown_connections 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7094197910 0x7f7094197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f70840a2230 0x7f70840a46f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7094105610 0x7f709419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 --2- 172.21.15.8:0/315513028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7094100540 0x7f709419d760 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 >> 172.21.15.8:0/315513028 conn(0x7f70940fbda0 msgr2=0x7f70940fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 shutdown_connections 2024-02-19T16:26:31.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.101+0000 7f7099ba0640 1 -- 172.21.15.8:0/315513028 wait complete. 2024-02-19T16:26:31.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:26:31.116 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:26:31.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 -- 172.21.15.8:0/1317787644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a0100f90 msgr2=0x7f35a0104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 --2- 172.21.15.8:0/1317787644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a0100f90 0x7f35a0104fd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3594009930 tx=0x7f359402bc30 comp rx=0 tx=0).stop 2024-02-19T16:26:31.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 -- 172.21.15.8:0/1317787644 shutdown_connections 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 --2- 172.21.15.8:0/1317787644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a01056b0 0x7f35a010d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 --2- 172.21.15.8:0/1317787644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a0100f90 0x7f35a0104fd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 --2- 172.21.15.8:0/1317787644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a01005e0 0x7f35a01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 -- 172.21.15.8:0/1317787644 >> 172.21.15.8:0/1317787644 conn(0x7f35a00775a0 msgr2=0x7f35a00779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 -- 172.21.15.8:0/1317787644 shutdown_connections 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.262+0000 7f35a482a640 1 -- 172.21.15.8:0/1317787644 wait complete. 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 Processor -- start 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 -- start start 2024-02-19T16:26:31.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a01005e0 0x7f35a0197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 0x7f35a01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a019c4a0 0x7f35a019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a019ce90 con 0x7f35a01056b0 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a019d000 con 0x7f35a019c4a0 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f35a482a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a019d1c0 con 0x7f35a01005e0 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 0x7f35a01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 0x7f35a01980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47116/0 (socket says 172.21.15.8:47116) 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 -- 172.21.15.8:0/3759769790 learned_addr learned my addr 172.21.15.8:0/3759769790 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359e59c640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a01005e0 0x7f35a0197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359ed9d640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a019c4a0 0x7f35a019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 -- 172.21.15.8:0/3759769790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a01005e0 msgr2=0x7f35a0197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a01005e0 0x7f35a0197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 -- 172.21.15.8:0/3759769790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a019c4a0 msgr2=0x7f35a019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a019c4a0 0x7f35a019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 -- 172.21.15.8:0/3759769790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594009560 con 0x7f35a01056b0 2024-02-19T16:26:31.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.263+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 0x7f35a01980f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3594009230 tx=0x7f3594004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.264+0000 7f358f7fe640 1 -- 172.21.15.8:0/3759769790 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940043d0 con 0x7f35a01056b0 2024-02-19T16:26:31.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.264+0000 7f358f7fe640 1 -- 172.21.15.8:0/3759769790 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35940052d0 con 0x7f35a01056b0 2024-02-19T16:26:31.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.264+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a01a4750 con 0x7f35a01056b0 2024-02-19T16:26:31.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.264+0000 7f358f7fe640 1 -- 172.21.15.8:0/3759769790 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594003d80 con 0x7f35a01056b0 2024-02-19T16:26:31.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.264+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a01a4bd0 con 0x7f35a01056b0 2024-02-19T16:26:31.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.266+0000 7f358f7fe640 1 -- 172.21.15.8:0/3759769790 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f3594005440 con 0x7f35a01056b0 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.267+0000 7f358f7fe640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f35880a2230 0x7f35880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.267+0000 7f35a7407740 1 Processor -- start 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.267+0000 7f358f7fe640 1 -- 172.21.15.8:0/3759769790 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f35940f0450 con 0x7f35a01056b0 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.267+0000 7f359e59c640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f35880a2230 0x7f35880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.267+0000 7f35a7407740 1 -- start start 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.267+0000 7f359e59c640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f35880a2230 0x7f35880a46f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359000b390 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f35a7407740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c68690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f35a7407740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 0x557a18c71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f35a7407740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f35a7407740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18bd7280 con 0x557a18c68bd0 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f35a7407740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18bd47b0 con 0x557a18c717b0 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f35a7407740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18bcff90 con 0x557a18c682b0 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 0x557a18c71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 0x557a18c71270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47130/0 (socket says 172.21.15.8:47130) 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 -- 172.21.15.8:0/3772910236 learned_addr learned my addr 172.21.15.8:0/3772910236 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359e59c640 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c68690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359ed9d640 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 -- 172.21.15.8:0/3772910236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 msgr2=0x557a18c68690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c68690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 -- 172.21.15.8:0/3772910236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 msgr2=0x557a18c73ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c73ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 -- 172.21.15.8:0/3772910236 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a18b8e570 con 0x557a18c68bd0 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 0x557a18c71270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f35940bcb30 tx=0x7f35940bc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6081480e0d3fa948 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.268+0000 7f358cff9640 1 -- 172.21.15.8:0/3772910236 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940bcba0 con 0x557a18c68bd0 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f358cff9640 1 -- 172.21.15.8:0/3772910236 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35940c0cf0 con 0x557a18c68bd0 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f358cff9640 1 -- 172.21.15.8:0/3772910236 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940eda70 con 0x557a18c68bd0 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 -- 172.21.15.8:0/3772910236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 msgr2=0x557a18c71270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 0x557a18c71270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f35940bcb30 tx=0x7f35940bc7a0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 -- 172.21.15.8:0/3772910236 shutdown_connections 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c68bd0 0x557a18c71270 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 --2- 172.21.15.8:0/3772910236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c68690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.269+0000 7f35a7407740 1 -- 172.21.15.8:0/3772910236 >> 172.21.15.8:0/3772910236 conn(0x557a18c62990 msgr2=0x557a18c62d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 -- 172.21.15.8:0/3772910236 shutdown_connections 2024-02-19T16:26:31.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 -- 172.21.15.8:0/3772910236 wait complete. 2024-02-19T16:26:31.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 -- start start 2024-02-19T16:26:31.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 Processor -- start 2024-02-19T16:26:31.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 0x557a18c6e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18c6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a188a3430 con 0x557a18c717b0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1885fe00 con 0x557a18c682b0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18b54fd0 con 0x557a18c68bd0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f35a7407740 10 client.0 _ll_register_callbacks cb 0x557a18baafd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f359e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 0x557a18c6e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f359e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 0x557a18c6e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:34970/0 (socket says 172.21.15.8:34970) 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f359ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18c6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.270+0000 7f359e59c640 1 -- 172.21.15.8:0/3094258243 learned_addr learned my addr 172.21.15.8:0/3094258243 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359ed9d640 1 -- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 msgr2=0x557a18c6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359ed9d640 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359ed9d640 1 -- 172.21.15.8:0/3094258243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 msgr2=0x557a18c6e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359ed9d640 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 0x557a18c6e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359ed9d640 1 -- 172.21.15.8:0/3094258243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a01a4bd0 con 0x557a18c717b0 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359e59c640 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 0x557a18c6e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f359ed9d640 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18c6f0a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3598005fd0 tx=0x7f359800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5074ae8a770caca9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f3585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:31.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f3585ffb640 1 -- 172.21.15.8:0/3094258243 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598016070 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f3585ffb640 1 -- 172.21.15.8:0/3094258243 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f3598009e30 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a188cf7e0 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f3585ffb640 1 -- 172.21.15.8:0/3094258243 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3598004b20 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f35a7407740 10 client.16512 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a18bce610 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.271+0000 7f3585ffb640 1 -- 172.21.15.8:0/3094258243 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a188cf7e0 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f3585ffb640 1 -- 172.21.15.8:0/3094258243 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f359801b470 con 0x557a18c717b0 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f35a7407740 10 client.16512 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:31.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f35a7407740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f35a7407740 20 client.16512 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f35a7407740 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18c817e0 0x557a18c83c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f35a7407740 4 client.16512 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a1890c720 con 0x557a18c817e0 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.272+0000 7f359e59c640 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18c817e0 0x557a18c83c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.273+0000 7f3585ffb640 10 client.16512 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.273+0000 7f359e59c640 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18c817e0 0x557a18c83c00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ff73f5882f75ea server_cookie=8541d2eab3f74b84 in_seq=0 out_seq=0 2024-02-19T16:26:31.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f3585ffb640 1 -- 172.21.15.8:0/3094258243 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557a1890c720 con 0x557a18c817e0 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f3585ffb640 10 client.16512 handle_command_reply: tid=0 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 client.16512 shutdown 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 msgr2=0x557a18c6f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18c6f0a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3598005fd0 tx=0x7f359800f790 comp rx=0 tx=0).stop 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 shutdown_connections 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18c6f0a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18c817e0 0x557a18c83c00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 --2- 172.21.15.8:0/3094258243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c682b0 0x557a18c6e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f3585ffb640 0 client.16512 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 >> 172.21.15.8:0/3094258243 conn(0x557a18c62990 msgr2=0x557a18c64730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 shutdown_connections 2024-02-19T16:26:31.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 1 -- 172.21.15.8:0/3094258243 wait complete. 2024-02-19T16:26:31.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.274+0000 7f35a7407740 20 client.16512 trim_cache size 0 max 16384 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 Processor -- start 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 -- start start 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 0x557a18c70d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18c81560 con 0x557a18c682b0 2024-02-19T16:26:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18c639e0 con 0x557a18c717b0 2024-02-19T16:26:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f35a7407740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18c63770 con 0x557a18c68bd0 2024-02-19T16:26:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 0x557a18c70d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 0x557a18c70d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47154/0 (socket says 172.21.15.8:47154) 2024-02-19T16:26:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359dd9b640 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 -- 172.21.15.8:0/3461335275 learned_addr learned my addr 172.21.15.8:0/3461335275 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359ed9d640 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 -- 172.21.15.8:0/3461335275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 msgr2=0x557a18c6eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 -- 172.21.15.8:0/3461335275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 msgr2=0x557a18c618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.293+0000 7f359e59c640 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.294+0000 7f359e59c640 1 -- 172.21.15.8:0/3461335275 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a18b6aac0 con 0x557a18c682b0 2024-02-19T16:26:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.294+0000 7f359e59c640 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 0x557a18c70d50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x557a18c84440 tx=0x7f3590013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e418c2049b1424 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.294+0000 7f3587fff640 1 -- 172.21.15.8:0/3461335275 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359002e070 con 0x557a18c682b0 2024-02-19T16:26:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.294+0000 7f3587fff640 1 -- 172.21.15.8:0/3461335275 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3590015070 con 0x557a18c682b0 2024-02-19T16:26:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.294+0000 7f3587fff640 1 -- 172.21.15.8:0/3461335275 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590024040 con 0x557a18c682b0 2024-02-19T16:26:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- 172.21.15.8:0/3461335275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 msgr2=0x557a18c70d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 0x557a18c70d50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x557a18c84440 tx=0x7f3590013000 comp rx=0 tx=0).stop 2024-02-19T16:26:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- 172.21.15.8:0/3461335275 shutdown_connections 2024-02-19T16:26:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c717b0 0x557a18c618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c68bd0 0x557a18c6eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- 172.21.15.8:0/3461335275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c682b0 0x557a18c70d50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- 172.21.15.8:0/3461335275 >> 172.21.15.8:0/3461335275 conn(0x557a18c62990 msgr2=0x557a18c611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- 172.21.15.8:0/3461335275 shutdown_connections 2024-02-19T16:26:31.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- 172.21.15.8:0/3461335275 wait complete. 2024-02-19T16:26:31.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- start start 2024-02-19T16:26:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 Processor -- start 2024-02-19T16:26:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 0x557a18c755f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18cd60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18cd6610 con 0x557a18c717b0 2024-02-19T16:26:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18cd6780 con 0x557a18c68bd0 2024-02-19T16:26:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a18cd6940 con 0x557a18c682b0 2024-02-19T16:26:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f35a7407740 10 client.0 _ll_register_callbacks cb 0x557a18c71700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:26:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f359ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18cd60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 0x557a18c755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f359ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18cd60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47160/0 (socket says 172.21.15.8:47160) 2024-02-19T16:26:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f359dd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 0x557a18c755f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:34986/0 (socket says 172.21.15.8:34986) 2024-02-19T16:26:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f359ed9d640 1 -- 172.21.15.8:0/911806475 learned_addr learned my addr 172.21.15.8:0/911806475 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:26:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.295+0000 7f359e59c640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359ed9d640 1 -- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 msgr2=0x557a18c750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359ed9d640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c750b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359ed9d640 1 -- 172.21.15.8:0/911806475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 msgr2=0x557a18c755f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:26:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359ed9d640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 0x557a18c755f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359ed9d640 1 -- 172.21.15.8:0/911806475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3590003ae0 con 0x557a18c717b0 2024-02-19T16:26:31.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359dd9b640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 0x557a18c755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:26:31.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f359ed9d640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18cd60d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3598000f80 tx=0x7f3598005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d058ee2fbb9a8e6e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:26:31.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f3585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:26:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f3585ffb640 1 -- 172.21.15.8:0/911806475 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598016040 con 0x557a18c717b0 2024-02-19T16:26:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f3585ffb640 1 -- 172.21.15.8:0/911806475 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f359801b9f0 con 0x557a18c717b0 2024-02-19T16:26:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35940c0cf0 con 0x557a18c717b0 2024-02-19T16:26:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f3585ffb640 1 -- 172.21.15.8:0/911806475 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f359801bbd0 con 0x557a18c717b0 2024-02-19T16:26:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f35a7407740 10 client.16518 fetch_fsmap learned FSMap version 28 2024-02-19T16:26:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.296+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a18b906a0 con 0x557a18c717b0 2024-02-19T16:26:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f3585ffb640 1 -- 172.21.15.8:0/911806475 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35940c0cf0 con 0x557a18c717b0 2024-02-19T16:26:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f3585ffb640 1 -- 172.21.15.8:0/911806475 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f35980055a0 con 0x557a18c717b0 2024-02-19T16:26:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f35a7407740 10 client.16518 fetch_fsmap finished waiting for FSMap version 28 2024-02-19T16:26:31.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f35a7407740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:26:31.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f35a7407740 20 client.16518 populate_metadata read hostname 'smithi008' 2024-02-19T16:26:31.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f35a7407740 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18cddff0 0x557a18ce0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:26:31.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f35a7407740 4 client.16518 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:26:31.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a1895b890 con 0x557a18cddff0 2024-02-19T16:26:31.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.297+0000 7f359e59c640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18cddff0 0x557a18ce0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:26:31.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f3585ffb640 10 client.16518 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f359e59c640 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18cddff0 0x557a18ce0410 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6397fb0c8f67a584 server_cookie=9b5649fa8ecefcca in_seq=0 out_seq=0 2024-02-19T16:26:31.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f3585ffb640 1 -- 172.21.15.8:0/911806475 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x557a1895b890 con 0x557a18cddff0 2024-02-19T16:26:31.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f3585ffb640 10 client.16518 handle_command_reply: tid=0 2024-02-19T16:26:31.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 client.16518 shutdown 2024-02-19T16:26:31.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 msgr2=0x557a18cd60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18cd60d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3598000f80 tx=0x7f3598005d00 comp rx=0 tx=0).stop 2024-02-19T16:26:31.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 shutdown_connections 2024-02-19T16:26:31.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557a18c717b0 0x557a18cd60d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a18c68bd0 0x557a18c755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557a18cddff0 0x557a18ce0410 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 --2- 172.21.15.8:0/911806475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557a18c682b0 0x557a18c750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f3585ffb640 0 client.16518 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:26:31.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 >> 172.21.15.8:0/911806475 conn(0x557a18c62990 msgr2=0x557a18c63b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 shutdown_connections 2024-02-19T16:26:31.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 1 -- 172.21.15.8:0/911806475 wait complete. 2024-02-19T16:26:31.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.298+0000 7f35a7407740 20 client.16518 trim_cache size 0 max 16384 2024-02-19T16:26:31.318 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:26:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2490 inodes in the stack)", 2024-02-19T16:26:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:26:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "27e08184-6757-42a5-bd66-b0928660ea9a": { 2024-02-19T16:26:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:26:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "27e08184-6757-42a5-bd66-b0928660ea9a", 2024-02-19T16:26:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:26:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:26:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:26:31.321 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:26:31.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f35880a2230 msgr2=0x7f35880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f35880a2230 0x7f35880a46f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359000b390 comp rx=0 tx=0).stop 2024-02-19T16:26:31.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 msgr2=0x7f35a01980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 0x7f35a01980f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3594009230 tx=0x7f3594004c00 comp rx=0 tx=0).stop 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 shutdown_connections 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a019c4a0 0x7f35a019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35a01056b0 0x7f35a01980f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f35880a2230 0x7f35880a46f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 --2- 172.21.15.8:0/3759769790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a01005e0 0x7f35a0197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:26:31.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 >> 172.21.15.8:0/3759769790 conn(0x7f35a00775a0 msgr2=0x7f35a0068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:26:31.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 shutdown_connections 2024-02-19T16:26:31.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:26:31.299+0000 7f35a482a640 1 -- 172.21.15.8:0/3759769790 wait complete. 2024-02-19T16:26:31.325 INFO:tasks.cephfs.filesystem:scrub status for tag:27e08184-6757-42a5-bd66-b0928660ea9a - {'path': '/', 'tag': '27e08184-6757-42a5-bd66-b0928660ea9a', 'options': 'recursive,force'} 2024-02-19T16:26:32.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: pgmap v112: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:32.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:32.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:32.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:32.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:31 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: pgmap v112: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:32.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:32.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:32.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:32.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:31 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: pgmap v112: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:31 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:32 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:26:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:32 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:32 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:32 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:26:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:32 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:32 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:32 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:26:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:32 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:32 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:32 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:26:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:32 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:32 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:32 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:26:33.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:32 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:32 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:33.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:32 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:26:33.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:32 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:33.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:32 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:26:34.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:33 smithi008 ceph-mon[32573]: pgmap v113: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:34.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:33 smithi191 ceph-mon[37203]: pgmap v113: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:34.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:33 smithi156 ceph-mon[36502]: pgmap v113: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:36.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:35 smithi008 ceph-mon[32573]: pgmap v114: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-19T16:26:36.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:35 smithi191 ceph-mon[37203]: pgmap v114: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-19T16:26:36.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:35 smithi156 ceph-mon[36502]: pgmap v114: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-19T16:26:38.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:37 smithi008 ceph-mon[32573]: pgmap v115: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:38.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:37 smithi191 ceph-mon[37203]: pgmap v115: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:38.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:37 smithi156 ceph-mon[36502]: pgmap v115: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-19T16:26:40.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:39 smithi008 ceph-mon[32573]: pgmap v116: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-19T16:26:40.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:39 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:26:40.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:39 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:26:40.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:39 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:26:40.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:39 smithi191 ceph-mon[37203]: pgmap v116: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-19T16:26:40.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:39 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:26:40.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:39 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:26:40.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:39 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:26:40.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:39 smithi156 ceph-mon[36502]: pgmap v116: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-02-19T16:26:40.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:39 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:26:40.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:39 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:26:40.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:39 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:26:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:41 smithi008 ceph-mon[32573]: pgmap v117: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:26:42.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:41 smithi191 ceph-mon[37203]: pgmap v117: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:26:42.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:41 smithi156 ceph-mon[36502]: pgmap v117: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:26:44.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:43 smithi008 ceph-mon[32573]: pgmap v118: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:26:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:43 smithi191 ceph-mon[37203]: pgmap v118: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:26:44.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:43 smithi156 ceph-mon[36502]: pgmap v118: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:26:46.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:45 smithi008 ceph-mon[32573]: pgmap v119: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:26:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:45 smithi191 ceph-mon[37203]: pgmap v119: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:26:46.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:45 smithi156 ceph-mon[36502]: pgmap v119: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:26:48.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:47 smithi008 ceph-mon[32573]: pgmap v120: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:48.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:47 smithi191 ceph-mon[37203]: pgmap v120: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:48.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:47 smithi156 ceph-mon[36502]: pgmap v120: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:48 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:26:49.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:48 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:26:49.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:48 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:26:50.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:49 smithi008 ceph-mon[32573]: pgmap v121: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:26:50.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:49 smithi191 ceph-mon[37203]: pgmap v121: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:26:50.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:49 smithi156 ceph-mon[36502]: pgmap v121: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:26:52.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:51 smithi008 ceph-mon[32573]: pgmap v122: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:52.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:51 smithi191 ceph-mon[37203]: pgmap v122: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:52.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:51 smithi156 ceph-mon[36502]: pgmap v122: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:54.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:53 smithi008 ceph-mon[32573]: pgmap v123: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:54.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:53 smithi191 ceph-mon[37203]: pgmap v123: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:54.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:53 smithi156 ceph-mon[36502]: pgmap v123: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:56.024 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:55 smithi008 ceph-mon[32573]: pgmap v124: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:26:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:55 smithi191 ceph-mon[37203]: pgmap v124: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:26:56.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:55 smithi156 ceph-mon[36502]: pgmap v124: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:26:58.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:57 smithi008 ceph-mon[32573]: pgmap v125: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:57 smithi191 ceph-mon[37203]: pgmap v125: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:26:58.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:57 smithi156 ceph-mon[36502]: pgmap v125: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:00.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:26:59 smithi008 ceph-mon[32573]: pgmap v126: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:26:59 smithi191 ceph-mon[37203]: pgmap v126: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:00.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:26:59 smithi156 ceph-mon[36502]: pgmap v126: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:01.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:00 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:27:01.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:00 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:27:01.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:00 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:27:01.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:27:01.327 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:27:01.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3463088625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101460 msgr2=0x7fa5741018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3463088625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101460 0x7fa5741018e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa568009930 tx=0x7fa56802bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:01.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3463088625 shutdown_connections 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3463088625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa574101e20 0x7fa57410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3463088625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101460 0x7fa5741018e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3463088625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574107500 0x7fa5741078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3463088625 >> 172.21.15.8:0/3463088625 conn(0x7fa5740fcf90 msgr2=0x7fa5740ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3463088625 shutdown_connections 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3463088625 wait complete. 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 Processor -- start 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.431+0000 7fa57a9c9640 1 -- start start 2024-02-19T16:27:01.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574101460 0x7fa57419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 0x7fa57419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa574107500 0x7fa574197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574197f80 con 0x7fa574101e20 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741980f0 con 0x7fa574101460 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741982b0 con 0x7fa574107500 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 0x7fa57419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 0x7fa57419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44440/0 (socket says 172.21.15.8:44440) 2024-02-19T16:27:01.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 -- 172.21.15.8:0/3805059619 learned_addr learned my addr 172.21.15.8:0/3805059619 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa578f66640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa574107500 0x7fa574197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa573fff640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574101460 0x7fa57419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 -- 172.21.15.8:0/3805059619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa574107500 msgr2=0x7fa574197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa574107500 0x7fa574197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 -- 172.21.15.8:0/3805059619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574101460 msgr2=0x7fa57419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574101460 0x7fa57419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 -- 172.21.15.8:0/3805059619 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009560 con 0x7fa574101e20 2024-02-19T16:27:01.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5737fe640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 0x7fa57419dcc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa568009230 tx=0x7fa56802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5717fa640 1 -- 172.21.15.8:0/3805059619 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568004350 con 0x7fa574101e20 2024-02-19T16:27:01.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5717fa640 1 -- 172.21.15.8:0/3805059619 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5680044b0 con 0x7fa574101e20 2024-02-19T16:27:01.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa574198530 con 0x7fa574101e20 2024-02-19T16:27:01.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.432+0000 7fa5717fa640 1 -- 172.21.15.8:0/3805059619 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568003d00 con 0x7fa574101e20 2024-02-19T16:27:01.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.433+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5741a4820 con 0x7fa574101e20 2024-02-19T16:27:01.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.435+0000 7fa5717fa640 1 -- 172.21.15.8:0/3805059619 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fa568004df0 con 0x7fa574101e20 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.435+0000 7fa57d5a6740 1 Processor -- start 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.435+0000 7fa57d5a6740 1 -- start start 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.435+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bece2b0 0x56368bece690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.435+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368becebd0 0x56368bed7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.435+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bed9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368be3d280 con 0x56368bed77b0 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368be3a7b0 con 0x56368becebd0 2024-02-19T16:27:01.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368be35f90 con 0x56368bece2b0 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa5717fa640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa55c0a2230 0x7fa55c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bed9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368becebd0 0x56368bed7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bed9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44444/0 (socket says 172.21.15.8:44444) 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa5737fe640 1 -- 172.21.15.8:0/1610129709 learned_addr learned my addr 172.21.15.8:0/1610129709 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa578f66640 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bece2b0 0x56368bece690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa578f66640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa55c0a2230 0x7fa55c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 -- 172.21.15.8:0/1610129709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bece2b0 msgr2=0x56368bece690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bece2b0 0x56368bece690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 -- 172.21.15.8:0/1610129709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 msgr2=0x56368bed9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa5717fa640 1 -- 172.21.15.8:0/3805059619 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa5680f0310 con 0x7fa574101e20 2024-02-19T16:27:01.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bed9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 -- 172.21.15.8:0/1610129709 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368bdf4570 con 0x56368becebd0 2024-02-19T16:27:01.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa573fff640 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368becebd0 0x56368bed7270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa564009bf0 tx=0x7fa564005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c4ae70121e1147 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa578f66640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa55c0a2230 0x7fa55c0a46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa56c005c40 tx=0x7fa56c009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa55a7fc640 1 -- 172.21.15.8:0/1610129709 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011e90 con 0x56368becebd0 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa55a7fc640 1 -- 172.21.15.8:0/1610129709 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa564005020 con 0x56368becebd0 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.436+0000 7fa55a7fc640 1 -- 172.21.15.8:0/1610129709 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564004b20 con 0x56368becebd0 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1610129709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368becebd0 msgr2=0x56368bed7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368becebd0 0x56368bed7270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa564009bf0 tx=0x7fa564005850 comp rx=0 tx=0).stop 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1610129709 shutdown_connections 2024-02-19T16:27:01.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bed9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368becebd0 0x56368bed7270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1610129709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bece2b0 0x56368bece690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1610129709 >> 172.21.15.8:0/1610129709 conn(0x56368bec8990 msgr2=0x56368bec8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1610129709 shutdown_connections 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1610129709 wait complete. 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.437+0000 7fa57d5a6740 1 -- start start 2024-02-19T16:27:01.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 Processor -- start 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bb09430 con 0x56368becebd0 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bac5e00 con 0x56368bece2b0 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bdbafd0 con 0x56368bed77b0 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa57d5a6740 10 client.0 _ll_register_callbacks cb 0x56368be10fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:01.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44460/0 (socket says 172.21.15.8:44460) 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 -- 172.21.15.8:0/2075039539 learned_addr learned my addr 172.21.15.8:0/2075039539 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa573fff640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa578f66640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 -- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 msgr2=0x56368bed4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 -- 172.21.15.8:0/2075039539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 msgr2=0x56368bed7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 -- 172.21.15.8:0/2075039539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa564009850 con 0x56368becebd0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5737fe640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed44d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa5680b8ec0 tx=0x7fa56802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afa560666628eb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.438+0000 7fa5537fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa5537fe640 1 -- 172.21.15.8:0/2075039539 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680bd160 con 0x56368becebd0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa5537fe640 1 -- 172.21.15.8:0/2075039539 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa5680bca60 con 0x56368becebd0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56368bb357e0 con 0x56368becebd0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa5537fe640 1 -- 172.21.15.8:0/2075039539 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5680c0bb0 con 0x56368becebd0 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 10 client.16530 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:01.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56368be34610 con 0x56368becebd0 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa5537fe640 1 -- 172.21.15.8:0/2075039539 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56368bb357e0 con 0x56368becebd0 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa5537fe640 1 -- 172.21.15.8:0/2075039539 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa56803c070 con 0x56368becebd0 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 10 client.16530 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 20 client.16530 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bee76b0 0x56368bee9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 4 client.16530 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56368bb72720 con 0x56368bee76b0 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.439+0000 7fa573fff640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bee76b0 0x56368bee9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa5537fe640 10 client.16530 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa573fff640 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bee76b0 0x56368bee9ad0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef31c52b9c7cc1d server_cookie=7fccfeead7b5bc44 in_seq=0 out_seq=0 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa5537fe640 1 -- 172.21.15.8:0/2075039539 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56368bb72720 con 0x56368bee76b0 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa5537fe640 10 client.16530 handle_command_reply: tid=0 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 client.16530 shutdown 2024-02-19T16:27:01.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 msgr2=0x56368bed44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed44d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa5680b8ec0 tx=0x7fa56802bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 shutdown_connections 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bee76b0 0x56368bee9ad0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed44d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa5537fe640 0 client.16530 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/2075039539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.440+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 >> 172.21.15.8:0/2075039539 conn(0x56368bec8990 msgr2=0x56368beca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.441+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 shutdown_connections 2024-02-19T16:27:01.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.441+0000 7fa57d5a6740 1 -- 172.21.15.8:0/2075039539 wait complete. 2024-02-19T16:27:01.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.441+0000 7fa57d5a6740 20 client.16530 trim_cache size 0 max 16384 2024-02-19T16:27:01.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.456+0000 7fa57d5a6740 1 Processor -- start 2024-02-19T16:27:01.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 -- start start 2024-02-19T16:27:01.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bec9790 con 0x56368becebd0 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368beea130 con 0x56368bece2b0 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bec2e20 con 0x56368bed77b0 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44464/0 (socket says 172.21.15.8:44464) 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa573fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 -- 172.21.15.8:0/1932512561 learned_addr learned my addr 172.21.15.8:0/1932512561 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa578f66640 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 -- 172.21.15.8:0/1932512561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 msgr2=0x56368bed7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 -- 172.21.15.8:0/1932512561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 msgr2=0x56368bed4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 -- 172.21.15.8:0/1932512561 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368bdd0ac0 con 0x56368becebd0 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa573fff640 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa5737fe640 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed6c00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa5680b9430 tx=0x7fa56804afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5749b665f681a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa559ffb640 1 -- 172.21.15.8:0/1932512561 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680c4d70 con 0x56368becebd0 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa559ffb640 1 -- 172.21.15.8:0/1932512561 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5680bc830 con 0x56368becebd0 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.457+0000 7fa559ffb640 1 -- 172.21.15.8:0/1932512561 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680c0bb0 con 0x56368becebd0 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1932512561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 msgr2=0x56368bed6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed6c00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa5680b9430 tx=0x7fa56804afb0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1932512561 shutdown_connections 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368bed77b0 0x56368bed7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368becebd0 0x56368bed6c00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1932512561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1932512561 >> 172.21.15.8:0/1932512561 conn(0x56368bec8990 msgr2=0x56368bed3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1932512561 shutdown_connections 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1932512561 wait complete. 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.458+0000 7fa57d5a6740 1 -- start start 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 Processor -- start 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368becebd0 0x56368bed8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bedc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bed9220 con 0x56368bed77b0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bedcb90 con 0x56368bece2b0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368bedcd20 con 0x56368becebd0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa57d5a6740 10 client.0 _ll_register_callbacks cb 0x7fa568009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bedc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa573fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bedc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44480/0 (socket says 172.21.15.8:44480) 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa5737fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368becebd0 0x56368bed8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 -- 172.21.15.8:0/1598373340 learned_addr learned my addr 172.21.15.8:0/1598373340 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 -- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368becebd0 msgr2=0x56368bed8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368becebd0 0x56368bed8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 -- 172.21.15.8:0/1598373340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 msgr2=0x56368bed85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 -- 172.21.15.8:0/1598373340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56401d040 con 0x56368bed77b0 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa573fff640 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa578f66640 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bedc650 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa56c003e90 tx=0x7fa56c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42a2a0ea7b859af server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.459+0000 7fa5537fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:01.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa5537fe640 1 -- 172.21.15.8:0/1598373340 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c025070 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa5537fe640 1 -- 172.21.15.8:0/1598373340 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa56c004b20 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa564005020 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa5537fe640 1 -- 172.21.15.8:0/1598373340 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56c004ce0 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 10 client.16536 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56368be34610 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa5537fe640 1 -- 172.21.15.8:0/1598373340 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa564005020 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa5537fe640 1 -- 172.21.15.8:0/1598373340 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa56c011d20 con 0x56368bed77b0 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 10 client.16536 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:01.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 20 client.16536 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bf43ef0 0x56368bf46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 4 client.16536 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56368bbc1890 con 0x56368bf43ef0 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.460+0000 7fa573fff640 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bf43ef0 0x56368bf46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa5537fe640 10 client.16536 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa573fff640 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bf43ef0 0x56368bf46310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78f6147ee3cc04a server_cookie=3c634c2b44af0851 in_seq=0 out_seq=0 2024-02-19T16:27:01.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa5537fe640 1 -- 172.21.15.8:0/1598373340 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56368bbc1890 con 0x56368bf43ef0 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa5537fe640 10 client.16536 handle_command_reply: tid=0 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 client.16536 shutdown 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 msgr2=0x56368bedc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bedc650 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa56c003e90 tx=0x7fa56c004a40 comp rx=0 tx=0).stop 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 shutdown_connections 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56368bed77b0 0x56368bedc650 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56368bf43ef0 0x56368bf46310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56368becebd0 0x56368bed8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 --2- 172.21.15.8:0/1598373340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56368bece2b0 0x56368bed85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa5537fe640 0 client.16536 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:01.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 >> 172.21.15.8:0/1598373340 conn(0x56368bec8990 msgr2=0x56368bec71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 shutdown_connections 2024-02-19T16:27:01.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 1 -- 172.21.15.8:0/1598373340 wait complete. 2024-02-19T16:27:01.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.461+0000 7fa57d5a6740 20 client.16536 trim_cache size 0 max 16384 2024-02-19T16:27:01.474 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:27:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:27:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:27:01.474 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:27:01.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa55c0a2230 msgr2=0x7fa55c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa55c0a2230 0x7fa55c0a46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa56c005c40 tx=0x7fa56c009500 comp rx=0 tx=0).stop 2024-02-19T16:27:01.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 msgr2=0x7fa57419dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 0x7fa57419dcc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa568009230 tx=0x7fa56802c650 comp rx=0 tx=0).stop 2024-02-19T16:27:01.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 shutdown_connections 2024-02-19T16:27:01.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa55c0a2230 0x7fa55c0a46f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa574107500 0x7fa574197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa574101e20 0x7fa57419dcc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 --2- 172.21.15.8:0/3805059619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574101460 0x7fa57419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 >> 172.21.15.8:0/3805059619 conn(0x7fa5740fcf90 msgr2=0x7fa5740fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 shutdown_connections 2024-02-19T16:27:01.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.462+0000 7fa57a9c9640 1 -- 172.21.15.8:0/3805059619 wait complete. 2024-02-19T16:27:01.477 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:27:01.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:27:01.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 -- 172.21.15.8:0/2595067036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb8102760 msgr2=0x7f8bb8102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 --2- 172.21.15.8:0/2595067036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb8102760 0x7f8bb8102b40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009900 tx=0x7f8bb002bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:01.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 -- 172.21.15.8:0/2595067036 shutdown_connections 2024-02-19T16:27:01.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 --2- 172.21.15.8:0/2595067036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb810bb30 0x7f8bb810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 --2- 172.21.15.8:0/2595067036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8103080 0x7f8bb810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 --2- 172.21.15.8:0/2595067036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb8102760 0x7f8bb8102b40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.580+0000 7f8bbf839640 1 -- 172.21.15.8:0/2595067036 >> 172.21.15.8:0/2595067036 conn(0x7f8bb80774f0 msgr2=0x7f8bb8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 -- 172.21.15.8:0/2595067036 shutdown_connections 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 -- 172.21.15.8:0/2595067036 wait complete. 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 Processor -- start 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 -- start start 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb8102760 0x7f8bb819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8103080 0x7f8bb819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 0x7f8bb8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8197fe0 con 0x7f8bb810bb30 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8198150 con 0x7f8bb8102760 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbf839640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8198310 con 0x7f8bb8103080 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbddd6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 0x7f8bb8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbd5d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb8102760 0x7f8bb819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbcdd4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8103080 0x7f8bb819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbddd6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 0x7f8bb8197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44504/0 (socket says 172.21.15.8:44504) 2024-02-19T16:27:01.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.581+0000 7f8bbddd6640 1 -- 172.21.15.8:0/1165721450 learned_addr learned my addr 172.21.15.8:0/1165721450 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbddd6640 1 -- 172.21.15.8:0/1165721450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8103080 msgr2=0x7f8bb819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbddd6640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8103080 0x7f8bb819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbddd6640 1 -- 172.21.15.8:0/1165721450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb8102760 msgr2=0x7f8bb819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbddd6640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb8102760 0x7f8bb819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbddd6640 1 -- 172.21.15.8:0/1165721450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8009840 con 0x7f8bb810bb30 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbd5d5640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb8102760 0x7f8bb819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbddd6640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 0x7f8bb8197990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800efd0 tx=0x7f8ba800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8009b40 con 0x7f8bb810bb30 2024-02-19T16:27:01.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ba8003b70 con 0x7f8bb810bb30 2024-02-19T16:27:01.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb0009560 con 0x7f8bb810bb30 2024-02-19T16:27:01.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80108f0 con 0x7f8bb810bb30 2024-02-19T16:27:01.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.582+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb81a46a0 con 0x7f8bb810bb30 2024-02-19T16:27:01.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.583+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb80682e0 con 0x7f8bb810bb30 2024-02-19T16:27:01.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.584+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8ba8003ce0 con 0x7f8bb810bb30 2024-02-19T16:27:01.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.585+0000 7f8bb67fc640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8ba00a2230 0x7f8ba00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.585+0000 7f8bbd5d5640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8ba00a2230 0x7f8ba00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.585+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8ba8014070 con 0x7f8bb810bb30 2024-02-19T16:27:01.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.585+0000 7f8bbd5d5640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8ba00a2230 0x7f8ba00a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009230 tx=0x7f8bb0005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.589+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f8ba80972f0 con 0x7f8bb810bb30 2024-02-19T16:27:01.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.813+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bb8198ec0 con 0x7f8bb810bb30 2024-02-19T16:27:01.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.814+0000 7f8bb67fc640 1 -- 172.21.15.8:0/1165721450 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f8ba809a9b0 con 0x7f8bb810bb30 2024-02-19T16:27:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:27:01.815 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:26:47.509116+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:27:01.816 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 29 2024-02-19T16:27:01.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8ba00a2230 msgr2=0x7f8ba00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8ba00a2230 0x7f8ba00a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009230 tx=0x7f8bb0005fd0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 msgr2=0x7f8bb8197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 0x7f8bb8197990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800efd0 tx=0x7f8ba800c6a0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 shutdown_connections 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8bb810bb30 0x7f8bb8197990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8ba00a2230 0x7f8ba00a46f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8103080 0x7f8bb819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 --2- 172.21.15.8:0/1165721450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bb8102760 0x7f8bb819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 >> 172.21.15.8:0/1165721450 conn(0x7f8bb80774f0 msgr2=0x7f8bb80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 shutdown_connections 2024-02-19T16:27:01.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.816+0000 7f8bbf839640 1 -- 172.21.15.8:0/1165721450 wait complete. 2024-02-19T16:27:01.828 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:27:01.828 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:27:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 -- 172.21.15.8:0/2375060337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c102760 msgr2=0x7fa45c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 --2- 172.21.15.8:0/2375060337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c102760 0x7fa45c102b40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa458009900 tx=0x7fa45802bcd0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 -- 172.21.15.8:0/2375060337 shutdown_connections 2024-02-19T16:27:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 --2- 172.21.15.8:0/2375060337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa45c10bb30 0x7fa45c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 --2- 172.21.15.8:0/2375060337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa45c103080 0x7fa45c10b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 --2- 172.21.15.8:0/2375060337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c102760 0x7fa45c102b40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 -- 172.21.15.8:0/2375060337 >> 172.21.15.8:0/2375060337 conn(0x7fa45c077550 msgr2=0x7fa45c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 -- 172.21.15.8:0/2375060337 shutdown_connections 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.932+0000 7fa4636c3640 1 -- 172.21.15.8:0/2375060337 wait complete. 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 Processor -- start 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 -- start start 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa45c102760 0x7fa45c1003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa45c103080 0x7fa45c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 0x7fa45c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1a5f70 con 0x7fa45c10bb30 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1a60e0 con 0x7fa45c103080 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa4636c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1a62a0 con 0x7fa45c102760 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 0x7fa45c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 0x7fa45c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44514/0 (socket says 172.21.15.8:44514) 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 -- 172.21.15.8:0/943168544 learned_addr learned my addr 172.21.15.8:0/943168544 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa46145f640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa45c102760 0x7fa45c1003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa460c5e640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa45c103080 0x7fa45c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 -- 172.21.15.8:0/943168544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa45c102760 msgr2=0x7fa45c1003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa45c102760 0x7fa45c1003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 -- 172.21.15.8:0/943168544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa45c103080 msgr2=0x7fa45c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa45c103080 0x7fa45c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.933+0000 7fa461c60640 1 -- 172.21.15.8:0/943168544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009560 con 0x7fa45c10bb30 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.934+0000 7fa461c60640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 0x7fa45c100e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa45400eb10 tx=0x7fa45400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.934+0000 7fa4527fc640 1 -- 172.21.15.8:0/943168544 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454004b20 con 0x7fa45c10bb30 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.934+0000 7fa4527fc640 1 -- 172.21.15.8:0/943168544 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa454004c80 con 0x7fa45c10bb30 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.934+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c1a6580 con 0x7fa45c10bb30 2024-02-19T16:27:01.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.934+0000 7fa4527fc640 1 -- 172.21.15.8:0/943168544 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454010650 con 0x7fa45c10bb30 2024-02-19T16:27:01.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.934+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c1a6a90 con 0x7fa45c10bb30 2024-02-19T16:27:01.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa4527fc640 1 -- 172.21.15.8:0/943168544 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fa454005020 con 0x7fa45c10bb30 2024-02-19T16:27:01.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 Processor -- start 2024-02-19T16:27:01.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 -- start start 2024-02-19T16:27:01.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebb4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebbd750 0x55f08ebbfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08eb6f960 con 0x55f08ebb4b70 2024-02-19T16:27:01.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08eb23310 con 0x55f08ebb4250 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.936+0000 7fa46637b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08eb20840 con 0x55f08ebbd750 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebb4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa46145f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa46145f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44526/0 (socket says 172.21.15.8:44526) 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebb4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48400/0 (socket says 172.21.15.8:48400) 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa46145f640 1 -- 172.21.15.8:0/242940357 learned_addr learned my addr 172.21.15.8:0/242940357 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebbd750 0x55f08ebbfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa4527fc640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa4480a22b0 0x7fa4480a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 -- 172.21.15.8:0/242940357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebbd750 msgr2=0x55f08ebbfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebbd750 0x55f08ebbfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 -- 172.21.15.8:0/242940357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 msgr2=0x55f08ebbd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 -- 172.21.15.8:0/242940357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f08eade060 con 0x55f08ebb4250 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa46145f640 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa461c60640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa4480a22b0 0x7fa4480a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa460c5e640 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebb4630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa45c101e40 tx=0x7fa44c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2364dd145849cb62 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa4527fc640 1 -- 172.21.15.8:0/943168544 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa454014070 con 0x7fa45c10bb30 2024-02-19T16:27:01.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa4477fe640 1 -- 172.21.15.8:0/242940357 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004050 con 0x55f08ebb4250 2024-02-19T16:27:01.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa4477fe640 1 -- 172.21.15.8:0/242940357 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa44c00fca0 con 0x55f08ebb4250 2024-02-19T16:27:01.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.937+0000 7fa4477fe640 1 -- 172.21.15.8:0/242940357 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c003ab0 con 0x55f08ebb4250 2024-02-19T16:27:01.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 -- 172.21.15.8:0/242940357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 msgr2=0x55f08ebb4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa461c60640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa4480a22b0 0x7fa4480a4770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa45409aee0 tx=0x7fa4540664f0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebb4630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa45c101e40 tx=0x7fa44c004730 comp rx=0 tx=0).stop 2024-02-19T16:27:01.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 -- 172.21.15.8:0/242940357 shutdown_connections 2024-02-19T16:27:01.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebbd750 0x55f08ebbfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 --2- 172.21.15.8:0/242940357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebb4630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 -- 172.21.15.8:0/242940357 >> 172.21.15.8:0/242940357 conn(0x55f08ebae980 msgr2=0x55f08ebaed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 -- 172.21.15.8:0/242940357 shutdown_connections 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 -- 172.21.15.8:0/242940357 wait complete. 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 -- start start 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.938+0000 7fa46637b740 1 Processor -- start 2024-02-19T16:27:01.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebbd750 0x55f08ebbb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08eb1c020 con 0x55f08ebb4b70 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08e7ef430 con 0x55f08ebbd750 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08e7abe00 con 0x55f08ebb4250 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46637b740 10 client.0 _ll_register_callbacks cb 0x55f08eac3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44534/0 (socket says 172.21.15.8:44534) 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 -- 172.21.15.8:0/887990622 learned_addr learned my addr 172.21.15.8:0/887990622 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa460c5e640 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebbd750 0x55f08ebbb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 -- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 msgr2=0x55f08ebba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 -- 172.21.15.8:0/887990622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebbd750 msgr2=0x55f08ebbb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebbd750 0x55f08ebbb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 -- 172.21.15.8:0/887990622 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c019040 con 0x55f08ebb4b70 2024-02-19T16:27:01.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa46145f640 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbad50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa458009230 tx=0x7fa45802cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e10e5873693c67 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.939+0000 7fa444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa444ff9640 1 -- 172.21.15.8:0/887990622 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458004040 con 0x55f08ebb4b70 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa444ff9640 1 -- 172.21.15.8:0/887990622 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa458046070 con 0x55f08ebb4b70 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f08e81b7e0 con 0x55f08ebb4b70 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa444ff9640 1 -- 172.21.15.8:0/887990622 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4580041a0 con 0x55f08ebb4b70 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f08eb1a6a0 con 0x55f08ebb4b70 2024-02-19T16:27:01.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa444ff9640 1 -- 172.21.15.8:0/887990622 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f08e81b7e0 con 0x55f08ebb4b70 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa444ff9640 1 -- 172.21.15.8:0/887990622 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa45803c040 con 0x55f08ebb4b70 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 10 client.16554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 20 client.16554 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ebcdc30 0x55f08ebd0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 4 client.16554 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f08e858720 con 0x55f08ebcdc30 2024-02-19T16:27:01.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.940+0000 7fa460c5e640 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ebcdc30 0x55f08ebd0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa444ff9640 10 client.16554 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:01.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa460c5e640 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ebcdc30 0x55f08ebd0050 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48f6ba9458ccfc38 server_cookie=e4ab982268771c85 in_seq=0 out_seq=0 2024-02-19T16:27:01.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa444ff9640 1 -- 172.21.15.8:0/887990622 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f08e858720 con 0x55f08ebcdc30 2024-02-19T16:27:01.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa444ff9640 10 client.16554 handle_command_reply: tid=0 2024-02-19T16:27:01.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa46637b740 1 client.16554 shutdown 2024-02-19T16:27:01.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 msgr2=0x55f08ebbad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.941+0000 7fa46637b740 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbad50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa458009230 tx=0x7fa45802cab0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 shutdown_connections 2024-02-19T16:27:01.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ebcdc30 0x55f08ebd0050 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebbd750 0x55f08ebbb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa444ff9640 0 client.16554 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:01.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebb4b70 0x55f08ebbad50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 --2- 172.21.15.8:0/887990622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 >> 172.21.15.8:0/887990622 conn(0x55f08ebae980 msgr2=0x55f08ebb06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 shutdown_connections 2024-02-19T16:27:01.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 1 -- 172.21.15.8:0/887990622 wait complete. 2024-02-19T16:27:01.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.942+0000 7fa46637b740 20 client.16554 trim_cache size 0 max 16384 2024-02-19T16:27:01.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 Processor -- start 2024-02-19T16:27:01.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 -- start start 2024-02-19T16:27:01.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4b70 0x55f08ebb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebb08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08eaa1130 con 0x55f08ebbd750 2024-02-19T16:27:01.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08ebb55f0 con 0x55f08ebb4b70 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46637b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08ebaa2b0 con 0x55f08ebb4250 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebb08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebb08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44540/0 (socket says 172.21.15.8:44540) 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa460c5e640 1 -- 172.21.15.8:0/1044589173 learned_addr learned my addr 172.21.15.8:0/1044589173 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa46145f640 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4b70 0x55f08ebb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.954+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa460c5e640 1 -- 172.21.15.8:0/1044589173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 msgr2=0x55f08ebbb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebbb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa460c5e640 1 -- 172.21.15.8:0/1044589173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4b70 msgr2=0x55f08ebb0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4b70 0x55f08ebb0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa460c5e640 1 -- 172.21.15.8:0/1044589173 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f08eab6c20 con 0x55f08ebbd750 2024-02-19T16:27:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebb08d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009a90 tx=0x7fa44c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c272dddcfcd935b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa446ffd640 1 -- 172.21.15.8:0/1044589173 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00f920 con 0x55f08ebbd750 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa446ffd640 1 -- 172.21.15.8:0/1044589173 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa44c038070 con 0x55f08ebbd750 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.955+0000 7fa446ffd640 1 -- 172.21.15.8:0/1044589173 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c032b00 con 0x55f08ebbd750 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- 172.21.15.8:0/1044589173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 msgr2=0x55f08ebb08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebb08d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009a90 tx=0x7fa44c005f90 comp rx=0 tx=0).stop 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- 172.21.15.8:0/1044589173 shutdown_connections 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebb08d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4b70 0x55f08ebb0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- 172.21.15.8:0/1044589173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4250 0x55f08ebbb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- 172.21.15.8:0/1044589173 >> 172.21.15.8:0/1044589173 conn(0x55f08ebae980 msgr2=0x55f08ebafc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- 172.21.15.8:0/1044589173 shutdown_connections 2024-02-19T16:27:01.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- 172.21.15.8:0/1044589173 wait complete. 2024-02-19T16:27:01.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- start start 2024-02-19T16:27:01.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 Processor -- start 2024-02-19T16:27:01.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebc1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4b70 0x55f08ebc1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebc24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08ec294b0 con 0x55f08ebbd750 2024-02-19T16:27:01.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08ec295f0 con 0x55f08ebb4250 2024-02-19T16:27:01.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08ec29730 con 0x55f08ebb4b70 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa46637b740 10 client.0 _ll_register_callbacks cb 0x7fa458009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.956+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebc24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebc24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44550/0 (socket says 172.21.15.8:44550) 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 -- 172.21.15.8:0/1869390927 learned_addr learned my addr 172.21.15.8:0/1869390927 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebc1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa46145f640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4b70 0x55f08ebc1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 -- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4b70 msgr2=0x55f08ebc1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4b70 0x55f08ebc1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 -- 172.21.15.8:0/1869390927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 msgr2=0x55f08ebc1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:01.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebc1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 -- 172.21.15.8:0/1869390927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f08eade060 con 0x55f08ebbd750 2024-02-19T16:27:01.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa460c5e640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebc24c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa44c000c00 tx=0x7fa44c034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03e865e90d88880 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa444ff9640 1 -- 172.21.15.8:0/1869390927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c032880 con 0x55f08ebbd750 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa444ff9640 1 -- 172.21.15.8:0/1869390927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa44c0329e0 con 0x55f08ebbd750 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa44c00fca0 con 0x55f08ebbd750 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa444ff9640 1 -- 172.21.15.8:0/1869390927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c032ba0 con 0x55f08ebbd750 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa46637b740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f08eadc8c0 con 0x55f08ebbd750 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.957+0000 7fa444ff9640 1 -- 172.21.15.8:0/1869390927 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa44c00fca0 con 0x55f08ebbd750 2024-02-19T16:27:01.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa444ff9640 1 -- 172.21.15.8:0/1869390927 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa44c01d070 con 0x55f08ebbd750 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 20 client.16560 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ec2aa60 0x55f08ec2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 4 client.16560 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f08e8a7890 con 0x55f08ec2aa60 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46145f640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ec2aa60 0x55f08ec2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:01.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa444ff9640 10 client.16560 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46145f640 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ec2aa60 0x55f08ec2ce50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b4c595695c9c637 server_cookie=e0b3d7b17b5fa4c9 in_seq=0 out_seq=0 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa444ff9640 1 -- 172.21.15.8:0/1869390927 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f08e8a7890 con 0x55f08ec2aa60 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa444ff9640 10 client.16560 handle_command_reply: tid=0 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 client.16560 shutdown 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 msgr2=0x55f08ebc24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebc24c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa44c000c00 tx=0x7fa44c034040 comp rx=0 tx=0).stop 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 shutdown_connections 2024-02-19T16:27:01.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55f08ec2aa60 0x55f08ec2ce50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f08ebbd750 0x55f08ebc24c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f08ebb4b70 0x55f08ebc1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa444ff9640 0 client.16560 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 --2- 172.21.15.8:0/1869390927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f08ebb4250 0x55f08ebc1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.958+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 >> 172.21.15.8:0/1869390927 conn(0x55f08ebae980 msgr2=0x55f08ebb0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 shutdown_connections 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa46637b740 1 -- 172.21.15.8:0/1869390927 wait complete. 2024-02-19T16:27:01.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa46637b740 20 client.16560 trim_cache size 0 max 16384 2024-02-19T16:27:01.985 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:27:01.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa4480a22b0 msgr2=0x7fa4480a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa4480a22b0 0x7fa4480a4770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa45409aee0 tx=0x7fa4540664f0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 msgr2=0x7fa45c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 0x7fa45c100e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa45400eb10 tx=0x7fa45400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 shutdown_connections 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c10bb30 0x7fa45c100e30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa45c103080 0x7fa45c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa4480a22b0 0x7fa4480a4770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 --2- 172.21.15.8:0/943168544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa45c102760 0x7fa45c1003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:01.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 >> 172.21.15.8:0/943168544 conn(0x7fa45c077550 msgr2=0x7fa45c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:01.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 shutdown_connections 2024-02-19T16:27:01.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:01.959+0000 7fa4636c3640 1 -- 172.21.15.8:0/943168544 wait complete. 2024-02-19T16:27:01.988 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:27:01.988 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:27:02.012 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:01 smithi008 ceph-mon[32573]: pgmap v127: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:02.012 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:01 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:27:02.012 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:01 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:27:02.012 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:01 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:01 smithi191 ceph-mon[37203]: pgmap v127: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:01 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:01 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:27:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:01 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:27:02.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:01 smithi156 ceph-mon[36502]: pgmap v127: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:02.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:01 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:27:02.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:01 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:27:02.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:01 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:27:02.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 -- 172.21.15.8:0/696717448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 msgr2=0x7f5a9010d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 --2- 172.21.15.8:0/696717448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9010d1d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009930 tx=0x7f5a8002bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 -- 172.21.15.8:0/696717448 shutdown_connections 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 --2- 172.21.15.8:0/696717448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9010d1d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 --2- 172.21.15.8:0/696717448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a90100740 0x7f5a90100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 --2- 172.21.15.8:0/696717448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a901053d0 0x7f5a901057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 -- 172.21.15.8:0/696717448 >> 172.21.15.8:0/696717448 conn(0x7f5a900761f0 msgr2=0x7f5a90076600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 -- 172.21.15.8:0/696717448 shutdown_connections 2024-02-19T16:27:02.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 -- 172.21.15.8:0/696717448 wait complete. 2024-02-19T16:27:02.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.092+0000 7f5a9548f640 1 Processor -- start 2024-02-19T16:27:02.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 -- start start 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a90100740 0x7f5a9019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a901053d0 0x7f5a901968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90196f10 con 0x7f5a90101100 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90197080 con 0x7f5a901053d0 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a9548f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90197240 con 0x7f5a90100740 2024-02-19T16:27:02.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a867fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a90100740 0x7f5a9019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a867fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44568/0 (socket says 172.21.15.8:44568) 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a867fc640 1 -- 172.21.15.8:0/3238674516 learned_addr learned my addr 172.21.15.8:0/3238674516 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a901053d0 0x7f5a901968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 -- 172.21.15.8:0/3238674516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a901053d0 msgr2=0x7f5a901968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a901053d0 0x7f5a901968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 -- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 msgr2=0x7f5a9019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 -- 172.21.15.8:0/3238674516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80009560 con 0x7f5a90100740 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a867fc640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.093+0000 7f5a8effd640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a90100740 0x7f5a9019c6c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009fd0 tx=0x7f5a8800b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.094+0000 7f5a8cff9640 1 -- 172.21.15.8:0/3238674516 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88013070 con 0x7f5a90100740 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.094+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a90197520 con 0x7f5a90100740 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.094+0000 7f5a8cff9640 1 -- 172.21.15.8:0/3238674516 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a88017450 con 0x7f5a90100740 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.094+0000 7f5a8cff9640 1 -- 172.21.15.8:0/3238674516 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88005280 con 0x7f5a90100740 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.094+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a90069ca0 con 0x7f5a90100740 2024-02-19T16:27:02.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.096+0000 7f5a8cff9640 1 -- 172.21.15.8:0/3238674516 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f5a880175c0 con 0x7f5a90100740 2024-02-19T16:27:02.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.096+0000 7f5a98061740 1 Processor -- start 2024-02-19T16:27:02.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.096+0000 7f5a98061740 1 -- start start 2024-02-19T16:27:02.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a98061740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816250 0x55b9a5816630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a98061740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 0x55b9a581f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a98061740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5821b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a98061740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a57d1960 con 0x55b9a5816b70 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a98061740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a5785310 con 0x55b9a581f750 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a98061740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a5782840 con 0x55b9a5816250 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 0x55b9a581f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8cff9640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5a7c0a2230 0x7f5a7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 0x55b9a581f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44584/0 (socket says 172.21.15.8:44584) 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816250 0x55b9a5816630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 -- 172.21.15.8:0/2914356060 learned_addr learned my addr 172.21.15.8:0/2914356060 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a867fc640 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5821b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5a7c0a2230 0x7f5a7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8cff9640 1 -- 172.21.15.8:0/3238674516 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f5a880cff90 con 0x7f5a90100740 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 -- 172.21.15.8:0/2914356060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816250 msgr2=0x55b9a5816630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816250 0x55b9a5816630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 -- 172.21.15.8:0/2914356060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 msgr2=0x55b9a5821b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5821b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 -- 172.21.15.8:0/2914356060 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a5740060 con 0x55b9a5816b70 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a867fc640 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5821b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:02.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a8effd640 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 0x55b9a581f210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5a8809c7e0 tx=0x7f5a88017cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc234d3d92be2e29 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.097+0000 7f5a857fa640 1 -- 172.21.15.8:0/2914356060 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8809bf80 con 0x55b9a5816b70 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5a7c0a2230 0x7f5a7c0a46f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005940 tx=0x7f5a8002bc30 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a857fa640 1 -- 172.21.15.8:0/2914356060 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a8809c620 con 0x55b9a5816b70 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a857fa640 1 -- 172.21.15.8:0/2914356060 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88098c90 con 0x55b9a5816b70 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 -- 172.21.15.8:0/2914356060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 msgr2=0x55b9a581f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 0x55b9a581f210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5a8809c7e0 tx=0x7f5a88017cf0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 -- 172.21.15.8:0/2914356060 shutdown_connections 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5821b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816b70 0x55b9a581f210 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 --2- 172.21.15.8:0/2914356060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816250 0x55b9a5816630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 -- 172.21.15.8:0/2914356060 >> 172.21.15.8:0/2914356060 conn(0x55b9a5810980 msgr2=0x55b9a5810d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 -- 172.21.15.8:0/2914356060 shutdown_connections 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.098+0000 7f5a98061740 1 -- 172.21.15.8:0/2914356060 wait complete. 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 -- start start 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 Processor -- start 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a581c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816b70 0x55b9a581cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a581f750 0x55b9a581d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a577e020 con 0x55b9a5816250 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a5451430 con 0x55b9a5816b70 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a540de00 con 0x55b9a581f750 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a98061740 10 client.0 _ll_register_callbacks cb 0x55b9a5725cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a581c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a581c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44588/0 (socket says 172.21.15.8:44588) 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 -- 172.21.15.8:0/946034457 learned_addr learned my addr 172.21.15.8:0/946034457 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a867fc640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816b70 0x55b9a581cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a581f750 0x55b9a581d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 -- 172.21.15.8:0/946034457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a581f750 msgr2=0x55b9a581d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a581f750 0x55b9a581d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 -- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816b70 msgr2=0x55b9a581cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816b70 0x55b9a581cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 -- 172.21.15.8:0/946034457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a90069ca0 con 0x55b9a5816250 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a867fc640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816b70 0x55b9a581cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a8effd640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a581c800 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5a88098a80 tx=0x7f5a880a4d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0a46d24939e23 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.099+0000 7f5a767fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a767fc640 1 -- 172.21.15.8:0/946034457 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880cd750 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a767fc640 1 -- 172.21.15.8:0/946034457 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f5a880a06c0 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9a547d7e0 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a767fc640 1 -- 172.21.15.8:0/946034457 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a880a4880 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 10 client.16572 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b9a577c6a0 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a767fc640 1 -- 172.21.15.8:0/946034457 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b9a547d7e0 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a767fc640 1 -- 172.21.15.8:0/946034457 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5a880a7d20 con 0x55b9a5816250 2024-02-19T16:27:02.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 10 client.16572 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 10 client.16572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 20 client.16572 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a582f7b0 0x55b9a5831bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 4 client.16572 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9a54ba720 con 0x55b9a582f7b0 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.100+0000 7f5a867fc640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a582f7b0 0x55b9a5831bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a767fc640 10 client.16572 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a867fc640 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a582f7b0 0x55b9a5831bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=568cf2c724fb61e6 server_cookie=a2eb34f07416d1ad in_seq=0 out_seq=0 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a767fc640 1 -- 172.21.15.8:0/946034457 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b9a54ba720 con 0x55b9a582f7b0 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a767fc640 10 client.16572 handle_command_reply: tid=0 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 client.16572 shutdown 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 msgr2=0x55b9a581c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a581c800 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5a88098a80 tx=0x7f5a880a4d90 comp rx=0 tx=0).stop 2024-02-19T16:27:02.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 shutdown_connections 2024-02-19T16:27:02.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a582f7b0 0x55b9a5831bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a581f750 0x55b9a581d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a98061740 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816b70 0x55b9a581cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.101+0000 7f5a767fc640 0 client.16572 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:02.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.102+0000 7f5a98061740 1 --2- 172.21.15.8:0/946034457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a581c800 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.102+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 >> 172.21.15.8:0/946034457 conn(0x55b9a5810980 msgr2=0x55b9a58126d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.102+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 shutdown_connections 2024-02-19T16:27:02.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.102+0000 7f5a98061740 1 -- 172.21.15.8:0/946034457 wait complete. 2024-02-19T16:27:02.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.102+0000 7f5a98061740 20 client.16572 trim_cache size 0 max 16384 2024-02-19T16:27:02.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.114+0000 7f5a98061740 1 Processor -- start 2024-02-19T16:27:02.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.114+0000 7f5a98061740 1 -- start start 2024-02-19T16:27:02.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a98061740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816250 0x55b9a581ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a98061740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a582aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a98061740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 0x55b9a582b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a98061740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a5703130 con 0x55b9a581f750 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a98061740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a582c6f0 con 0x55b9a5816250 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a98061740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a5811980 con 0x55b9a5816b70 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 0x55b9a582b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a867fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a582aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816250 0x55b9a581ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 0x55b9a582b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44604/0 (socket says 172.21.15.8:44604) 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a867fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a582aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:38658/0 (socket says 172.21.15.8:38658) 2024-02-19T16:27:02.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 -- 172.21.15.8:0/4103973757 learned_addr learned my addr 172.21.15.8:0/4103973757 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 -- 172.21.15.8:0/4103973757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 msgr2=0x55b9a582aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a582aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 -- 172.21.15.8:0/4103973757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816250 msgr2=0x55b9a581ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816250 0x55b9a581ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 -- 172.21.15.8:0/4103973757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a5718c20 con 0x55b9a581f750 2024-02-19T16:27:02.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a867fc640 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a582aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8effd640 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816250 0x55b9a581ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.115+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 0x55b9a582b010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009230 tx=0x7f5a80005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83b91d1bcda6d77 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a84ff9640 1 -- 172.21.15.8:0/4103973757 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80004110 con 0x55b9a581f750 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a84ff9640 1 -- 172.21.15.8:0/4103973757 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a80004270 con 0x55b9a581f750 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a84ff9640 1 -- 172.21.15.8:0/4103973757 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80005030 con 0x55b9a581f750 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 -- 172.21.15.8:0/4103973757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 msgr2=0x55b9a582b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 0x55b9a582b010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009230 tx=0x7f5a80005fd0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 -- 172.21.15.8:0/4103973757 shutdown_connections 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a581f750 0x55b9a582b010 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a582aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 --2- 172.21.15.8:0/4103973757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a5816250 0x55b9a581ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 -- 172.21.15.8:0/4103973757 >> 172.21.15.8:0/4103973757 conn(0x55b9a5810980 msgr2=0x55b9a5812640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 -- 172.21.15.8:0/4103973757 shutdown_connections 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.116+0000 7f5a98061740 1 -- 172.21.15.8:0/4103973757 wait complete. 2024-02-19T16:27:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 -- start start 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 Processor -- start 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a5827470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a58237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5823e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a58244d0 con 0x55b9a5816250 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a5824640 con 0x55b9a581f750 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a588b070 con 0x55b9a5816b70 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a98061740 10 client.0 _ll_register_callbacks cb 0x7f5a78009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:02.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a5827470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a5827470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44608/0 (socket says 172.21.15.8:44608) 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 -- 172.21.15.8:0/1821626296 learned_addr learned my addr 172.21.15.8:0/1821626296 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a867fc640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a58237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8f7fe640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5823e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 -- 172.21.15.8:0/1821626296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 msgr2=0x55b9a58237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a58237f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 -- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 msgr2=0x55b9a5823e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5823e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.117+0000 7f5a8effd640 1 -- 172.21.15.8:0/1821626296 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9a577c6a0 con 0x55b9a5816250 2024-02-19T16:27:02.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a8effd640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a5827470 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a8809ba70 tx=0x7f5a880a4ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756d759ef4d61da8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a767fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:02.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a767fc640 1 -- 172.21.15.8:0/1821626296 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880988c0 con 0x55b9a5816250 2024-02-19T16:27:02.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a767fc640 1 -- 172.21.15.8:0/1821626296 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f5a88098050 con 0x55b9a5816250 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a8809c620 con 0x55b9a5816250 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a767fc640 1 -- 172.21.15.8:0/1821626296 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a880a06c0 con 0x55b9a5816250 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5a78009d10 con 0x55b9a5816250 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a767fc640 1 -- 172.21.15.8:0/1821626296 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a8809c620 con 0x55b9a5816250 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a767fc640 1 -- 172.21.15.8:0/1821626296 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5a88082020 con 0x55b9a5816250 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:02.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 10 client.16578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 20 client.16578 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a588c420 0x55b9a588e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 4 client.16578 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9a5509890 con 0x55b9a588c420 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.118+0000 7f5a867fc640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a588c420 0x55b9a588e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a767fc640 10 client.16578 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a867fc640 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a588c420 0x55b9a588e840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4ebbb08ecddfc5 server_cookie=367a02d02eb63cf3 in_seq=0 out_seq=0 2024-02-19T16:27:02.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a767fc640 1 -- 172.21.15.8:0/1821626296 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b9a5509890 con 0x55b9a588c420 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a767fc640 10 client.16578 handle_command_reply: tid=0 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 client.16578 shutdown 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 msgr2=0x55b9a5827470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a5827470 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a8809ba70 tx=0x7f5a880a4ab0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 shutdown_connections 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55b9a588c420 0x55b9a588e840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9a581f750 0x55b9a5823e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9a5816b70 0x55b9a58237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a767fc640 0 client.16578 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:02.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 --2- 172.21.15.8:0/1821626296 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b9a5816250 0x55b9a5827470 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 >> 172.21.15.8:0/1821626296 conn(0x55b9a5810980 msgr2=0x55b9a580f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 shutdown_connections 2024-02-19T16:27:02.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 1 -- 172.21.15.8:0/1821626296 wait complete. 2024-02-19T16:27:02.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.119+0000 7f5a98061740 20 client.16578 trim_cache size 0 max 16384 2024-02-19T16:27:02.136 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:27:02.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5a7c0a2230 msgr2=0x7f5a7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5a7c0a2230 0x7f5a7c0a46f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005940 tx=0x7f5a8002bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:02.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a90100740 msgr2=0x7f5a9019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a90100740 0x7f5a9019c6c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009fd0 tx=0x7f5a8800b690 comp rx=0 tx=0).stop 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 shutdown_connections 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a901053d0 0x7f5a901968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101100 0x7f5a9019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f5a7c0a2230 0x7f5a7c0a46f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 --2- 172.21.15.8:0/3238674516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a90100740 0x7f5a9019c6c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 >> 172.21.15.8:0/3238674516 conn(0x7f5a900761f0 msgr2=0x7f5a900fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 shutdown_connections 2024-02-19T16:27:02.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.120+0000 7f5a9548f640 1 -- 172.21.15.8:0/3238674516 wait complete. 2024-02-19T16:27:02.139 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:27:02.139 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:27:02.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 -- 172.21.15.8:0/3036802090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e01014a0 msgr2=0x7fe8e0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/3036802090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e01014a0 0x7fe8e0101920 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009900 tx=0x7fe8d002bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 -- 172.21.15.8:0/3036802090 shutdown_connections 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/3036802090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8e0101e60 0x7fe8e010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/3036802090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e01014a0 0x7fe8e0101920 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/3036802090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e0107560 0x7fe8e0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 -- 172.21.15.8:0/3036802090 >> 172.21.15.8:0/3036802090 conn(0x7fe8e00788f0 msgr2=0x7fe8e0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 -- 172.21.15.8:0/3036802090 shutdown_connections 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.282+0000 7fe8e7e42640 1 -- 172.21.15.8:0/3036802090 wait complete. 2024-02-19T16:27:02.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 Processor -- start 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 -- start start 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8e01014a0 0x7fe8e019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e0101e60 0x7fe8e019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 0x7fe8e01978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0197f10 con 0x7fe8e0107560 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0198080 con 0x7fe8e01014a0 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e7e42640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0198240 con 0x7fe8e0101e60 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 0x7fe8e01978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 0x7fe8e01978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44618/0 (socket says 172.21.15.8:44618) 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e5bde640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8e01014a0 0x7fe8e019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 -- 172.21.15.8:0/2840950059 learned_addr learned my addr 172.21.15.8:0/2840950059 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e0101e60 0x7fe8e019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 -- 172.21.15.8:0/2840950059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e0101e60 msgr2=0x7fe8e019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e0101e60 0x7fe8e019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 -- 172.21.15.8:0/2840950059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8e01014a0 msgr2=0x7fe8e019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8e01014a0 0x7fe8e019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.283+0000 7fe8e63df640 1 -- 172.21.15.8:0/2840950059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8009840 con 0x7fe8e0107560 2024-02-19T16:27:02.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.284+0000 7fe8e63df640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 0x7fe8e01978c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800efd0 tx=0x7fe8d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.284+0000 7fe8d6ffd640 1 -- 172.21.15.8:0/2840950059 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8009b40 con 0x7fe8e0107560 2024-02-19T16:27:02.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.284+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d0009560 con 0x7fe8e0107560 2024-02-19T16:27:02.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.284+0000 7fe8d6ffd640 1 -- 172.21.15.8:0/2840950059 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8d8003b70 con 0x7fe8e0107560 2024-02-19T16:27:02.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.284+0000 7fe8d6ffd640 1 -- 172.21.15.8:0/2840950059 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80108f0 con 0x7fe8e0107560 2024-02-19T16:27:02.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.284+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e01a45d0 con 0x7fe8e0107560 2024-02-19T16:27:02.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.286+0000 7fe8d6ffd640 1 -- 172.21.15.8:0/2840950059 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe8d8003ce0 con 0x7fe8e0107560 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 Processor -- start 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 -- start start 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x55651889d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889db70 0x5565188a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565188a6750 0x5565188a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518858960 con 0x55651889d250 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651880c310 con 0x55651889db70 2024-02-19T16:27:02.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518809840 con 0x5565188a6750 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8d6ffd640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8cc0a2230 0x7fe8cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e63df640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8cc0a2230 0x7fe8cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x55651889d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x55651889d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44624/0 (socket says 172.21.15.8:44624) 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e53dd640 1 -- 172.21.15.8:0/1233074817 learned_addr learned my addr 172.21.15.8:0/1233074817 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e5bde640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889db70 0x5565188a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8d6ffd640 1 -- 172.21.15.8:0/2840950059 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fe8d8014070 con 0x7fe8e0107560 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565188a6750 0x5565188a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e5bde640 1 -- 172.21.15.8:0/1233074817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565188a6750 msgr2=0x5565188a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565188a6750 0x5565188a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e5bde640 1 -- 172.21.15.8:0/1233074817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 msgr2=0x55651889d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x55651889d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.287+0000 7fe8e5bde640 1 -- 172.21.15.8:0/1233074817 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565187c7060 con 0x55651889db70 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.288+0000 7fe8e63df640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8cc0a2230 0x7fe8cc0a46f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80974d0 tx=0x7fe8d8009f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.288+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889db70 0x5565188a6210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009bd0 tx=0x7fe8dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe69147d4fe8b3f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.288+0000 7fe8cbfff640 1 -- 172.21.15.8:0/1233074817 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc004050 con 0x55651889db70 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.288+0000 7fe8cbfff640 1 -- 172.21.15.8:0/1233074817 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc005500 con 0x55651889db70 2024-02-19T16:27:02.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.288+0000 7fe8cbfff640 1 -- 172.21.15.8:0/1233074817 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc004c20 con 0x55651889db70 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1233074817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889db70 msgr2=0x5565188a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889db70 0x5565188a6210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009bd0 tx=0x7fe8dc011810 comp rx=0 tx=0).stop 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1233074817 shutdown_connections 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565188a6750 0x5565188a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889db70 0x5565188a6210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1233074817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x55651889d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1233074817 >> 172.21.15.8:0/1233074817 conn(0x556518897980 msgr2=0x556518897d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1233074817 shutdown_connections 2024-02-19T16:27:02.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1233074817 wait complete. 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 -- start start 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 Processor -- start 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889d250 0x5565188a3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.289+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 0x5565188a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518805020 con 0x5565188a6750 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565184d8430 con 0x55651889d250 2024-02-19T16:27:02.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518494e00 con 0x55651889db70 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 10 client.0 _ll_register_callbacks cb 0x5565187accb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 0x5565188a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e5bde640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 0x5565188a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44634/0 (socket says 172.21.15.8:44634) 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 -- 172.21.15.8:0/1870217928 learned_addr learned my addr 172.21.15.8:0/1870217928 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889d250 0x5565188a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 -- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 msgr2=0x5565188a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 -- 172.21.15.8:0/1870217928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889d250 msgr2=0x5565188a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889d250 0x5565188a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 -- 172.21.15.8:0/1870217928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc009830 con 0x5565188a6750 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 0x5565188a40e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009900 tx=0x7fe8d0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f1170d7e7d40b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8c97fa640 1 -- 172.21.15.8:0/1870217928 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0004480 con 0x5565188a6750 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8c97fa640 1 -- 172.21.15.8:0/1870217928 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fe8d0004df0 con 0x5565188a6750 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565185047e0 con 0x5565188a6750 2024-02-19T16:27:02.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8c97fa640 1 -- 172.21.15.8:0/1870217928 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8d0004fb0 con 0x5565188a6750 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 10 client.16590 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.290+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5565188036a0 con 0x5565188a6750 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8c97fa640 1 -- 172.21.15.8:0/1870217928 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5565185047e0 con 0x5565188a6750 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8c97fa640 1 -- 172.21.15.8:0/1870217928 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe8d004d030 con 0x5565188a6750 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8eaa16740 10 client.16590 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8eaa16740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8eaa16740 20 client.16590 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5565188b6a30 0x5565188b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8eaa16740 4 client.16590 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556518541720 con 0x5565188b6a30 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5565188b6a30 0x5565188b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5565188b6a30 0x5565188b8e50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f20cab76f36b3fd1 server_cookie=78ca185954e29a84 in_seq=0 out_seq=0 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.291+0000 7fe8c97fa640 10 client.16590 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.292+0000 7fe8c97fa640 1 -- 172.21.15.8:0/1870217928 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556518541720 con 0x5565188b6a30 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.292+0000 7fe8c97fa640 10 client.16590 handle_command_reply: tid=0 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.292+0000 7fe8eaa16740 1 client.16590 shutdown 2024-02-19T16:27:02.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.292+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 msgr2=0x5565188a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.292+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 0x5565188a40e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009900 tx=0x7fe8d0005e40 comp rx=0 tx=0).stop 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 shutdown_connections 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5565188a6750 0x5565188a40e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5565188b6a30 0x5565188b8e50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/1870217928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55651889d250 0x5565188a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8c97fa640 0 client.16590 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 >> 172.21.15.8:0/1870217928 conn(0x556518897980 msgr2=0x5565188996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 shutdown_connections 2024-02-19T16:27:02.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 1 -- 172.21.15.8:0/1870217928 wait complete. 2024-02-19T16:27:02.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.293+0000 7fe8eaa16740 20 client.16590 trim_cache size 0 max 16384 2024-02-19T16:27:02.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 Processor -- start 2024-02-19T16:27:02.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 -- start start 2024-02-19T16:27:02.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x5565188b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651878a130 con 0x55651889d250 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565188a9420 con 0x5565188a6750 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565188b6800 con 0x55651889db70 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44648/0 (socket says 172.21.15.8:44648) 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8e53dd640 1 -- 172.21.15.8:0/3280787059 learned_addr learned my addr 172.21.15.8:0/3280787059 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.305+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x5565188b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 -- 172.21.15.8:0/3280787059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 msgr2=0x5565188b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 -- 172.21.15.8:0/3280787059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 msgr2=0x5565188b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x5565188b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 -- 172.21.15.8:0/3280787059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55651879fc20 con 0x55651889d250 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188a5d90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe8d002cb80 tx=0x7fe8d002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f275eaf2ed540e15 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8cb7fe640 1 -- 172.21.15.8:0/3280787059 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00046c0 con 0x55651889d250 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8cb7fe640 1 -- 172.21.15.8:0/3280787059 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8d0049070 con 0x55651889d250 2024-02-19T16:27:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8cb7fe640 1 -- 172.21.15.8:0/3280787059 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00052d0 con 0x55651889d250 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 -- 172.21.15.8:0/3280787059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 msgr2=0x5565188a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188a5d90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe8d002cb80 tx=0x7fe8d002bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 -- 172.21.15.8:0/3280787059 shutdown_connections 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x5565188b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/3280787059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188a5d90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.306+0000 7fe8eaa16740 1 -- 172.21.15.8:0/3280787059 >> 172.21.15.8:0/3280787059 conn(0x556518897980 msgr2=0x556518896b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 -- 172.21.15.8:0/3280787059 shutdown_connections 2024-02-19T16:27:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 -- 172.21.15.8:0/3280787059 wait complete. 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 -- start start 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 Processor -- start 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x556518911300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518911840 con 0x55651889d250 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518911980 con 0x5565188a6750 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556518911ac0 con 0x55651889db70 2024-02-19T16:27:02.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8eaa16740 10 client.0 _ll_register_callbacks cb 0x7fe8d0004a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8e53dd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188aa1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44652/0 (socket says 172.21.15.8:44652) 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8e53dd640 1 -- 172.21.15.8:0/982112876 learned_addr learned my addr 172.21.15.8:0/982112876 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.307+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x556518911300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8e53dd640 1 -- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 msgr2=0x5565188aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8e53dd640 1 -- 172.21.15.8:0/982112876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 msgr2=0x556518911300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x556518911300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8e53dd640 1 -- 172.21.15.8:0/982112876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565188036a0 con 0x55651889d250 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8e53dd640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188aa1d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe8d002c7b0 tx=0x7fe8d0003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb20f675f5a973d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8c97fa640 1 -- 172.21.15.8:0/982112876 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d002c810 con 0x55651889d250 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8c97fa640 1 -- 172.21.15.8:0/982112876 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fe8d0000e40 con 0x55651889d250 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8dc005500 con 0x55651889d250 2024-02-19T16:27:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8c97fa640 1 -- 172.21.15.8:0/982112876 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8d002c970 con 0x55651889d250 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5565187c58c0 con 0x55651889d250 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8c97fa640 1 -- 172.21.15.8:0/982112876 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc005500 con 0x55651889d250 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8c97fa640 1 -- 172.21.15.8:0/982112876 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe8d004d040 con 0x55651889d250 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 20 client.16596 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556518912ec0 0x5565189152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 4 client.16596 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.308+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556518590890 con 0x556518912ec0 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556518912ec0 0x5565189152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8e5bde640 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556518912ec0 0x5565189152e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7849a6cc75a6a03e server_cookie=f0b55c4562a805a2 in_seq=0 out_seq=0 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8c97fa640 10 client.16596 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8c97fa640 1 -- 172.21.15.8:0/982112876 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556518590890 con 0x556518912ec0 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8c97fa640 10 client.16596 handle_command_reply: tid=0 2024-02-19T16:27:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 client.16596 shutdown 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 msgr2=0x5565188aa1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188aa1d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe8d002c7b0 tx=0x7fe8d0003700 comp rx=0 tx=0).stop 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 shutdown_connections 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565188a6750 0x556518911300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556518912ec0 0x5565189152e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55651889db70 0x5565188aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8c97fa640 0 client.16596 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 --2- 172.21.15.8:0/982112876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55651889d250 0x5565188aa1d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 >> 172.21.15.8:0/982112876 conn(0x556518897980 msgr2=0x5565188962f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 shutdown_connections 2024-02-19T16:27:02.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.309+0000 7fe8eaa16740 1 -- 172.21.15.8:0/982112876 wait complete. 2024-02-19T16:27:02.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8eaa16740 20 client.16596 trim_cache size 0 max 16384 2024-02-19T16:27:02.323 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:27:02.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8cc0a2230 msgr2=0x7fe8cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8cc0a2230 0x7fe8cc0a46f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80974d0 tx=0x7fe8d8009f90 comp rx=0 tx=0).stop 2024-02-19T16:27:02.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 msgr2=0x7fe8e01978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 0x7fe8e01978c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800efd0 tx=0x7fe8d800c6a0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 shutdown_connections 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8e0107560 0x7fe8e01978c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8cc0a2230 0x7fe8cc0a46f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e0101e60 0x7fe8e019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 --2- 172.21.15.8:0/2840950059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8e01014a0 0x7fe8e019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 >> 172.21.15.8:0/2840950059 conn(0x7fe8e00788f0 msgr2=0x7fe8e0100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 shutdown_connections 2024-02-19T16:27:02.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:02.310+0000 7fe8e7e42640 1 -- 172.21.15.8:0/2840950059 wait complete. 2024-02-19T16:27:03.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:02 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1165721450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:27:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:02 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1165721450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:27:03.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:02 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1165721450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:27:03.327 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:27:03.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:27:03.327 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:27:03.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 -- 172.21.15.8:0/1477865291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 msgr2=0x7fa59010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 --2- 172.21.15.8:0/1477865291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59010d950 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa580009900 tx=0x7fa58002bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:03.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 -- 172.21.15.8:0/1477865291 shutdown_connections 2024-02-19T16:27:03.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 --2- 172.21.15.8:0/1477865291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa59010de90 0x7fa590110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 --2- 172.21.15.8:0/1477865291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59010d950 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 --2- 172.21.15.8:0/1477865291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 0x7fa590068ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 -- 172.21.15.8:0/1477865291 >> 172.21.15.8:0/1477865291 conn(0x7fa5900774f0 msgr2=0x7fa590077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 -- 172.21.15.8:0/1477865291 shutdown_connections 2024-02-19T16:27:03.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 -- 172.21.15.8:0/1477865291 wait complete. 2024-02-19T16:27:03.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 Processor -- start 2024-02-19T16:27:03.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.445+0000 7fa59693e640 1 -- start start 2024-02-19T16:27:03.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa59693e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 0x7fa590199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa59693e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa59693e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa59010de90 0x7fa59019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa59693e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019ec10 con 0x7fa590068ff0 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa59693e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019ed80 con 0x7fa59010de90 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa59693e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019ef40 con 0x7fa5900686d0 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 0x7fa590199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44678/0 (socket says 172.21.15.8:44678) 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 0x7fa590199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:38746/0 (socket says 172.21.15.8:38746) 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3985087336 learned_addr learned my addr 172.21.15.8:0/3985087336 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa594edb640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa59010de90 0x7fa59019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3985087336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 msgr2=0x7fa590199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 0x7fa590199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3985087336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa59010de90 msgr2=0x7fa59019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa59010de90 0x7fa59019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3985087336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa580009560 con 0x7fa590068ff0 2024-02-19T16:27:03.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59019a290 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa580005940 tx=0x7fa58002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.446+0000 7fa58d7fa640 1 -- 172.21.15.8:0/3985087336 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580004350 con 0x7fa590068ff0 2024-02-19T16:27:03.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.447+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59019f1c0 con 0x7fa590068ff0 2024-02-19T16:27:03.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.447+0000 7fa58d7fa640 1 -- 172.21.15.8:0/3985087336 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5800044b0 con 0x7fa590068ff0 2024-02-19T16:27:03.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.447+0000 7fa58d7fa640 1 -- 172.21.15.8:0/3985087336 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580003d30 con 0x7fa590068ff0 2024-02-19T16:27:03.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.447+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901a6aa0 con 0x7fa590068ff0 2024-02-19T16:27:03.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa58d7fa640 1 -- 172.21.15.8:0/3985087336 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fa580004df0 con 0x7fa590068ff0 2024-02-19T16:27:03.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 Processor -- start 2024-02-19T16:27:03.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 -- start start 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa58d7fa640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa5780a2230 0x7fa5780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d90512f90 con 0x556d905a4d30 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d905104c0 con 0x556d905a4380 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa5995fa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9050bca0 con 0x556d905ad8b0 2024-02-19T16:27:03.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa58ffff640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa5780a2230 0x7fa5780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa594edb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa594edb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48564/0 (socket says 172.21.15.8:48564) 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa594edb640 1 -- 172.21.15.8:0/3416396570 learned_addr learned my addr 172.21.15.8:0/3416396570 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.449+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44680/0 (socket says 172.21.15.8:44680) 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58d7fa640 1 -- 172.21.15.8:0/3985087336 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa5800f03f0 con 0x7fa590068ff0 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa594edb640 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3416396570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 msgr2=0x556d905afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3416396570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 msgr2=0x556d905a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58f7fe640 1 -- 172.21.15.8:0/3416396570 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d904ce0b0 con 0x556d905a4d30 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa594edb640 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58ffff640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa5780a2230 0x7fa5780a46f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa584009bd0 tx=0x7fa584005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ad2e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa5800ed8a0 tx=0x7fa580031c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a12803a60f852fd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa5767fc640 1 -- 172.21.15.8:0/3416396570 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800bc4a0 con 0x556d905a4d30 2024-02-19T16:27:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa5767fc640 1 -- 172.21.15.8:0/3416396570 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5800c21b0 con 0x556d905a4d30 2024-02-19T16:27:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.450+0000 7fa5767fc640 1 -- 172.21.15.8:0/3416396570 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800c1f30 con 0x556d905a4d30 2024-02-19T16:27:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 -- 172.21.15.8:0/3416396570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 msgr2=0x556d905ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ad2e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa5800ed8a0 tx=0x7fa580031c20 comp rx=0 tx=0).stop 2024-02-19T16:27:03.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 -- 172.21.15.8:0/3416396570 shutdown_connections 2024-02-19T16:27:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ad2e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 --2- 172.21.15.8:0/3416396570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 -- 172.21.15.8:0/3416396570 >> 172.21.15.8:0/3416396570 conn(0x556d9059e840 msgr2=0x556d9059ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 -- 172.21.15.8:0/3416396570 shutdown_connections 2024-02-19T16:27:03.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 -- 172.21.15.8:0/3416396570 wait complete. 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.451+0000 7fa5995fa740 1 -- start start 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 Processor -- start 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905aa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d904b0f40 con 0x556d905a4d30 2024-02-19T16:27:03.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d901df4b0 con 0x556d905a4380 2024-02-19T16:27:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d90491670 con 0x556d905ad8b0 2024-02-19T16:27:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa5995fa740 10 client.0 _ll_register_callbacks cb 0x556d904fef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44694/0 (socket says 172.21.15.8:44694) 2024-02-19T16:27:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1046306560 learned_addr learned my addr 172.21.15.8:0/1046306560 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa594edb640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 msgr2=0x556d905ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1046306560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 msgr2=0x556d905aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1046306560 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901a6aa0 con 0x556d905a4d30 2024-02-19T16:27:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.452+0000 7fa594edb640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905aadc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa5800ed950 tx=0x7fa5800c2220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ab7c627a5cfc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa56f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1046306560 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800c1bb0 con 0x556d905a4d30 2024-02-19T16:27:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1046306560 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa5800035f0 con 0x556d905a4d30 2024-02-19T16:27:03.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d904e5f40 con 0x556d905a4d30 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1046306560 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5800bc4a0 con 0x556d905a4d30 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 10 client.16608 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d9050a320 con 0x556d905a4d30 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1046306560 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d904e5f40 con 0x556d905a4d30 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1046306560 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa5800dc040 con 0x556d905a4d30 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 10 client.16608 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:03.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:03.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 20 client.16608 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:03.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d905bd6b0 0x556d905bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 4 client.16608 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:03.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d90502310 con 0x556d905bd6b0 2024-02-19T16:27:03.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.453+0000 7fa58ffff640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d905bd6b0 0x556d905bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.454+0000 7fa58ffff640 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d905bd6b0 0x556d905bfaa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f7781cbe020217 server_cookie=f33c0d1b0a146670 in_seq=0 out_seq=0 2024-02-19T16:27:03.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.454+0000 7fa56f7fe640 10 client.16608 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.454+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1046306560 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556d90502310 con 0x556d905bd6b0 2024-02-19T16:27:03.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.454+0000 7fa56f7fe640 10 client.16608 handle_command_reply: tid=0 2024-02-19T16:27:03.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 client.16608 shutdown 2024-02-19T16:27:03.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 msgr2=0x556d905aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905aadc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa5800ed950 tx=0x7fa5800c2220 comp rx=0 tx=0).stop 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 shutdown_connections 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d905bd6b0 0x556d905bfaa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa56f7fe640 0 client.16608 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905aadc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1046306560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 >> 172.21.15.8:0/1046306560 conn(0x556d9059e840 msgr2=0x556d905a07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 shutdown_connections 2024-02-19T16:27:03.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 1 -- 172.21.15.8:0/1046306560 wait complete. 2024-02-19T16:27:03.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.455+0000 7fa5995fa740 20 client.16608 trim_cache size 0 max 16384 2024-02-19T16:27:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 Processor -- start 2024-02-19T16:27:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 -- start start 2024-02-19T16:27:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d905a9c20 con 0x556d905a4d30 2024-02-19T16:27:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d90502340 con 0x556d905a4380 2024-02-19T16:27:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.471+0000 7fa5995fa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d905ba6b0 con 0x556d905ad8b0 2024-02-19T16:27:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44698/0 (socket says 172.21.15.8:44698) 2024-02-19T16:27:03.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 -- 172.21.15.8:0/278652564 learned_addr learned my addr 172.21.15.8:0/278652564 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa594edb640 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 -- 172.21.15.8:0/278652564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 msgr2=0x556d905b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 -- 172.21.15.8:0/278652564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 msgr2=0x556d905acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 -- 172.21.15.8:0/278652564 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d904a6c00 con 0x556d905a4d30 2024-02-19T16:27:03.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ab120 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa5800bc840 tx=0x7fa5800bb050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafa3105b36b5cc8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa575ffb640 1 -- 172.21.15.8:0/278652564 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800bc4a0 con 0x556d905a4d30 2024-02-19T16:27:03.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa575ffb640 1 -- 172.21.15.8:0/278652564 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5800035f0 con 0x556d905a4d30 2024-02-19T16:27:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.472+0000 7fa575ffb640 1 -- 172.21.15.8:0/278652564 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800f0030 con 0x556d905a4d30 2024-02-19T16:27:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- 172.21.15.8:0/278652564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 msgr2=0x556d905ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ab120 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa5800bc840 tx=0x7fa5800bb050 comp rx=0 tx=0).stop 2024-02-19T16:27:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- 172.21.15.8:0/278652564 shutdown_connections 2024-02-19T16:27:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905ab120 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- 172.21.15.8:0/278652564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- 172.21.15.8:0/278652564 >> 172.21.15.8:0/278652564 conn(0x556d9059e840 msgr2=0x556d9059d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- 172.21.15.8:0/278652564 shutdown_connections 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- 172.21.15.8:0/278652564 wait complete. 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- start start 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 Processor -- start 2024-02-19T16:27:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d90617da0 con 0x556d905a4d30 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d90617ee0 con 0x556d905a4380 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.473+0000 7fa5995fa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d90618020 con 0x556d905ad8b0 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa5995fa740 10 client.0 _ll_register_callbacks cb 0x7fa5800a7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44712/0 (socket says 172.21.15.8:44712) 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1767846349 learned_addr learned my addr 172.21.15.8:0/1767846349 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa594edb640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58ffff640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 msgr2=0x556d905b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1767846349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 msgr2=0x556d905b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 -- 172.21.15.8:0/1767846349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901a6aa0 con 0x556d905a4d30 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa58f7fe640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905b10b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa5800badf0 tx=0x7fa5800c2220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e9da9728751287 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa56f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1767846349 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800c5050 con 0x556d905a4d30 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1767846349 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa5800bcb40 con 0x556d905a4d30 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d90531900 con 0x556d905a4d30 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1767846349 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5800bc4a0 con 0x556d905a4d30 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa5995fa740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:03.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.474+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d904cc910 con 0x556d905a4d30 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1767846349 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d90531900 con 0x556d905a4d30 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1767846349 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa5800dc040 con 0x556d905a4d30 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa5995fa740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa5995fa740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa5995fa740 20 client.16614 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d906192a0 0x556d9061b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa5995fa740 4 client.16614 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556d9020b860 con 0x556d906192a0 2024-02-19T16:27:03.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa58ffff640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d906192a0 0x556d9061b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa56f7fe640 10 client.16614 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.475+0000 7fa58ffff640 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d906192a0 0x556d9061b690 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0d201713380e31 server_cookie=58afc6ac957b14ee in_seq=0 out_seq=0 2024-02-19T16:27:03.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.476+0000 7fa56f7fe640 1 -- 172.21.15.8:0/1767846349 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556d9020b860 con 0x556d906192a0 2024-02-19T16:27:03.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.476+0000 7fa56f7fe640 10 client.16614 handle_command_reply: tid=0 2024-02-19T16:27:03.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.476+0000 7fa5995fa740 1 client.16614 shutdown 2024-02-19T16:27:03.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 msgr2=0x556d905b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905b10b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa5800badf0 tx=0x7fa5800c2220 comp rx=0 tx=0).stop 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 shutdown_connections 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556d906192a0 0x556d9061b690 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d905ad8b0 0x556d905b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556d905a4d30 0x556d905b10b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa56f7fe640 0 client.16614 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 --2- 172.21.15.8:0/1767846349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d905a4380 0x556d905b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 >> 172.21.15.8:0/1767846349 conn(0x556d9059e840 msgr2=0x556d9059d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 shutdown_connections 2024-02-19T16:27:03.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 1 -- 172.21.15.8:0/1767846349 wait complete. 2024-02-19T16:27:03.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa5995fa740 20 client.16614 trim_cache size 0 max 16384 2024-02-19T16:27:03.494 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:27:03.495 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:27:03.495 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "50f57503-7198-471d-b2a5-4b5544919839", 2024-02-19T16:27:03.495 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:27:03.495 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:27:03.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa5780a2230 msgr2=0x7fa5780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa59693e640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa5780a2230 0x7fa5780a46f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa584009bd0 tx=0x7fa584005f90 comp rx=0 tx=0).stop 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 msgr2=0x7fa59019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.477+0000 7fa59693e640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59019a290 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa580005940 tx=0x7fa58002c670 comp rx=0 tx=0).stop 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 shutdown_connections 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa59010de90 0x7fa59019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa590068ff0 0x7fa59019a290 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa5780a2230 0x7fa5780a46f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 --2- 172.21.15.8:0/3985087336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5900686d0 0x7fa590199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 >> 172.21.15.8:0/3985087336 conn(0x7fa5900774f0 msgr2=0x7fa59010f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 shutdown_connections 2024-02-19T16:27:03.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.478+0000 7fa59693e640 1 -- 172.21.15.8:0/3985087336 wait complete. 2024-02-19T16:27:03.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:27:03.498 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:27:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 -- 172.21.15.8:0/3916807032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac100f90 msgr2=0x7fd8ac104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 --2- 172.21.15.8:0/3916807032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac100f90 0x7fd8ac104fd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009930 tx=0x7fd89c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 -- 172.21.15.8:0/3916807032 shutdown_connections 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 --2- 172.21.15.8:0/3916807032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1056b0 0x7fd8ac10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 --2- 172.21.15.8:0/3916807032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac100f90 0x7fd8ac104fd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 --2- 172.21.15.8:0/3916807032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac1005e0 0x7fd8ac1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 -- 172.21.15.8:0/3916807032 >> 172.21.15.8:0/3916807032 conn(0x7fd8ac0775a0 msgr2=0x7fd8ac0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 -- 172.21.15.8:0/3916807032 shutdown_connections 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 -- 172.21.15.8:0/3916807032 wait complete. 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 Processor -- start 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.606+0000 7fd8b478a640 1 -- start start 2024-02-19T16:27:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b478a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1005e0 0x7fd8ac19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b478a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac1056b0 0x7fd8ac19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b478a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 0x7fd8ac197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b478a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1982f0 con 0x7fd8ac197900 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b478a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac198460 con 0x7fd8ac1056b0 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b478a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac198620 con 0x7fd8ac1005e0 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 0x7fd8ac197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1005e0 0x7fd8ac19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1005e0 0x7fd8ac19d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:38800/0 (socket says 172.21.15.8:38800) 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b1d25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac1056b0 0x7fd8ac19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 0x7fd8ac197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44724/0 (socket says 172.21.15.8:44724) 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2526640 1 -- 172.21.15.8:0/741503003 learned_addr learned my addr 172.21.15.8:0/741503003 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 -- 172.21.15.8:0/741503003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1005e0 msgr2=0x7fd8ac19d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1005e0 0x7fd8ac19d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 -- 172.21.15.8:0/741503003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac1056b0 msgr2=0x7fd8ac19dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac1056b0 0x7fd8ac19dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 -- 172.21.15.8:0/741503003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8009490 con 0x7fd8ac197900 2024-02-19T16:27:03.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.607+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 0x7fd8ac197db0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009fd0 tx=0x7fd8a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.608+0000 7fd8a37fe640 1 -- 172.21.15.8:0/741503003 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400ce20 con 0x7fd8ac197900 2024-02-19T16:27:03.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.608+0000 7fd8a37fe640 1 -- 172.21.15.8:0/741503003 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a4005020 con 0x7fd8ac197900 2024-02-19T16:27:03.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.608+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c009560 con 0x7fd8ac197900 2024-02-19T16:27:03.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.608+0000 7fd8a37fe640 1 -- 172.21.15.8:0/741503003 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4010690 con 0x7fd8ac197900 2024-02-19T16:27:03.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.608+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac1a49b0 con 0x7fd8ac197900 2024-02-19T16:27:03.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.610+0000 7fd8a37fe640 1 -- 172.21.15.8:0/741503003 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fd8a4005500 con 0x7fd8ac197900 2024-02-19T16:27:03.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.610+0000 7fd8b7364740 1 Processor -- start 2024-02-19T16:27:03.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.610+0000 7fd8b7364740 1 -- start start 2024-02-19T16:27:03.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8a37fe640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8980a22b0 0x7fd8980a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890babd0 0x55e4890c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890c37b0 0x55e4890c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b7364740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e489029280 con 0x55e4890babd0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b7364740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4890267b0 con 0x55e4890ba2b0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b7364740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e489021f90 con 0x55e4890c37b0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8a37fe640 1 -- 172.21.15.8:0/741503003 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd8a4014070 con 0x7fd8ac197900 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b1d25640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8980a22b0 0x7fd8980a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890c37b0 0x55e4890c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b1d25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890babd0 0x55e4890c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890c37b0 0x55e4890c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:38806/0 (socket says 172.21.15.8:38806) 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2d27640 1 -- 172.21.15.8:0/3433583341 learned_addr learned my addr 172.21.15.8:0/3433583341 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2526640 1 -- 172.21.15.8:0/3433583341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890c37b0 msgr2=0x55e4890c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2526640 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890c37b0 0x55e4890c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2526640 1 -- 172.21.15.8:0/3433583341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890babd0 msgr2=0x55e4890c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2526640 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890babd0 0x55e4890c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b2526640 1 -- 172.21.15.8:0/3433583341 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e488fe0570 con 0x55e4890ba2b0 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b1d25640 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890babd0 0x55e4890c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:03.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.611+0000 7fd8b1d25640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8980a22b0 0x7fd8980a4770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000c00 tx=0x7fd89c004750 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.612+0000 7fd8b2526640 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890ba690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009e40 tx=0x7fd8a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f3167ee8ca437c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.612+0000 7fd8a0ff9640 1 -- 172.21.15.8:0/3433583341 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005020 con 0x55e4890ba2b0 2024-02-19T16:27:03.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.612+0000 7fd8a0ff9640 1 -- 172.21.15.8:0/3433583341 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8005660 con 0x55e4890ba2b0 2024-02-19T16:27:03.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.612+0000 7fd8a0ff9640 1 -- 172.21.15.8:0/3433583341 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800f6e0 con 0x55e4890ba2b0 2024-02-19T16:27:03.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.612+0000 7fd8b7364740 1 -- 172.21.15.8:0/3433583341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 msgr2=0x55e4890ba690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.612+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890ba690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009e40 tx=0x7fd8a8011810 comp rx=0 tx=0).stop 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- 172.21.15.8:0/3433583341 shutdown_connections 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890c37b0 0x55e4890c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890babd0 0x55e4890c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3433583341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890ba690 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- 172.21.15.8:0/3433583341 >> 172.21.15.8:0/3433583341 conn(0x55e4890b4990 msgr2=0x55e4890b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- 172.21.15.8:0/3433583341 shutdown_connections 2024-02-19T16:27:03.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- 172.21.15.8:0/3433583341 wait complete. 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- start start 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 Processor -- start 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488cf5430 con 0x55e4890c37b0 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488cb1e00 con 0x55e4890ba2b0 2024-02-19T16:27:03.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488fa6fd0 con 0x55e4890babd0 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.613+0000 7fd8b7364740 10 client.0 _ll_register_callbacks cb 0x55e488ffcfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b1d25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48630/0 (socket says 172.21.15.8:48630) 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 -- 172.21.15.8:0/600534192 learned_addr learned my addr 172.21.15.8:0/600534192 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b1d25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:38814/0 (socket says 172.21.15.8:38814) 2024-02-19T16:27:03.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 -- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 msgr2=0x55e4890c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 -- 172.21.15.8:0/600534192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 msgr2=0x55e4890c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 -- 172.21.15.8:0/600534192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a8009aa0 con 0x55e4890ba2b0 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b2526640 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890c31f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801b040 tx=0x7fd8a80050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998df4388703eae2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd895ffb640 1 -- 172.21.15.8:0/600534192 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80054e0 con 0x55e4890ba2b0 2024-02-19T16:27:03.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e488d217e0 con 0x55e4890ba2b0 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.614+0000 7fd895ffb640 1 -- 172.21.15.8:0/600534192 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd8a8004020 con 0x55e4890ba2b0 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd895ffb640 1 -- 172.21.15.8:0/600534192 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a80041e0 con 0x55e4890ba2b0 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 10 client.26396 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e489020610 con 0x55e4890ba2b0 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd895ffb640 1 -- 172.21.15.8:0/600534192 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e488d217e0 con 0x55e4890ba2b0 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd895ffb640 1 -- 172.21.15.8:0/600534192 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd8a8024460 con 0x55e4890ba2b0 2024-02-19T16:27:03.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 10 client.26396 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 20 client.26396 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e4890d7d40 0x55e4890da160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 4 client.26396 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.615+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e488d5e720 con 0x55e4890d7d40 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.616+0000 7fd8b1d25640 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e4890d7d40 0x55e4890da160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.616+0000 7fd8b1d25640 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e4890d7d40 0x55e4890da160 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372f8041c25d94fc server_cookie=803d3ffa6973bb59 in_seq=0 out_seq=0 2024-02-19T16:27:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.616+0000 7fd895ffb640 10 client.26396 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd895ffb640 1 -- 172.21.15.8:0/600534192 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e488d5e720 con 0x55e4890d7d40 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd895ffb640 10 client.26396 handle_command_reply: tid=0 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 client.26396 shutdown 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 msgr2=0x55e4890c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890c31f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801b040 tx=0x7fd8a80050e0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 shutdown_connections 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e4890d7d40 0x55e4890da160 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd895ffb640 0 client.26396 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 --2- 172.21.15.8:0/600534192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890c31f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 >> 172.21.15.8:0/600534192 conn(0x55e4890b4990 msgr2=0x55e4890b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 shutdown_connections 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.617+0000 7fd8b7364740 1 -- 172.21.15.8:0/600534192 wait complete. 2024-02-19T16:27:03.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.618+0000 7fd8b7364740 20 client.26396 trim_cache size 0 max 16384 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 Processor -- start 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 -- start start 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4890b41d0 con 0x55e4890c37b0 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4890b59e0 con 0x55e4890ba2b0 2024-02-19T16:27:03.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.636+0000 7fd8b7364740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4890b5790 con 0x55e4890babd0 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44734/0 (socket says 172.21.15.8:44734) 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 -- 172.21.15.8:0/4036600588 learned_addr learned my addr 172.21.15.8:0/4036600588 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b1d25640 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48632/0 (socket says 172.21.15.8:48632) 2024-02-19T16:27:03.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 -- 172.21.15.8:0/4036600588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 msgr2=0x55e4890c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 -- 172.21.15.8:0/4036600588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 msgr2=0x55e4890b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 -- 172.21.15.8:0/4036600588 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e488fbcac0 con 0x55e4890c37b0 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2526640 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c31f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40cc440 tx=0x7fd8a4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd3f8654d621d10 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd897fff640 1 -- 172.21.15.8:0/4036600588 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b0070 con 0x55e4890c37b0 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd897fff640 1 -- 172.21.15.8:0/4036600588 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a409b7e0 con 0x55e4890c37b0 2024-02-19T16:27:03.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.637+0000 7fd897fff640 1 -- 172.21.15.8:0/4036600588 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a404a2d0 con 0x55e4890c37b0 2024-02-19T16:27:03.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 -- 172.21.15.8:0/4036600588 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 msgr2=0x55e4890c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c31f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40cc440 tx=0x7fd8a4005f70 comp rx=0 tx=0).stop 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 -- 172.21.15.8:0/4036600588 shutdown_connections 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890c31f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890babd0 0x55e4890c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 --2- 172.21.15.8:0/4036600588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890ba2b0 0x55e4890b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 -- 172.21.15.8:0/4036600588 >> 172.21.15.8:0/4036600588 conn(0x55e4890b4990 msgr2=0x55e4890b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 -- 172.21.15.8:0/4036600588 shutdown_connections 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 -- 172.21.15.8:0/4036600588 wait complete. 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.638+0000 7fd8b7364740 1 -- start start 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 Processor -- start 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890ba2b0 0x55e4890be2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890babd0 0x55e4890caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4890cb680 con 0x55e4890c37b0 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4890cb7f0 con 0x55e4890babd0 2024-02-19T16:27:03.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e489133630 con 0x55e4890ba2b0 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b7364740 10 client.0 _ll_register_callbacks cb 0x7fd8a40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44744/0 (socket says 172.21.15.8:44744) 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b1d25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890babd0 0x55e4890caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2526640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890ba2b0 0x55e4890be2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 -- 172.21.15.8:0/3916060992 learned_addr learned my addr 172.21.15.8:0/3916060992 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 -- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890ba2b0 msgr2=0x55e4890be2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890ba2b0 0x55e4890be2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 -- 172.21.15.8:0/3916060992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890babd0 msgr2=0x55e4890caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890babd0 0x55e4890caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 -- 172.21.15.8:0/3916060992 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e489020610 con 0x55e4890c37b0 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd8b2d27640 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890caf80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd8a409ade0 tx=0x7fd8a40a3620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d4d537aca35f6a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.639+0000 7fd895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd895ffb640 1 -- 172.21.15.8:0/3916060992 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b0070 con 0x55e4890c37b0 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd895ffb640 1 -- 172.21.15.8:0/3916060992 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd8a409b0e0 con 0x55e4890c37b0 2024-02-19T16:27:03.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8a8005660 con 0x55e4890c37b0 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd895ffb640 1 -- 172.21.15.8:0/3916060992 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a40a3880 con 0x55e4890c37b0 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd8ac1a49b0 con 0x55e4890c37b0 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd895ffb640 1 -- 172.21.15.8:0/3916060992 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8005660 con 0x55e4890c37b0 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd895ffb640 1 -- 172.21.15.8:0/3916060992 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd8a401f070 con 0x55e4890c37b0 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:03.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 20 client.16632 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e489134b30 0x55e489136f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 4 client.16632 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e488dad890 con 0x55e489134b30 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.640+0000 7fd8b2526640 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e489134b30 0x55e489136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.641+0000 7fd8b2526640 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e489134b30 0x55e489136f20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6958169107046d37 server_cookie=a06049cfa19f851 in_seq=0 out_seq=0 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.641+0000 7fd895ffb640 10 client.16632 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.641+0000 7fd895ffb640 1 -- 172.21.15.8:0/3916060992 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55e488dad890 con 0x55e489134b30 2024-02-19T16:27:03.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.641+0000 7fd895ffb640 10 client.16632 handle_command_reply: tid=0 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.641+0000 7fd8b7364740 1 client.16632 shutdown 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 msgr2=0x55e4890caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890caf80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd8a409ade0 tx=0x7fd8a40a3620 comp rx=0 tx=0).stop 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 shutdown_connections 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e4890c37b0 0x55e4890caf80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4890babd0 0x55e4890caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e489134b30 0x55e489136f20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 --2- 172.21.15.8:0/3916060992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4890ba2b0 0x55e4890be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd895ffb640 0 client.16632 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:03.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 >> 172.21.15.8:0/3916060992 conn(0x55e4890b4990 msgr2=0x55e4890b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 shutdown_connections 2024-02-19T16:27:03.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 1 -- 172.21.15.8:0/3916060992 wait complete. 2024-02-19T16:27:03.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b7364740 20 client.16632 trim_cache size 0 max 16384 2024-02-19T16:27:03.653 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2605 inodes in the stack)", 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: "50f57503-7198-471d-b2a5-4b5544919839": { 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "50f57503-7198-471d-b2a5-4b5544919839", 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:27:03.654 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:27:03.655 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:27:03.655 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:27:03.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8980a22b0 msgr2=0x7fd8980a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b478a640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8980a22b0 0x7fd8980a4770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000c00 tx=0x7fd89c004750 comp rx=0 tx=0).stop 2024-02-19T16:27:03.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 msgr2=0x7fd8ac197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:03.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.642+0000 7fd8b478a640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 0x7fd8ac197db0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009fd0 tx=0x7fd8a400ef10 comp rx=0 tx=0).stop 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 shutdown_connections 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8ac197900 0x7fd8ac197db0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8ac1056b0 0x7fd8ac19dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd8980a22b0 0x7fd8980a4770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 --2- 172.21.15.8:0/741503003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8ac1005e0 0x7fd8ac19d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 >> 172.21.15.8:0/741503003 conn(0x7fd8ac0775a0 msgr2=0x7fd8ac0689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 shutdown_connections 2024-02-19T16:27:03.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:03.643+0000 7fd8b478a640 1 -- 172.21.15.8:0/741503003 wait complete. 2024-02-19T16:27:03.658 INFO:tasks.cephfs.filesystem:scrub status for tag:50f57503-7198-471d-b2a5-4b5544919839 - {'path': '/', 'tag': '50f57503-7198-471d-b2a5-4b5544919839', 'options': 'recursive,force'} 2024-02-19T16:27:04.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:03 smithi008 ceph-mon[32573]: pgmap v128: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:03 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:03 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:03 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:03 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:04.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:03 smithi191 ceph-mon[37203]: pgmap v128: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:04.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:03 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:04.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:03 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:04.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:03 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:04.029 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:03 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:04.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:03 smithi156 ceph-mon[36502]: pgmap v128: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:03 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:03 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:03 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:03 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:27:05.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:05.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:05.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:05.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:04 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:27:05.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:05.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:05.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:05.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:04 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:27:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:05.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:05.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:04 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:06.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:05 smithi008 ceph-mon[32573]: pgmap v129: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:06.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:05 smithi008 ceph-mon[32573]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:27:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:05 smithi191 ceph-mon[37203]: pgmap v129: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:05 smithi191 ceph-mon[37203]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:27:06.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:05 smithi156 ceph-mon[36502]: pgmap v129: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:06.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:05 smithi156 ceph-mon[36502]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-19T16:27:08.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:07 smithi008 ceph-mon[32573]: pgmap v130: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:08.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:07 smithi191 ceph-mon[37203]: pgmap v130: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:08.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:07 smithi156 ceph-mon[36502]: pgmap v130: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:10.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:09 smithi008 ceph-mon[32573]: pgmap v131: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:09 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:27:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:09 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:27:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:09 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:27:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:09 smithi191 ceph-mon[37203]: pgmap v131: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:09 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:27:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:09 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:27:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:09 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:27:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:09 smithi156 ceph-mon[36502]: pgmap v131: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:09 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:27:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:09 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:27:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:09 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:27:12.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:11 smithi191 ceph-mon[37203]: pgmap v132: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:12.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:11 smithi156 ceph-mon[36502]: pgmap v132: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:12.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:11 smithi008 ceph-mon[32573]: pgmap v132: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:13 smithi191 ceph-mon[37203]: pgmap v133: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:14.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:13 smithi156 ceph-mon[36502]: pgmap v133: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:13 smithi008 ceph-mon[32573]: pgmap v133: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:27:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:15 smithi191 ceph-mon[37203]: pgmap v134: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:16.072 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:15 smithi008 ceph-mon[32573]: pgmap v134: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:16.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:15 smithi156 ceph-mon[36502]: pgmap v134: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:27:18.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:17 smithi156 ceph-mon[36502]: pgmap v135: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:27:18.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:17 smithi008 ceph-mon[32573]: pgmap v135: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:27:18.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:17 smithi191 ceph-mon[37203]: pgmap v135: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:27:20.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:19 smithi156 ceph-mon[36502]: pgmap v136: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:27:20.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:19 smithi008 ceph-mon[32573]: pgmap v136: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:27:20.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:19 smithi191 ceph-mon[37203]: pgmap v136: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:27:22.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:21 smithi156 ceph-mon[36502]: pgmap v137: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:27:22.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:21 smithi008 ceph-mon[32573]: pgmap v137: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:27:22.279 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:21 smithi191 ceph-mon[37203]: pgmap v137: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:27:24.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:23 smithi156 ceph-mon[36502]: pgmap v138: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:27:24.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:23 smithi008 ceph-mon[32573]: pgmap v138: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:27:24.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:23 smithi191 ceph-mon[37203]: pgmap v138: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:27:26.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:25 smithi156 ceph-mon[36502]: pgmap v139: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:27:26.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:25 smithi008 ceph-mon[32573]: pgmap v139: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:27:26.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:25 smithi191 ceph-mon[37203]: pgmap v139: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:27:28.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:27 smithi156 ceph-mon[36502]: pgmap v140: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:28.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:27 smithi008 ceph-mon[32573]: pgmap v140: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:28.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:27 smithi191 ceph-mon[37203]: pgmap v140: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:30.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:29 smithi156 ceph-mon[36502]: pgmap v141: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:30.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:29 smithi008 ceph-mon[32573]: pgmap v141: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:30.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:29 smithi191 ceph-mon[37203]: pgmap v141: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:32.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:31 smithi156 ceph-mon[36502]: pgmap v142: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:32.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:31 smithi008 ceph-mon[32573]: pgmap v142: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:32.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:31 smithi191 ceph-mon[37203]: pgmap v142: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:33.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:27:33.659 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:27:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 -- 172.21.15.8:0/4006990856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 msgr2=0x7f04d4102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 --2- 172.21.15.8:0/4006990856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d4102af0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009900 tx=0x7f04c402bcd0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 -- 172.21.15.8:0/4006990856 shutdown_connections 2024-02-19T16:27:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 --2- 172.21.15.8:0/4006990856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04d410ba50 0x7f04d410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 --2- 172.21.15.8:0/4006990856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4103030 0x7f04d410b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 --2- 172.21.15.8:0/4006990856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d4102af0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 -- 172.21.15.8:0/4006990856 >> 172.21.15.8:0/4006990856 conn(0x7f04d40775a0 msgr2=0x7f04d40779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 -- 172.21.15.8:0/4006990856 shutdown_connections 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.778+0000 7f04d9302640 1 -- 172.21.15.8:0/4006990856 wait complete. 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 Processor -- start 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 -- start start 2024-02-19T16:27:33.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4103030 0x7f04d419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04d410ba50 0x7f04d4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4197f80 con 0x7f04d4102710 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d41980f0 con 0x7f04d4103030 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d9302640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d41982b0 con 0x7f04d410ba50 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49516/0 (socket says 172.21.15.8:49516) 2024-02-19T16:27:33.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 -- 172.21.15.8:0/3106017611 learned_addr learned my addr 172.21.15.8:0/3106017611 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04cbfff640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4103030 0x7f04d419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d37fe640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04d410ba50 0x7f04d4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 -- 172.21.15.8:0/3106017611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04d410ba50 msgr2=0x7f04d4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04d410ba50 0x7f04d4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 -- 172.21.15.8:0/3106017611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4103030 msgr2=0x7f04d419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4103030 0x7f04d419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 -- 172.21.15.8:0/3106017611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4009560 con 0x7f04d4102710 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.779+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d419d780 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f04c4005940 tx=0x7f04c402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.780+0000 7f04d0ff9640 1 -- 172.21.15.8:0/3106017611 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004350 con 0x7f04d4102710 2024-02-19T16:27:33.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.780+0000 7f04d0ff9640 1 -- 172.21.15.8:0/3106017611 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04c40044b0 con 0x7f04d4102710 2024-02-19T16:27:33.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.780+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d4198530 con 0x7f04d4102710 2024-02-19T16:27:33.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.780+0000 7f04d0ff9640 1 -- 172.21.15.8:0/3106017611 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4003d30 con 0x7f04d4102710 2024-02-19T16:27:33.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.780+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d41a4820 con 0x7f04d4102710 2024-02-19T16:27:33.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.782+0000 7f04d0ff9640 1 -- 172.21.15.8:0/3106017611 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f04c4004df0 con 0x7f04d4102710 2024-02-19T16:27:33.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.782+0000 7f04dbfb8740 1 Processor -- start 2024-02-19T16:27:33.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.782+0000 7f04dbfb8740 1 -- start start 2024-02-19T16:27:33.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d0ff9640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04c00a22b0 0x7f04c00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc720690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc729270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc72bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc68f280 con 0x55f5dc7297b0 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc68c7b0 con 0x55f5dc720bd0 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc687f90 con 0x55f5dc7202b0 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d37fe640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04c00a22b0 0x7f04c00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc720690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d0ff9640 1 -- 172.21.15.8:0/3106017611 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f04c40f09a0 con 0x7f04d4102710 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04cbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc729270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc720690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:37730/0 (socket says 172.21.15.8:37730) 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 -- 172.21.15.8:0/2895841407 learned_addr learned my addr 172.21.15.8:0/2895841407 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d37fe640 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc72bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 -- 172.21.15.8:0/2895841407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 msgr2=0x55f5dc729270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc729270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 -- 172.21.15.8:0/2895841407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 msgr2=0x55f5dc72bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc72bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 -- 172.21.15.8:0/2895841407 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5dc646570 con 0x55f5dc7202b0 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d37fe640 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc72bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d37fe640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04c00a22b0 0x7f04c00a4770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55f5dc71bb70 tx=0x7f04cc009b70 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.783+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc720690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04c40c5300 tx=0x7f04c40bcbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=302a3d20897e0199 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04c97fa640 1 -- 172.21.15.8:0/2895841407 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40cfac0 con 0x55f5dc7202b0 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04c97fa640 1 -- 172.21.15.8:0/2895841407 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04c40cf8e0 con 0x55f5dc7202b0 2024-02-19T16:27:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04c97fa640 1 -- 172.21.15.8:0/2895841407 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40bd120 con 0x55f5dc7202b0 2024-02-19T16:27:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 -- 172.21.15.8:0/2895841407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 msgr2=0x55f5dc720690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc720690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04c40c5300 tx=0x7f04c40bcbd0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 -- 172.21.15.8:0/2895841407 shutdown_connections 2024-02-19T16:27:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc72bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc729270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/2895841407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc720690 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 -- 172.21.15.8:0/2895841407 >> 172.21.15.8:0/2895841407 conn(0x55f5dc71a990 msgr2=0x55f5dc71ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 -- 172.21.15.8:0/2895841407 shutdown_connections 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.784+0000 7f04dbfb8740 1 -- 172.21.15.8:0/2895841407 wait complete. 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 -- start start 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 Processor -- start 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc7290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc726450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc726990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc35b430 con 0x55f5dc7297b0 2024-02-19T16:27:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc317e00 con 0x55f5dc720bd0 2024-02-19T16:27:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc60cfd0 con 0x55f5dc7202b0 2024-02-19T16:27:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04dbfb8740 10 client.0 _ll_register_callbacks cb 0x55f5dc662fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc726990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc7290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc726990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49530/0 (socket says 172.21.15.8:49530) 2024-02-19T16:27:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 -- 172.21.15.8:0/108007277 learned_addr learned my addr 172.21.15.8:0/108007277 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04cbfff640 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc726450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 -- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 msgr2=0x55f5dc7290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc7290c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 -- 172.21.15.8:0/108007277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 msgr2=0x55f5dc726450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc726450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.785+0000 7f04d37fe640 1 -- 172.21.15.8:0/108007277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d41a4820 con 0x55f5dc7297b0 2024-02-19T16:27:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04d37fe640 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc726990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00fb60 tx=0x7f04cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f20452262b5fb3c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04ba7fc640 1 -- 172.21.15.8:0/108007277 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004760 con 0x55f5dc7297b0 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04ba7fc640 1 -- 172.21.15.8:0/108007277 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f04cc029460 con 0x55f5dc7297b0 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5dc3877e0 con 0x55f5dc7297b0 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04ba7fc640 1 -- 172.21.15.8:0/108007277 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04cc029620 con 0x55f5dc7297b0 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 10 client.16644 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5dc686610 con 0x55f5dc7297b0 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04ba7fc640 1 -- 172.21.15.8:0/108007277 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f5dc3877e0 con 0x55f5dc7297b0 2024-02-19T16:27:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04ba7fc640 1 -- 172.21.15.8:0/108007277 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04cc01e470 con 0x55f5dc7297b0 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 10 client.16644 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 20 client.16644 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc7395b0 0x55f5dc73b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 4 client.16644 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.786+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5dc3c4720 con 0x55f5dc7395b0 2024-02-19T16:27:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.787+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc7395b0 0x55f5dc73b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.787+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc7395b0 0x55f5dc73b9d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc631ad3ab630bb6 server_cookie=2e5d211a0edc925f in_seq=0 out_seq=0 2024-02-19T16:27:33.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.787+0000 7f04ba7fc640 10 client.16644 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:33.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04ba7fc640 1 -- 172.21.15.8:0/108007277 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f5dc3c4720 con 0x55f5dc7395b0 2024-02-19T16:27:33.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04ba7fc640 10 client.16644 handle_command_reply: tid=0 2024-02-19T16:27:33.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 client.16644 shutdown 2024-02-19T16:27:33.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 msgr2=0x55f5dc726990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc726990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00fb60 tx=0x7f04cc004ab0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 shutdown_connections 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7297b0 0x55f5dc726990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc726450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc7395b0 0x55f5dc73b9d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/108007277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7202b0 0x55f5dc7290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04ba7fc640 0 client.16644 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:33.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 >> 172.21.15.8:0/108007277 conn(0x55f5dc71a990 msgr2=0x55f5dc71c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 shutdown_connections 2024-02-19T16:27:33.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 1 -- 172.21.15.8:0/108007277 wait complete. 2024-02-19T16:27:33.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.788+0000 7f04dbfb8740 20 client.16644 trim_cache size 0 max 16384 2024-02-19T16:27:33.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 Processor -- start 2024-02-19T16:27:33.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 -- start start 2024-02-19T16:27:33.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc71c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc720bd0 0x55f5dc728b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc7297b0 0x55f5dc7290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc71b9e0 con 0x55f5dc7202b0 2024-02-19T16:27:33.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc735ff0 con 0x55f5dc7297b0 2024-02-19T16:27:33.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc71ff10 con 0x55f5dc720bd0 2024-02-19T16:27:33.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc71c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc71c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49538/0 (socket says 172.21.15.8:49538) 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 -- 172.21.15.8:0/839875499 learned_addr learned my addr 172.21.15.8:0/839875499 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d37fe640 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc7297b0 0x55f5dc7290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04cbfff640 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc720bd0 0x55f5dc728b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 -- 172.21.15.8:0/839875499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc720bd0 msgr2=0x55f5dc728b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc720bd0 0x55f5dc728b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 -- 172.21.15.8:0/839875499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc7297b0 msgr2=0x55f5dc7290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc7297b0 0x55f5dc7290c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.805+0000 7f04d2ffd640 1 -- 172.21.15.8:0/839875499 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5dc622ac0 con 0x55f5dc7202b0 2024-02-19T16:27:33.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.806+0000 7f04d37fe640 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc7297b0 0x55f5dc7290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:33.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.806+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc71c570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f04c402c820 tx=0x7f04c4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc98ad3c24cbde3a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.806+0000 7f04c8ff9640 1 -- 172.21.15.8:0/839875499 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40035f0 con 0x55f5dc7202b0 2024-02-19T16:27:33.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.806+0000 7f04c8ff9640 1 -- 172.21.15.8:0/839875499 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04c40bd790 con 0x55f5dc7202b0 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.806+0000 7f04c8ff9640 1 -- 172.21.15.8:0/839875499 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40bb390 con 0x55f5dc7202b0 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- 172.21.15.8:0/839875499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 msgr2=0x55f5dc71c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc71c570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f04c402c820 tx=0x7f04c4003f10 comp rx=0 tx=0).stop 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- 172.21.15.8:0/839875499 shutdown_connections 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc7297b0 0x55f5dc7290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc720bd0 0x55f5dc728b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/839875499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc71c570 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- 172.21.15.8:0/839875499 >> 172.21.15.8:0/839875499 conn(0x55f5dc71a990 msgr2=0x55f5dc725ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- 172.21.15.8:0/839875499 shutdown_connections 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- 172.21.15.8:0/839875499 wait complete. 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- start start 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 Processor -- start 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc72b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc7226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7297b0 0x55f5dc722c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc7232f0 con 0x55f5dc7202b0 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc723460 con 0x55f5dc720bd0 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5dc723620 con 0x55f5dc7297b0 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04dbfb8740 10 client.0 _ll_register_callbacks cb 0x7f04cc009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc72b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc72b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49542/0 (socket says 172.21.15.8:49542) 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.807+0000 7f04d2ffd640 1 -- 172.21.15.8:0/960214668 learned_addr learned my addr 172.21.15.8:0/960214668 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04cbfff640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc7226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d37fe640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7297b0 0x55f5dc722c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d2ffd640 1 -- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7297b0 msgr2=0x55f5dc722c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7297b0 0x55f5dc722c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d2ffd640 1 -- 172.21.15.8:0/960214668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 msgr2=0x55f5dc7226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc7226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d2ffd640 1 -- 172.21.15.8:0/960214668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5dc686610 con 0x55f5dc7202b0 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d37fe640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7297b0 0x55f5dc722c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04d2ffd640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc72b470 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f04c4000c00 tx=0x7f04c40bd090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1117cac31da2b88d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04ba7fc640 1 -- 172.21.15.8:0/960214668 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40035f0 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04ba7fc640 1 -- 172.21.15.8:0/960214668 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f04c402ce40 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04c40cf8e0 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04ba7fc640 1 -- 172.21.15.8:0/960214668 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c40c11e0 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 10 client.16650 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5dc646570 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04ba7fc640 1 -- 172.21.15.8:0/960214668 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04c40cf8e0 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04ba7fc640 1 -- 172.21.15.8:0/960214668 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04c403c070 con 0x55f5dc7202b0 2024-02-19T16:27:33.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 10 client.16650 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 20 client.16650 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc796160 0x55f5dc798550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 4 client.16650 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.808+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5dc413890 con 0x55f5dc796160 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04cbfff640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc796160 0x55f5dc798550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04cbfff640 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc796160 0x55f5dc798550 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e7728c588662e1 server_cookie=8c849473f64ca136 in_seq=0 out_seq=0 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04ba7fc640 10 client.16650 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:33.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04ba7fc640 1 -- 172.21.15.8:0/960214668 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f5dc413890 con 0x55f5dc796160 2024-02-19T16:27:33.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04ba7fc640 10 client.16650 handle_command_reply: tid=0 2024-02-19T16:27:33.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 client.16650 shutdown 2024-02-19T16:27:33.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 msgr2=0x55f5dc72b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc72b470 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f04c4000c00 tx=0x7f04c40bd090 comp rx=0 tx=0).stop 2024-02-19T16:27:33.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 shutdown_connections 2024-02-19T16:27:33.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f5dc796160 0x55f5dc798550 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dc7297b0 0x55f5dc722c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5dc720bd0 0x55f5dc7226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04ba7fc640 0 client.16650 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 --2- 172.21.15.8:0/960214668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f5dc7202b0 0x55f5dc72b470 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 >> 172.21.15.8:0/960214668 conn(0x55f5dc71a990 msgr2=0x55f5dc7191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 shutdown_connections 2024-02-19T16:27:33.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.809+0000 7f04dbfb8740 1 -- 172.21.15.8:0/960214668 wait complete. 2024-02-19T16:27:33.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04dbfb8740 20 client.16650 trim_cache size 0 max 16384 2024-02-19T16:27:33.825 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:27:33.825 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:27:33.825 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:27:33.825 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:27:33.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04c00a22b0 msgr2=0x7f04c00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04c00a22b0 0x7f04c00a4770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55f5dc71bb70 tx=0x7f04cc009b70 comp rx=0 tx=0).stop 2024-02-19T16:27:33.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 msgr2=0x7f04d419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d419d780 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f04c4005940 tx=0x7f04c402c710 comp rx=0 tx=0).stop 2024-02-19T16:27:33.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 shutdown_connections 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04c00a22b0 0x7f04c00a4770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04d410ba50 0x7f04d4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4103030 0x7f04d419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 --2- 172.21.15.8:0/3106017611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04d4102710 0x7f04d419d780 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 >> 172.21.15.8:0/3106017611 conn(0x7f04d40775a0 msgr2=0x7f04d40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 shutdown_connections 2024-02-19T16:27:33.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.810+0000 7f04d9302640 1 -- 172.21.15.8:0/3106017611 wait complete. 2024-02-19T16:27:33.829 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:27:33.829 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:27:33.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 -- 172.21.15.8:0/1267120117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4106880 msgr2=0x7f67a410e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/1267120117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4106880 0x7f67a410e400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009be0 tx=0x7f67a002bef0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 -- 172.21.15.8:0/1267120117 shutdown_connections 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/1267120117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4106880 0x7f67a410e400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/1267120117 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a406a7a0 0x7f67a4106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/1267120117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a4069df0 0x7f67a406a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.930+0000 7f67ab2ed640 1 -- 172.21.15.8:0/1267120117 >> 172.21.15.8:0/1267120117 conn(0x7f67a4077490 msgr2=0x7f67a40778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 -- 172.21.15.8:0/1267120117 shutdown_connections 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 -- 172.21.15.8:0/1267120117 wait complete. 2024-02-19T16:27:33.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 Processor -- start 2024-02-19T16:27:33.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 -- start start 2024-02-19T16:27:33.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 0x7f67a4100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a4106880 0x7f67a4100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4100ea0 0x7f67a41a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41a61b0 con 0x7f67a4069df0 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41a6320 con 0x7f67a4106880 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67ab2ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41a64e0 con 0x7f67a4100ea0 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.931+0000 7f67a9089640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 0x7f67a4100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 0x7f67a4100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49564/0 (socket says 172.21.15.8:49564) 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 -- 172.21.15.8:0/296224262 learned_addr learned my addr 172.21.15.8:0/296224262 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a988a640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4100ea0 0x7f67a41a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a8888640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a4106880 0x7f67a4100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 -- 172.21.15.8:0/296224262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4100ea0 msgr2=0x7f67a41a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4100ea0 0x7f67a41a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 -- 172.21.15.8:0/296224262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a4106880 msgr2=0x7f67a4100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a4106880 0x7f67a4100960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 -- 172.21.15.8:0/296224262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009560 con 0x7f67a4069df0 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67a9089640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 0x7f67a4100420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f679c00e9c0 tx=0x7f679c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00cde0 con 0x7f67a4069df0 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a0009840 con 0x7f67a4069df0 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f679c0035f0 con 0x7f67a4069df0 2024-02-19T16:27:33.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c010650 con 0x7f67a4069df0 2024-02-19T16:27:33.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.932+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a41a6b40 con 0x7f67a4069df0 2024-02-19T16:27:33.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.933+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4068730 con 0x7f67a4069df0 2024-02-19T16:27:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.934+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f679c003ad0 con 0x7f67a4069df0 2024-02-19T16:27:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.935+0000 7f679a7fc640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f67900a2260 0x7f67900a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.935+0000 7f67a8888640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f67900a2260 0x7f67900a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.935+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f679c014070 con 0x7f67a4069df0 2024-02-19T16:27:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.935+0000 7f67a8888640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f67900a2260 0x7f67900a4720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67a4101eb0 tx=0x7f6794005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:33.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:33.939+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f679c097120 con 0x7f67a4069df0 2024-02-19T16:27:34.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:33 smithi008 ceph-mon[32573]: pgmap v143: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:34.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:33 smithi156 ceph-mon[36502]: pgmap v143: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:34.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.159+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67a4109af0 con 0x7f67a4069df0 2024-02-19T16:27:34.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.160+0000 7f679a7fc640 1 -- 172.21.15.8:0/296224262 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f679c003760 con 0x7f67a4069df0 2024-02-19T16:27:34.162 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:27:34.162 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:27:34.163 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:27:34.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f67900a2260 msgr2=0x7f67900a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f67900a2260 0x7f67900a4720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67a4101eb0 tx=0x7f6794005f90 comp rx=0 tx=0).stop 2024-02-19T16:27:34.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 msgr2=0x7f67a4100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 0x7f67a4100420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f679c00e9c0 tx=0x7f679c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:27:34.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 shutdown_connections 2024-02-19T16:27:34.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f67900a2260 0x7f67900a4720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4100ea0 0x7f67a41a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67a4106880 0x7f67a4100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 --2- 172.21.15.8:0/296224262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67a4069df0 0x7f67a4100420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 >> 172.21.15.8:0/296224262 conn(0x7f67a4077490 msgr2=0x7f67a40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 shutdown_connections 2024-02-19T16:27:34.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.162+0000 7f67ab2ed640 1 -- 172.21.15.8:0/296224262 wait complete. 2024-02-19T16:27:34.174 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:27:34.174 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:27:34.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:33 smithi191 ceph-mon[37203]: pgmap v143: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:34.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/2745381171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c0102760 msgr2=0x7fe8c0102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/2745381171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c0102760 0x7fe8c0102b40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009900 tx=0x7fe8bc02bcd0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/2745381171 shutdown_connections 2024-02-19T16:27:34.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/2745381171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8c010bb30 0x7fe8c010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/2745381171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8c0103080 0x7fe8c010b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/2745381171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c0102760 0x7fe8c0102b40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/2745381171 >> 172.21.15.8:0/2745381171 conn(0x7fe8c0077550 msgr2=0x7fe8c0077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/2745381171 shutdown_connections 2024-02-19T16:27:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.277+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/2745381171 wait complete. 2024-02-19T16:27:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 Processor -- start 2024-02-19T16:27:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 -- start start 2024-02-19T16:27:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8c0102760 0x7fe8c019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8c0103080 0x7fe8c019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 0x7fe8c0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0198050 con 0x7fe8c010bb30 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01981c0 con 0x7fe8c0102760 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c7fc9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0198380 con 0x7fe8c0103080 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 0x7fe8c0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 0x7fe8c0197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49576/0 (socket says 172.21.15.8:49576) 2024-02-19T16:27:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 -- 172.21.15.8:0/369254364 learned_addr learned my addr 172.21.15.8:0/369254364 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c5564640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8c0103080 0x7fe8c019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8c0102760 0x7fe8c019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 -- 172.21.15.8:0/369254364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8c0103080 msgr2=0x7fe8c019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8c0103080 0x7fe8c019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 -- 172.21.15.8:0/369254364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8c0102760 msgr2=0x7fe8c019d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8c0102760 0x7fe8c019d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.278+0000 7fe8c6566640 1 -- 172.21.15.8:0/369254364 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009840 con 0x7fe8c010bb30 2024-02-19T16:27:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.279+0000 7fe8c6566640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 0x7fe8c0197a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800efd0 tx=0x7fe8b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.279+0000 7fe8b6ffd640 1 -- 172.21.15.8:0/369254364 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8009b40 con 0x7fe8c010bb30 2024-02-19T16:27:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.279+0000 7fe8b6ffd640 1 -- 172.21.15.8:0/369254364 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8b8003b70 con 0x7fe8c010bb30 2024-02-19T16:27:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.279+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc009560 con 0x7fe8c010bb30 2024-02-19T16:27:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.279+0000 7fe8b6ffd640 1 -- 172.21.15.8:0/369254364 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80108f0 con 0x7fe8c010bb30 2024-02-19T16:27:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.279+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c01a46c0 con 0x7fe8c010bb30 2024-02-19T16:27:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.281+0000 7fe8b6ffd640 1 -- 172.21.15.8:0/369254364 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe8b8003ce0 con 0x7fe8c010bb30 2024-02-19T16:27:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.281+0000 7fe8caba4740 1 Processor -- start 2024-02-19T16:27:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.281+0000 7fe8caba4740 1 -- start start 2024-02-19T16:27:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.281+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e62630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.281+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e62b70 0x560cc4e6b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.281+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 0x560cc4e6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8caba4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4e1d960 con 0x560cc4e6b750 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8caba4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4dd1310 con 0x560cc4e62250 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8caba4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4dce840 con 0x560cc4e62b70 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8b6ffd640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8ac0a2230 0x7fe8ac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5564640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 0x560cc4e6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c6566640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8ac0a2230 0x7fe8ac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e62b70 0x560cc4e6b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5564640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 0x560cc4e6db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49582/0 (socket says 172.21.15.8:49582) 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5564640 1 -- 172.21.15.8:0/3327456580 learned_addr learned my addr 172.21.15.8:0/3327456580 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5564640 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8b6ffd640 1 -- 172.21.15.8:0/369254364 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fe8b8014070 con 0x7fe8c010bb30 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 -- 172.21.15.8:0/3327456580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 msgr2=0x560cc4e62630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e62630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 -- 172.21.15.8:0/3327456580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 msgr2=0x560cc4e6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 0x560cc4e6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 -- 172.21.15.8:0/3327456580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc4d8c060 con 0x560cc4e62b70 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5564640 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 0x560cc4e6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c6566640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8ac0a2230 0x7fe8ac0a46f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8b809af40 tx=0x7fe8b80c58b0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e62b70 0x560cc4e6b210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc02c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f9068c74599681 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.282+0000 7fe8abfff640 1 -- 172.21.15.8:0/3327456580 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0047c0 con 0x560cc4e62b70 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8abfff640 1 -- 172.21.15.8:0/3327456580 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc004920 con 0x560cc4e62b70 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8abfff640 1 -- 172.21.15.8:0/3327456580 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc005050 con 0x560cc4e62b70 2024-02-19T16:27:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 -- 172.21.15.8:0/3327456580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e62b70 msgr2=0x560cc4e6b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e62b70 0x560cc4e6b210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc02c740 comp rx=0 tx=0).stop 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 -- 172.21.15.8:0/3327456580 shutdown_connections 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e6b750 0x560cc4e6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e62b70 0x560cc4e6b210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3327456580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 -- 172.21.15.8:0/3327456580 >> 172.21.15.8:0/3327456580 conn(0x560cc4e5c980 msgr2=0x560cc4e5cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 -- 172.21.15.8:0/3327456580 shutdown_connections 2024-02-19T16:27:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.283+0000 7fe8caba4740 1 -- 172.21.15.8:0/3327456580 wait complete. 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 -- start start 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 Processor -- start 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 0x560cc4e68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4dca020 con 0x560cc4e62b70 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4a9d430 con 0x560cc4e62250 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4a59e00 con 0x560cc4e6b750 2024-02-19T16:27:34.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8caba4740 10 client.0 _ll_register_callbacks cb 0x560cc4d71cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 0x560cc4e68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 0x560cc4e68bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49598/0 (socket says 172.21.15.8:49598) 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 -- 172.21.15.8:0/2048844122 learned_addr learned my addr 172.21.15.8:0/2048844122 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5564640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5564640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 -- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 msgr2=0x560cc4e69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 -- 172.21.15.8:0/2048844122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 msgr2=0x560cc4e686b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e686b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.284+0000 7fe8c5d65640 1 -- 172.21.15.8:0/2048844122 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c010ec20 con 0x560cc4e62b70 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 0x560cc4e68bf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc036040 tx=0x7fe8bc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648985d56f3e9f96 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8a97fa640 1 -- 172.21.15.8:0/2048844122 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0047c0 con 0x560cc4e62b70 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8a97fa640 1 -- 172.21.15.8:0/2048844122 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fe8bc050070 con 0x560cc4e62b70 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc4ac97e0 con 0x560cc4e62b70 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8a97fa640 1 -- 172.21.15.8:0/2048844122 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8bc004500 con 0x560cc4e62b70 2024-02-19T16:27:34.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 10 client.16668 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560cc4dc86a0 con 0x560cc4e62b70 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8a97fa640 1 -- 172.21.15.8:0/2048844122 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560cc4ac97e0 con 0x560cc4e62b70 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8a97fa640 1 -- 172.21.15.8:0/2048844122 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8bc03c050 con 0x560cc4e62b70 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 10 client.16668 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 10 client.16668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 20 client.16668 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4e7b570 0x560cc4e7d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 4 client.16668 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.285+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cc4b06720 con 0x560cc4e7b570 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.286+0000 7fe8c5564640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4e7b570 0x560cc4e7d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.286+0000 7fe8a97fa640 10 client.16668 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.286+0000 7fe8c5564640 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4e7b570 0x560cc4e7d990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a573bcbe0d15bb3 server_cookie=9a9f4a04b4ee5e98 in_seq=0 out_seq=0 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.286+0000 7fe8a97fa640 1 -- 172.21.15.8:0/2048844122 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560cc4b06720 con 0x560cc4e7b570 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.286+0000 7fe8a97fa640 10 client.16668 handle_command_reply: tid=0 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 client.16668 shutdown 2024-02-19T16:27:34.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 msgr2=0x560cc4e68bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 0x560cc4e68bf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc036040 tx=0x7fe8bc005020 comp rx=0 tx=0).stop 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 shutdown_connections 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4e7b570 0x560cc4e7d990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62b70 0x560cc4e68bf0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8a97fa640 0 client.16668 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:34.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 --2- 172.21.15.8:0/2048844122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62250 0x560cc4e686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 >> 172.21.15.8:0/2048844122 conn(0x560cc4e5c980 msgr2=0x560cc4e5e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 shutdown_connections 2024-02-19T16:27:34.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 1 -- 172.21.15.8:0/2048844122 wait complete. 2024-02-19T16:27:34.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.287+0000 7fe8caba4740 20 client.16668 trim_cache size 0 max 16384 2024-02-19T16:27:34.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.299+0000 7fe8caba4740 1 Processor -- start 2024-02-19T16:27:34.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.299+0000 7fe8caba4740 1 -- start start 2024-02-19T16:27:34.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e6ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e76950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8caba4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4d4f130 con 0x560cc4e62250 2024-02-19T16:27:34.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8caba4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4ac97e0 con 0x560cc4e62b70 2024-02-19T16:27:34.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8caba4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4e5d980 con 0x560cc4e6b750 2024-02-19T16:27:34.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e6ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e6ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49610/0 (socket says 172.21.15.8:49610) 2024-02-19T16:27:34.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 -- 172.21.15.8:0/3526109558 learned_addr learned my addr 172.21.15.8:0/3526109558 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e76950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 -- 172.21.15.8:0/3526109558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 msgr2=0x560cc4e76950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e76950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 -- 172.21.15.8:0/3526109558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 msgr2=0x560cc4e68f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e68f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 -- 172.21.15.8:0/3526109558 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc4d64c20 con 0x560cc4e62250 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8c5564640 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e6ade0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00043a0 tx=0x7fe8b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7926a3aecaefa21a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8ab7fe640 1 -- 172.21.15.8:0/3526109558 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0016040 con 0x560cc4e62250 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8ab7fe640 1 -- 172.21.15.8:0/3526109558 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8b0005500 con 0x560cc4e62250 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.300+0000 7fe8ab7fe640 1 -- 172.21.15.8:0/3526109558 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00039d0 con 0x560cc4e62250 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 -- 172.21.15.8:0/3526109558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 msgr2=0x560cc4e6ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e6ade0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00043a0 tx=0x7fe8b00046c0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 -- 172.21.15.8:0/3526109558 shutdown_connections 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e76950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e68f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 --2- 172.21.15.8:0/3526109558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e6ade0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 -- 172.21.15.8:0/3526109558 >> 172.21.15.8:0/3526109558 conn(0x560cc4e5c980 msgr2=0x560cc4e5bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 -- 172.21.15.8:0/3526109558 shutdown_connections 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 -- 172.21.15.8:0/3526109558 wait complete. 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 -- start start 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.301+0000 7fe8caba4740 1 Processor -- start 2024-02-19T16:27:34.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e72ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4e737c0 con 0x560cc4e62250 2024-02-19T16:27:34.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4e661f0 con 0x560cc4e62b70 2024-02-19T16:27:34.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4e663b0 con 0x560cc4e6b750 2024-02-19T16:27:34.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 10 client.0 _ll_register_callbacks cb 0x7fe8b0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:34.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e72ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e72ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49624/0 (socket says 172.21.15.8:49624) 2024-02-19T16:27:34.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 -- 172.21.15.8:0/387990311 learned_addr learned my addr 172.21.15.8:0/387990311 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 -- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 msgr2=0x560cc4e65cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e65cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 -- 172.21.15.8:0/387990311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 msgr2=0x560cc4e65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 -- 172.21.15.8:0/387990311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b0004450 con 0x560cc4e62250 2024-02-19T16:27:34.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8c5564640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e72ef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0004270 tx=0x7fe8b0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a844fa58700e60e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8a97fa640 1 -- 172.21.15.8:0/387990311 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0016070 con 0x560cc4e62250 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8a97fa640 1 -- 172.21.15.8:0/387990311 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fe8b0024030 con 0x560cc4e62250 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8bc004920 con 0x560cc4e62250 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8a97fa640 1 -- 172.21.15.8:0/387990311 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b0004870 con 0x560cc4e62250 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 10 client.16674 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.302+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560cc4d8a8c0 con 0x560cc4e62250 2024-02-19T16:27:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8a97fa640 1 -- 172.21.15.8:0/387990311 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc004920 con 0x560cc4e62250 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8a97fa640 1 -- 172.21.15.8:0/387990311 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8b0020420 con 0x560cc4e62250 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8caba4740 10 client.16674 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8caba4740 10 client.16674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8caba4740 20 client.16674 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8caba4740 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4ed8110 0x560cc4eda500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8caba4740 4 client.16674 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:27:34.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560cc4b55890 con 0x560cc4ed8110 2024-02-19T16:27:34.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4ed8110 0x560cc4eda500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8a97fa640 10 client.16674 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:34.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.303+0000 7fe8c5d65640 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4ed8110 0x560cc4eda500 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d83e1cf3cac8a8d5 server_cookie=a7ac4f3ea45e9cbf in_seq=0 out_seq=0 2024-02-19T16:27:34.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8a97fa640 1 -- 172.21.15.8:0/387990311 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560cc4b55890 con 0x560cc4ed8110 2024-02-19T16:27:34.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8a97fa640 10 client.16674 handle_command_reply: tid=0 2024-02-19T16:27:34.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 client.16674 shutdown 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 msgr2=0x560cc4e72ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e72ef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0004270 tx=0x7fe8b0005f90 comp rx=0 tx=0).stop 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 shutdown_connections 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x560cc4ed8110 0x560cc4eda500 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc4e6b750 0x560cc4e65cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8a97fa640 0 client.16674 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:27:34.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cc4e62b70 0x560cc4e65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 --2- 172.21.15.8:0/387990311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560cc4e62250 0x560cc4e72ef0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 >> 172.21.15.8:0/387990311 conn(0x560cc4e5c980 msgr2=0x560cc4e5db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 shutdown_connections 2024-02-19T16:27:34.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 1 -- 172.21.15.8:0/387990311 wait complete. 2024-02-19T16:27:34.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8caba4740 20 client.16674 trim_cache size 0 max 16384 2024-02-19T16:27:34.327 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:27:34.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8ac0a2230 msgr2=0x7fe8ac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8ac0a2230 0x7fe8ac0a46f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8b809af40 tx=0x7fe8b80c58b0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 msgr2=0x7fe8c0197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 0x7fe8c0197a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800efd0 tx=0x7fe8b800c6a0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 shutdown_connections 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8c010bb30 0x7fe8c0197a00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe8ac0a2230 0x7fe8ac0a46f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8c0103080 0x7fe8c019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 --2- 172.21.15.8:0/369254364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8c0102760 0x7fe8c019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.304+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 >> 172.21.15.8:0/369254364 conn(0x7fe8c0077550 msgr2=0x7fe8c00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.305+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 shutdown_connections 2024-02-19T16:27:34.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.305+0000 7fe8c7fc9640 1 -- 172.21.15.8:0/369254364 wait complete. 2024-02-19T16:27:34.330 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:27:34.330 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:27:34.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 -- 172.21.15.8:0/532571939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54981056a0 msgr2=0x7f549810d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 --2- 172.21.15.8:0/532571939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54981056a0 0x7f549810d270 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5490009cd0 tx=0x7f549002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 -- 172.21.15.8:0/532571939 shutdown_connections 2024-02-19T16:27:34.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 --2- 172.21.15.8:0/532571939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54981056a0 0x7f549810d270 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 --2- 172.21.15.8:0/532571939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5498100f80 0x7f5498104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 --2- 172.21.15.8:0/532571939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54981005d0 0x7f54981009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.433+0000 7f549e4d4640 1 -- 172.21.15.8:0/532571939 >> 172.21.15.8:0/532571939 conn(0x7f54980774f0 msgr2=0x7f5498077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 -- 172.21.15.8:0/532571939 shutdown_connections 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 -- 172.21.15.8:0/532571939 wait complete. 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 Processor -- start 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 -- start start 2024-02-19T16:27:34.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54981005d0 0x7f549819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54981056a0 0x7f549819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 0x7f5498197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498198370 con 0x7f5498197980 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54981984e0 con 0x7f54981056a0 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.434+0000 7f549e4d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54981986a0 con 0x7f54981005d0 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 0x7f5498197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 0x7f5498197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49646/0 (socket says 172.21.15.8:49646) 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 -- 172.21.15.8:0/1583252505 learned_addr learned my addr 172.21.15.8:0/1583252505 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f5497fff640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54981005d0 0x7f549819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f54977fe640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54981056a0 0x7f549819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 -- 172.21.15.8:0/1583252505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54981005d0 msgr2=0x7f549819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54981005d0 0x7f549819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 -- 172.21.15.8:0/1583252505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54981056a0 msgr2=0x7f549819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54981056a0 0x7f549819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 -- 172.21.15.8:0/1583252505 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009900 con 0x7f5498197980 2024-02-19T16:27:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549ca71640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 0x7f5498197e30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5490004af0 tx=0x7f5490032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f54957fa640 1 -- 172.21.15.8:0/1583252505 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900328b0 con 0x7f5498197980 2024-02-19T16:27:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f54957fa640 1 -- 172.21.15.8:0/1583252505 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5490003e50 con 0x7f5498197980 2024-02-19T16:27:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54981a4640 con 0x7f5498197980 2024-02-19T16:27:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f54957fa640 1 -- 172.21.15.8:0/1583252505 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490004740 con 0x7f5498197980 2024-02-19T16:27:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.435+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54981a4ac0 con 0x7f5498197980 2024-02-19T16:27:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.437+0000 7f54957fa640 1 -- 172.21.15.8:0/1583252505 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f549000a040 con 0x7f5498197980 2024-02-19T16:27:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 Processor -- start 2024-02-19T16:27:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 -- start start 2024-02-19T16:27:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195306630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54957fa640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f54800a2230 0x7f54800a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 0x558195311b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581952c1960 con 0x55819530f750 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558195275310 con 0x558195306250 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54a118d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558195272840 con 0x558195306b70 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 0x558195311b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f5497fff640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f54800a2230 0x7f54800a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54977fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195306630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 0x558195311b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49660/0 (socket says 172.21.15.8:49660) 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 -- 172.21.15.8:0/2505900847 learned_addr learned my addr 172.21.15.8:0/2505900847 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54957fa640 1 -- 172.21.15.8:0/1583252505 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f54900f0ec0 con 0x7f5498197980 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54977fe640 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 -- 172.21.15.8:0/2505900847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 msgr2=0x55819530f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 -- 172.21.15.8:0/2505900847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 msgr2=0x558195306630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195306630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f549ca71640 1 -- 172.21.15.8:0/2505900847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558195230060 con 0x55819530f750 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.438+0000 7f54977fe640 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195306630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f5497fff640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f54800a2230 0x7f54800a46f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5498105230 tx=0x7f5488005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f549ca71640 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 0x558195311b40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f54900c5880 tx=0x7f549002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef2ff370adb0432 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f547e7fc640 1 -- 172.21.15.8:0/2505900847 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900cfe60 con 0x55819530f750 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f547e7fc640 1 -- 172.21.15.8:0/2505900847 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54900cfc80 con 0x55819530f750 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f547e7fc640 1 -- 172.21.15.8:0/2505900847 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900bd680 con 0x55819530f750 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 -- 172.21.15.8:0/2505900847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 msgr2=0x558195311b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 0x558195311b40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f54900c5880 tx=0x7f549002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 -- 172.21.15.8:0/2505900847 shutdown_connections 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819530f750 0x558195311b40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 --2- 172.21.15.8:0/2505900847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195306630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 -- 172.21.15.8:0/2505900847 >> 172.21.15.8:0/2505900847 conn(0x558195300980 msgr2=0x558195300d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.439+0000 7f54a118d740 1 -- 172.21.15.8:0/2505900847 shutdown_connections 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 -- 172.21.15.8:0/2505900847 wait complete. 2024-02-19T16:27:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 -- start start 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 Processor -- start 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819526e020 con 0x558195306250 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558194f41430 con 0x55819530f750 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558194efde00 con 0x558195306b70 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54a118d740 10 client.0 _ll_register_callbacks cb 0x558195215cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f5497fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f5497fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49668/0 (socket says 172.21.15.8:49668) 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f5497fff640 1 -- 172.21.15.8:0/709074280 learned_addr learned my addr 172.21.15.8:0/709074280 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f549ca71640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.440+0000 7f54977fe640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f5497fff640 1 -- 172.21.15.8:0/709074280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 msgr2=0x55819530cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f5497fff640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f5497fff640 1 -- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 msgr2=0x55819530d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f5497fff640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f5497fff640 1 -- 172.21.15.8:0/709074280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54981a4ac0 con 0x558195306250 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f5497fff640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530c730 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5488003ad0 tx=0x7f5488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7ffe9b484c6ae server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54777fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54777fe640 1 -- 172.21.15.8:0/709074280 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800ecc0 con 0x558195306250 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54777fe640 1 -- 172.21.15.8:0/709074280 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f548800ee20 con 0x558195306250 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558194f6d7e0 con 0x558195306250 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54777fe640 1 -- 172.21.15.8:0/709074280 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f548801e420 con 0x558195306250 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54a118d740 10 client.16686 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55819526c6a0 con 0x558195306250 2024-02-19T16:27:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.441+0000 7f54777fe640 1 -- 172.21.15.8:0/709074280 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558194f6d7e0 con 0x558195306250 2024-02-19T16:27:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54777fe640 1 -- 172.21.15.8:0/709074280 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5488024020 con 0x558195306250 2024-02-19T16:27:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54a118d740 10 client.16686 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54a118d740 10 client.16686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:27:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54a118d740 20 client.16686 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54a118d740 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819531f9c0 0x558195321de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54a118d740 4 client.16686 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:34.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558194faa720 con 0x55819531f9c0 2024-02-19T16:27:34.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54977fe640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819531f9c0 0x558195321de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54777fe640 10 client.16686 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:34.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.442+0000 7f54977fe640 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819531f9c0 0x558195321de0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8e172d4aef0a958 server_cookie=e43d53ac5b8eeff8 in_seq=0 out_seq=0 2024-02-19T16:27:34.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54777fe640 1 -- 172.21.15.8:0/709074280 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558194faa720 con 0x55819531f9c0 2024-02-19T16:27:34.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54777fe640 10 client.16686 handle_command_reply: tid=0 2024-02-19T16:27:34.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 client.16686 shutdown 2024-02-19T16:27:34.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 msgr2=0x55819530c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530c730 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5488003ad0 tx=0x7f5488004730 comp rx=0 tx=0).stop 2024-02-19T16:27:34.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 shutdown_connections 2024-02-19T16:27:34.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819531f9c0 0x558195321de0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 --2- 172.21.15.8:0/709074280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530c730 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54777fe640 0 client.16686 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 >> 172.21.15.8:0/709074280 conn(0x558195300980 msgr2=0x5581953026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 shutdown_connections 2024-02-19T16:27:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 1 -- 172.21.15.8:0/709074280 wait complete. 2024-02-19T16:27:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.443+0000 7f54a118d740 20 client.16686 trim_cache size 0 max 16384 2024-02-19T16:27:34.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.455+0000 7f54a118d740 1 Processor -- start 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.455+0000 7f54a118d740 1 -- start start 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54a118d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54a118d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54a118d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54a118d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581951f3130 con 0x558195306250 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54a118d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558195301780 con 0x55819530f750 2024-02-19T16:27:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54a118d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558195300840 con 0x558195306b70 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49684/0 (socket says 172.21.15.8:49684) 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 -- 172.21.15.8:0/2598645222 learned_addr learned my addr 172.21.15.8:0/2598645222 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f54977fe640 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f549ca71640 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 -- 172.21.15.8:0/2598645222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 msgr2=0x55819530b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 -- 172.21.15.8:0/2598645222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 msgr2=0x55819530cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 -- 172.21.15.8:0/2598645222 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558195208c20 con 0x558195306250 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f5497fff640 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530ee60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5488003aa0 tx=0x7f5488003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f43dd1ea3c59ca server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f547dffb640 1 -- 172.21.15.8:0/2598645222 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488026ab0 con 0x558195306250 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f547dffb640 1 -- 172.21.15.8:0/2598645222 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5488026c10 con 0x558195306250 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.456+0000 7f547dffb640 1 -- 172.21.15.8:0/2598645222 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488033870 con 0x558195306250 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 -- 172.21.15.8:0/2598645222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 msgr2=0x55819530ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530ee60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5488003aa0 tx=0x7f5488003730 comp rx=0 tx=0).stop 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 -- 172.21.15.8:0/2598645222 shutdown_connections 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55819530f750 0x55819530cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558195306b70 0x55819530b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 --2- 172.21.15.8:0/2598645222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306250 0x55819530ee60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 -- 172.21.15.8:0/2598645222 >> 172.21.15.8:0/2598645222 conn(0x558195300980 msgr2=0x558195302660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 -- 172.21.15.8:0/2598645222 shutdown_connections 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 -- 172.21.15.8:0/2598645222 wait complete. 2024-02-19T16:27:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 -- start start 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.457+0000 7f54a118d740 1 Processor -- start 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195316d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 0x5581953172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55819530f750 0x558195309970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819530a040 con 0x558195306b70 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819530a1b0 con 0x558195306250 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819530a370 con 0x55819530f750 2024-02-19T16:27:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 10 client.0 _ll_register_callbacks cb 0x7f54880040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 0x5581953172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 0x5581953172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49690/0 (socket says 172.21.15.8:49690) 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 -- 172.21.15.8:0/2089668418 learned_addr learned my addr 172.21.15.8:0/2089668418 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f5497fff640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195316d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f549ca71640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55819530f750 0x558195309970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 -- 172.21.15.8:0/2089668418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55819530f750 msgr2=0x558195309970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55819530f750 0x558195309970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 -- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 msgr2=0x558195316d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195316d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 -- 172.21.15.8:0/2089668418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548802b040 con 0x558195306b70 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f5497fff640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195316d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f549ca71640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55819530f750 0x558195309970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54977fe640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 0x5581953172d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f548c004700 tx=0x7f548c022f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8850a1aaaa0b5227 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54777fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54777fe640 1 -- 172.21.15.8:0/2089668418 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c016040 con 0x558195306b70 2024-02-19T16:27:34.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54777fe640 1 -- 172.21.15.8:0/2089668418 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f548c010b40 con 0x558195306b70 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.458+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54900cfc80 con 0x558195306b70 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54777fe640 1 -- 172.21.15.8:0/2089668418 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f548c010d00 con 0x558195306b70 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 10 client.16692 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55819522e8c0 con 0x558195306b70 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54777fe640 1 -- 172.21.15.8:0/2089668418 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54900cfc80 con 0x558195306b70 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54777fe640 1 -- 172.21.15.8:0/2089668418 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f548c020420 con 0x558195306b70 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 10 client.16692 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:34.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 10 client.16692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 20 client.16692 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819537c5f0 0x55819537ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 4 client.16692 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558194ff9890 con 0x55819537c5f0 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f5497fff640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819537c5f0 0x55819537ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f54777fe640 10 client.16692 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.459+0000 7f5497fff640 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819537c5f0 0x55819537ea10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cb8af3dcf0fc109 server_cookie=fc8f36174a177e7e in_seq=0 out_seq=0 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54777fe640 1 -- 172.21.15.8:0/2089668418 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558194ff9890 con 0x55819537c5f0 2024-02-19T16:27:34.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54777fe640 10 client.16692 handle_command_reply: tid=0 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 client.16692 shutdown 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 msgr2=0x5581953172d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 0x5581953172d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f548c004700 tx=0x7f548c022f40 comp rx=0 tx=0).stop 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 shutdown_connections 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55819537c5f0 0x55819537ea10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55819530f750 0x558195309970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54777fe640 0 client.16692 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:27:34.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558195306b70 0x5581953172d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 --2- 172.21.15.8:0/2089668418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558195306250 0x558195316d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 >> 172.21.15.8:0/2089668418 conn(0x558195300980 msgr2=0x5581952ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 shutdown_connections 2024-02-19T16:27:34.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 1 -- 172.21.15.8:0/2089668418 wait complete. 2024-02-19T16:27:34.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f54a118d740 20 client.16692 trim_cache size 0 max 16384 2024-02-19T16:27:34.478 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:27:34.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f54800a2230 msgr2=0x7f54800a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f549e4d4640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f54800a2230 0x7f54800a46f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5498105230 tx=0x7f5488005f90 comp rx=0 tx=0).stop 2024-02-19T16:27:34.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 msgr2=0x7f5498197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.460+0000 7f549e4d4640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 0x7f5498197e30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5490004af0 tx=0x7f5490032570 comp rx=0 tx=0).stop 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 shutdown_connections 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5498197980 0x7f5498197e30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54981056a0 0x7f549819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f54800a2230 0x7f54800a46f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 --2- 172.21.15.8:0/1583252505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54981005d0 0x7f549819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 >> 172.21.15.8:0/1583252505 conn(0x7f54980774f0 msgr2=0x7f54980fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 shutdown_connections 2024-02-19T16:27:34.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.461+0000 7f549e4d4640 1 -- 172.21.15.8:0/1583252505 wait complete. 2024-02-19T16:27:34.482 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:27:34.482 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:27:34.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3330230299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4107560 msgr2=0x7f9ee4107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3330230299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4107560 0x7f9ee4107940 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3330230299 shutdown_connections 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3330230299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ee4101e60 0x7f9ee410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3330230299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee41014a0 0x7f9ee4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3330230299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4107560 0x7f9ee4107940 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3330230299 >> 172.21.15.8:0/3330230299 conn(0x7f9ee40788f0 msgr2=0x7f9ee4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3330230299 shutdown_connections 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3330230299 wait complete. 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 Processor -- start 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.585+0000 7f9ee9e76640 1 -- start start 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ee41014a0 0x7f9ee419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 0x7f9ee419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee4107560 0x7f9ee41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4197ef0 con 0x7f9ee4101e60 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4198060 con 0x7f9ee4107560 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4198220 con 0x7f9ee41014a0 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 0x7f9ee419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 0x7f9ee419dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49708/0 (socket says 172.21.15.8:49708) 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 -- 172.21.15.8:0/3875816295 learned_addr learned my addr 172.21.15.8:0/3875816295 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ee41014a0 0x7f9ee419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 -- 172.21.15.8:0/3875816295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ee41014a0 msgr2=0x7f9ee419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ee41014a0 0x7f9ee419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 -- 172.21.15.8:0/3875816295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee4107560 msgr2=0x7f9ee41978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee4107560 0x7f9ee41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 -- 172.21.15.8:0/3875816295 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc009840 con 0x7f9ee4101e60 2024-02-19T16:27:34.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9edaffd640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 0x7f9ee419dc30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00e9f0 tx=0x7f9ecc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee17fa640 1 -- 172.21.15.8:0/3875816295 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc009700 con 0x7f9ee4101e60 2024-02-19T16:27:34.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed4009560 con 0x7f9ee4101e60 2024-02-19T16:27:34.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee17fa640 1 -- 172.21.15.8:0/3875816295 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ecc00cd80 con 0x7f9ee4101e60 2024-02-19T16:27:34.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee17fa640 1 -- 172.21.15.8:0/3875816295 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc018920 con 0x7f9ee4101e60 2024-02-19T16:27:34.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.586+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee41986f0 con 0x7f9ee4101e60 2024-02-19T16:27:34.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.588+0000 7f9ee17fa640 1 -- 172.21.15.8:0/3875816295 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f9ecc018a80 con 0x7f9ee4101e60 2024-02-19T16:27:34.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 Processor -- start 2024-02-19T16:27:34.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 -- start start 2024-02-19T16:27:34.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9ee17fa640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9ed00a2230 0x7f9ed00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9ee17fa640 1 -- 172.21.15.8:0/3875816295 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f9ecc014070 con 0x7f9ee4101e60 2024-02-19T16:27:34.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9ee3fff640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9ed00a2230 0x7f9ed00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed25602630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2560db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed255bd960 con 0x55ed25602b70 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25571310 con 0x55ed2560b750 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2556e840 con 0x55ed25602250 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49722/0 (socket says 172.21.15.8:49722) 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9edaffd640 1 -- 172.21.15.8:0/1711074461 learned_addr learned my addr 172.21.15.8:0/1711074461 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed25602630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.589+0000 7f9ee3fff640 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2560db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1711074461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 msgr2=0x55ed2560db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2560db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1711074461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 msgr2=0x55ed2560b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1711074461 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2552c060 con 0x55ed25602250 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9edaffd640 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee3fff640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9ed00a2230 0x7f9ed00a46f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009270 tx=0x7f9edc012bb0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed25602630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000c00 tx=0x7f9ed402c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d65238d007ac0d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ed9ffb640 1 -- 172.21.15.8:0/1711074461 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed402c890 con 0x55ed25602250 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ed9ffb640 1 -- 172.21.15.8:0/1711074461 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed402c9f0 con 0x55ed25602250 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.590+0000 7f9ed9ffb640 1 -- 172.21.15.8:0/1711074461 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4005050 con 0x55ed25602250 2024-02-19T16:27:34.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1711074461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 msgr2=0x55ed25602630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed25602630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000c00 tx=0x7f9ed402c6a0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1711074461 shutdown_connections 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2560db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1711074461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed25602630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1711074461 >> 172.21.15.8:0/1711074461 conn(0x55ed255fc980 msgr2=0x55ed255fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1711074461 shutdown_connections 2024-02-19T16:27:34.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1711074461 wait complete. 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 -- start start 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 Processor -- start 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 0x55ed25608780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.591+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602b70 0x55ed25608cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25609200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2556a020 con 0x55ed25602250 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2523d430 con 0x55ed25602b70 2024-02-19T16:27:34.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed251f9e00 con 0x55ed2560b750 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 10 client.0 _ll_register_callbacks cb 0x55ed25511cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 0x55ed25608780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 0x55ed25608780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49728/0 (socket says 172.21.15.8:49728) 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1343403943 learned_addr learned my addr 172.21.15.8:0/1343403943 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee3fff640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25609200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9edaffd640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602b70 0x55ed25608cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 msgr2=0x55ed25609200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25609200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1343403943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602b70 msgr2=0x55ed25608cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602b70 0x55ed25608cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 -- 172.21.15.8:0/1343403943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee410ec20 con 0x55ed25602250 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 0x55ed25608780 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005b50 tx=0x7f9ed4005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730b80c5a43a07d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ec2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/1343403943 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed402c890 con 0x55ed25602250 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/1343403943 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f9ed4004df0 con 0x55ed25602250 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed252697e0 con 0x55ed25602250 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/1343403943 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ed40044d0 con 0x55ed25602250 2024-02-19T16:27:34.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 10 client.16704 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:34.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.592+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed255686a0 con 0x55ed25602250 2024-02-19T16:27:34.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/1343403943 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed252697e0 con 0x55ed25602250 2024-02-19T16:27:34.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/1343403943 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ed403c050 con 0x55ed25602250 2024-02-19T16:27:34.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9eeca4f740 10 client.16704 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:34.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9eeca4f740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:34.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9eeca4f740 20 client.16704 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:34.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed2561b850 0x55ed2561dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9eeca4f740 4 client.16704 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:34.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed252a6720 con 0x55ed2561b850 2024-02-19T16:27:34.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9edaffd640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed2561b850 0x55ed2561dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9ec2ffd640 10 client.16704 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:34.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.593+0000 7f9edaffd640 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed2561b850 0x55ed2561dc40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e268ada90323683a server_cookie=553154415e613d6b in_seq=0 out_seq=0 2024-02-19T16:27:34.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.594+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/1343403943 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed252a6720 con 0x55ed2561b850 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.594+0000 7f9ec2ffd640 10 client.16704 handle_command_reply: tid=0 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.594+0000 7f9eeca4f740 1 client.16704 shutdown 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 msgr2=0x55ed25608780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 0x55ed25608780 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005b50 tx=0x7f9ed4005b80 comp rx=0 tx=0).stop 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 shutdown_connections 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed2561b850 0x55ed2561dc40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25609200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602b70 0x55ed25608cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9ec2ffd640 0 client.16704 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:34.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/1343403943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602250 0x55ed25608780 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 >> 172.21.15.8:0/1343403943 conn(0x55ed255fc980 msgr2=0x55ed255fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 shutdown_connections 2024-02-19T16:27:34.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 1 -- 172.21.15.8:0/1343403943 wait complete. 2024-02-19T16:27:34.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.595+0000 7f9eeca4f740 20 client.16704 trim_cache size 0 max 16384 2024-02-19T16:27:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 Processor -- start 2024-02-19T16:27:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 -- start start 2024-02-19T16:27:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602250 0x55ed2560aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed255f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25608cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed254ef130 con 0x55ed25602b70 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2561b660 con 0x55ed25602250 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed255f82e0 con 0x55ed2560b750 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed255f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed255f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49732/0 (socket says 172.21.15.8:49732) 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.607+0000 7f9edaffd640 1 -- 172.21.15.8:0/3505852596 learned_addr learned my addr 172.21.15.8:0/3505852596 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602250 0x55ed2560aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9ee3fff640 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25608cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9edaffd640 1 -- 172.21.15.8:0/3505852596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 msgr2=0x55ed25608cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9edaffd640 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25608cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9edaffd640 1 -- 172.21.15.8:0/3505852596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602250 msgr2=0x55ed2560aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9edaffd640 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602250 0x55ed2560aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9edaffd640 1 -- 172.21.15.8:0/3505852596 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed25504c20 con 0x55ed25602b70 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602250 0x55ed2560aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9edaffd640 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed255f7a10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00e9c0 tx=0x7f9ecc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f584f31c00fe972 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9ed97fa640 1 -- 172.21.15.8:0/3505852596 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc0a3670 con 0x55ed25602b70 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9ed97fa640 1 -- 172.21.15.8:0/3505852596 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ecc0991d0 con 0x55ed25602b70 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.608+0000 7f9ed97fa640 1 -- 172.21.15.8:0/3505852596 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc0b9260 con 0x55ed25602b70 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- 172.21.15.8:0/3505852596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 msgr2=0x55ed255f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed255f7a10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00e9c0 tx=0x7f9ecc01a040 comp rx=0 tx=0).stop 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- 172.21.15.8:0/3505852596 shutdown_connections 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2560b750 0x55ed25608cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed255f7a10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/3505852596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed25602250 0x55ed2560aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- 172.21.15.8:0/3505852596 >> 172.21.15.8:0/3505852596 conn(0x55ed255fc980 msgr2=0x55ed255fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- 172.21.15.8:0/3505852596 shutdown_connections 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- 172.21.15.8:0/3505852596 wait complete. 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- start start 2024-02-19T16:27:34.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 Processor -- start 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed2560f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2566f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2566f750 con 0x55ed25602b70 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2566f8c0 con 0x55ed2560b750 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2566fa80 con 0x55ed25602250 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9eeca4f740 10 client.0 _ll_register_callbacks cb 0x7f9ecc096a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:34.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9edaffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49744/0 (socket says 172.21.15.8:49744) 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.609+0000 7f9edaffd640 1 -- 172.21.15.8:0/171017885 learned_addr learned my addr 172.21.15.8:0/171017885 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed2560f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ee3fff640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2566f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9edaffd640 1 -- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 msgr2=0x55ed2560f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9edaffd640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed2560f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9edaffd640 1 -- 172.21.15.8:0/171017885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 msgr2=0x55ed2566f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9edaffd640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2566f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9edaffd640 1 -- 172.21.15.8:0/171017885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed255686a0 con 0x55ed25602b70 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9edaffd640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560fae0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc096bb0 tx=0x7f9ecc00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346f6e3aad9e857e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ec2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/171017885 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc097170 con 0x55ed25602b70 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/171017885 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f9ecc0a3670 con 0x55ed25602b70 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed255f80f0 con 0x55ed25602b70 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/171017885 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ecc0975b0 con 0x55ed25602b70 2024-02-19T16:27:34.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9eeca4f740 10 client.16710 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f9ed4009560 con 0x55ed25602b70 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/171017885 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed255f80f0 con 0x55ed25602b70 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.610+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/171017885 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ecc09a320 con 0x55ed25602b70 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 10 client.16710 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 20 client.16710 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed25677fa0 0x55ed2567a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:34.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 4 client.16710 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9ed402c9f0 con 0x55ed25677fa0 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed25677fa0 0x55ed2567a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9ee37fe640 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed25677fa0 0x55ed2567a3c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e899ed2600dcbfb6 server_cookie=680d85f87f78964f in_seq=0 out_seq=0 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9ec2ffd640 10 client.16710 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9ec2ffd640 1 -- 172.21.15.8:0/171017885 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9ed402c9f0 con 0x55ed25677fa0 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9ec2ffd640 10 client.16710 handle_command_reply: tid=0 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 client.16710 shutdown 2024-02-19T16:27:34.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 msgr2=0x55ed2560fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560fae0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc096bb0 tx=0x7f9ecc00cef0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 shutdown_connections 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2560b750 0x55ed2566f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed25602b70 0x55ed2560fae0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ed25677fa0 0x55ed2567a3c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 --2- 172.21.15.8:0/171017885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed25602250 0x55ed2560f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9ec2ffd640 0 client.16710 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:34.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 >> 172.21.15.8:0/171017885 conn(0x55ed255fc980 msgr2=0x55ed255fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 shutdown_connections 2024-02-19T16:27:34.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.611+0000 7f9eeca4f740 1 -- 172.21.15.8:0/171017885 wait complete. 2024-02-19T16:27:34.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9eeca4f740 20 client.16710 trim_cache size 0 max 16384 2024-02-19T16:27:34.630 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9ed00a2230 msgr2=0x7f9ed00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9ed00a2230 0x7f9ed00a46f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009270 tx=0x7f9edc012bb0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 msgr2=0x7f9ee419dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 0x7f9ee419dc30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00e9f0 tx=0x7f9ecc00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 shutdown_connections 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee4107560 0x7f9ee41978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ee4101e60 0x7f9ee419dc30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9ed00a2230 0x7f9ed00a46f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 --2- 172.21.15.8:0/3875816295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ee41014a0 0x7f9ee419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:34.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 >> 172.21.15.8:0/3875816295 conn(0x7f9ee40788f0 msgr2=0x7f9ee4100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:34.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 shutdown_connections 2024-02-19T16:27:34.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:34.612+0000 7f9ee9e76640 1 -- 172.21.15.8:0/3875816295 wait complete. 2024-02-19T16:27:35.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:34 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/296224262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:27:35.177 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:34 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/296224262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:27:35.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:34 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/296224262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:27:35.634 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:27:35.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:27:35.635 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:27:35.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 -- 172.21.15.8:0/1014717827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a8100c90 msgr2=0x7fd6a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 --2- 172.21.15.8:0/1014717827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a8100c90 0x7fd6a810d950 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd698009900 tx=0x7fd69802bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:35.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 -- 172.21.15.8:0/1014717827 shutdown_connections 2024-02-19T16:27:35.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 --2- 172.21.15.8:0/1014717827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a810de90 0x7fd6a8110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 --2- 172.21.15.8:0/1014717827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a8100c90 0x7fd6a810d950 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 --2- 172.21.15.8:0/1014717827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6a81002e0 0x7fd6a81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.753+0000 7fd6afab5640 1 -- 172.21.15.8:0/1014717827 >> 172.21.15.8:0/1014717827 conn(0x7fd6a8077550 msgr2=0x7fd6a8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 -- 172.21.15.8:0/1014717827 shutdown_connections 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 -- 172.21.15.8:0/1014717827 wait complete. 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 Processor -- start 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 -- start start 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6a81002e0 0x7fd6a8199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 0x7fd6a819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 0x7fd6a819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a819ec90 con 0x7fd6a810de90 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a819ee00 con 0x7fd6a8100c90 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6afab5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a819efc0 con 0x7fd6a81002e0 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 0x7fd6a819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.754+0000 7fd6ad050640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 0x7fd6a819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad851640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6a81002e0 0x7fd6a8199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 0x7fd6a819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:32962/0 (socket says 172.21.15.8:32962) 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 0x7fd6a819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46840/0 (socket says 172.21.15.8:46840) 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 -- 172.21.15.8:0/683204171 learned_addr learned my addr 172.21.15.8:0/683204171 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 -- 172.21.15.8:0/683204171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6a81002e0 msgr2=0x7fd6a8199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6a81002e0 0x7fd6a8199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 -- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 msgr2=0x7fd6a819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 0x7fd6a819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 -- 172.21.15.8:0/683204171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009560 con 0x7fd6a8100c90 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ae052640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 0x7fd6a819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:35.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6ad050640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 0x7fd6a819a310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd698005940 tx=0x7fd69802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd69effd640 1 -- 172.21.15.8:0/683204171 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698004350 con 0x7fd6a8100c90 2024-02-19T16:27:35.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a819f240 con 0x7fd6a8100c90 2024-02-19T16:27:35.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd69effd640 1 -- 172.21.15.8:0/683204171 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6980044b0 con 0x7fd6a8100c90 2024-02-19T16:27:35.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd69effd640 1 -- 172.21.15.8:0/683204171 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698003d30 con 0x7fd6a8100c90 2024-02-19T16:27:35.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.755+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a81a6b20 con 0x7fd6a8100c90 2024-02-19T16:27:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.757+0000 7fd69effd640 1 -- 172.21.15.8:0/683204171 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fd698004df0 con 0x7fd6a8100c90 2024-02-19T16:27:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 Processor -- start 2024-02-19T16:27:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 -- start start 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd69effd640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6940a2230 0x7fd6940a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9ae380 0x55b20b9ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b91cf90 con 0x55b20b9ae380 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b91a4c0 con 0x55b20b9aed30 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6b2769740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b915ca0 con 0x55b20b9b78b0 2024-02-19T16:27:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9ae380 0x55b20b9ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6ad851640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6940a2230 0x7fd6940a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6ad050640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9ae380 0x55b20b9ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46848/0 (socket says 172.21.15.8:46848) 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.758+0000 7fd6ae052640 1 -- 172.21.15.8:0/1801786378 learned_addr learned my addr 172.21.15.8:0/1801786378 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ae052640 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd69effd640 1 -- 172.21.15.8:0/683204171 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd6980f03f0 con 0x7fd6a8100c90 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad050640 1 -- 172.21.15.8:0/1801786378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 msgr2=0x55b20b9b9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad050640 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad050640 1 -- 172.21.15.8:0/1801786378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9ae380 msgr2=0x55b20b9ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad050640 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9ae380 0x55b20b9ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad050640 1 -- 172.21.15.8:0/1801786378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b20b8d80b0 con 0x55b20b9aed30 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ae052640 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad851640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6940a2230 0x7fd6940a46f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009bd0 tx=0x7fd6a4005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd6ad050640 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b72e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ed8a0 tx=0x7fd6980037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1942be8df4ce1a9c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd693fff640 1 -- 172.21.15.8:0/1801786378 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980bc4a0 con 0x55b20b9aed30 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd693fff640 1 -- 172.21.15.8:0/1801786378 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6980c9cf0 con 0x55b20b9aed30 2024-02-19T16:27:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.759+0000 7fd693fff640 1 -- 172.21.15.8:0/1801786378 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980c0d00 con 0x55b20b9aed30 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 -- 172.21.15.8:0/1801786378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 msgr2=0x55b20b9b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b72e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ed8a0 tx=0x7fd6980037d0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 -- 172.21.15.8:0/1801786378 shutdown_connections 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b72e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 --2- 172.21.15.8:0/1801786378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9ae380 0x55b20b9ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 -- 172.21.15.8:0/1801786378 >> 172.21.15.8:0/1801786378 conn(0x55b20b9a8840 msgr2=0x55b20b9a8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 -- 172.21.15.8:0/1801786378 shutdown_connections 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 -- 172.21.15.8:0/1801786378 wait complete. 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 -- start start 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.760+0000 7fd6b2769740 1 Processor -- start 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9ae380 0x55b20b9b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b8baf40 con 0x55b20b9b78b0 2024-02-19T16:27:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b5e94b0 con 0x55b20b9aed30 2024-02-19T16:27:35.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b89b670 con 0x55b20b9ae380 2024-02-19T16:27:35.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6b2769740 10 client.0 _ll_register_callbacks cb 0x55b20b908f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:35.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ad851640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9ae380 0x55b20b9b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ad851640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9ae380 0x55b20b9b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:48584/0 (socket says 172.21.15.8:48584) 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ad851640 1 -- 172.21.15.8:0/3040056805 learned_addr learned my addr 172.21.15.8:0/3040056805 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46858/0 (socket says 172.21.15.8:46858) 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ad050640 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 -- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9ae380 msgr2=0x55b20b9b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9ae380 0x55b20b9b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 -- 172.21.15.8:0/3040056805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 msgr2=0x55b20b9b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 -- 172.21.15.8:0/3040056805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a81a6b20 con 0x55b20b9b78b0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.761+0000 7fd6ae052640 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9b5240 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00099e0 tx=0x7fd6a00108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a5b4c2cd6995e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6917fa640 1 -- 172.21.15.8:0/3040056805 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a001e630 con 0x55b20b9b78b0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6917fa640 1 -- 172.21.15.8:0/3040056805 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd6a001e790 con 0x55b20b9b78b0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b20b8eff40 con 0x55b20b9b78b0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6917fa640 1 -- 172.21.15.8:0/3040056805 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a001e950 con 0x55b20b9b78b0 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 10 client.16722 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b20b914320 con 0x55b20b9b78b0 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6917fa640 1 -- 172.21.15.8:0/3040056805 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b20b8eff40 con 0x55b20b9b78b0 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6917fa640 1 -- 172.21.15.8:0/3040056805 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd6a000fb50 con 0x55b20b9b78b0 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 10 client.16722 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 20 client.16722 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20b9c7cd0 0x55b20b9ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 4 client.16722 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b20b90c310 con 0x55b20b9c7cd0 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.762+0000 7fd6ad851640 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20b9c7cd0 0x55b20b9ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.763+0000 7fd6ad851640 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20b9c7cd0 0x55b20b9ca0f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41967e86f840b040 server_cookie=3e87aa167f417171 in_seq=0 out_seq=0 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.763+0000 7fd6917fa640 10 client.16722 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6917fa640 1 -- 172.21.15.8:0/3040056805 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b20b90c310 con 0x55b20b9c7cd0 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6917fa640 10 client.16722 handle_command_reply: tid=0 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 client.16722 shutdown 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 msgr2=0x55b20b9b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9b5240 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00099e0 tx=0x7fd6a00108c0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 shutdown_connections 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9b5240 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9aed30 0x55b20b9b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20b9c7cd0 0x55b20b9ca0f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 --2- 172.21.15.8:0/3040056805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9ae380 0x55b20b9b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6917fa640 0 client.16722 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 >> 172.21.15.8:0/3040056805 conn(0x55b20b9a8840 msgr2=0x55b20b9aa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 shutdown_connections 2024-02-19T16:27:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 1 -- 172.21.15.8:0/3040056805 wait complete. 2024-02-19T16:27:35.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.764+0000 7fd6b2769740 20 client.16722 trim_cache size 0 max 16384 2024-02-19T16:27:35.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.780+0000 7fd6b2769740 1 Processor -- start 2024-02-19T16:27:35.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.780+0000 7fd6b2769740 1 -- start start 2024-02-19T16:27:35.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.780+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.780+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9aed30 0x55b20b9b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.780+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6b2769740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b90c310 con 0x55b20b9b78b0 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6b2769740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b8eff40 con 0x55b20b9ae380 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6b2769740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b9ca750 con 0x55b20b9aed30 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9c2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46862/0 (socket says 172.21.15.8:46862) 2024-02-19T16:27:35.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 -- 172.21.15.8:0/2838049449 learned_addr learned my addr 172.21.15.8:0/2838049449 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ad050640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9aed30 0x55b20b9b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ad851640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 -- 172.21.15.8:0/2838049449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9aed30 msgr2=0x55b20b9b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9aed30 0x55b20b9b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 -- 172.21.15.8:0/2838049449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 msgr2=0x55b20b9b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 -- 172.21.15.8:0/2838049449 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b20b8b0c00 con 0x55b20b9b78b0 2024-02-19T16:27:35.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ad050640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9aed30 0x55b20b9b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ad851640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6ae052640 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9c2bd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0013720 tx=0x7fd6a0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b6f491be0ccfcf server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6937fe640 1 -- 172.21.15.8:0/2838049449 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0003b70 con 0x55b20b9b78b0 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6937fe640 1 -- 172.21.15.8:0/2838049449 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0013bc0 con 0x55b20b9b78b0 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.781+0000 7fd6937fe640 1 -- 172.21.15.8:0/2838049449 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000f6d0 con 0x55b20b9b78b0 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 -- 172.21.15.8:0/2838049449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 msgr2=0x55b20b9c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9c2bd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0013720 tx=0x7fd6a0009480 comp rx=0 tx=0).stop 2024-02-19T16:27:35.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 -- 172.21.15.8:0/2838049449 shutdown_connections 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9b78b0 0x55b20b9c2bd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9aed30 0x55b20b9b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 --2- 172.21.15.8:0/2838049449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 -- 172.21.15.8:0/2838049449 >> 172.21.15.8:0/2838049449 conn(0x55b20b9a8840 msgr2=0x55b20b9a78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 -- 172.21.15.8:0/2838049449 shutdown_connections 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 -- 172.21.15.8:0/2838049449 wait complete. 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 -- start start 2024-02-19T16:27:35.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.782+0000 7fd6b2769740 1 Processor -- start 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9bee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 0x55b20b9bf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b9b1cc0 con 0x55b20b9aed30 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b9b1e30 con 0x55b20b9ae380 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b20b9b1ff0 con 0x55b20b9b78b0 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 10 client.0 _ll_register_callbacks cb 0x7fd6a0013800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:35.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 0x55b20b9bf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 0x55b20b9bf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46866/0 (socket says 172.21.15.8:46866) 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 -- 172.21.15.8:0/11645764 learned_addr learned my addr 172.21.15.8:0/11645764 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ae052640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad851640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 -- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 msgr2=0x55b20b9b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 -- 172.21.15.8:0/11645764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 msgr2=0x55b20b9bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 -- 172.21.15.8:0/11645764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b20b914320 con 0x55b20b9aed30 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6ad050640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 0x55b20b9bf3d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ed950 tx=0x7fd6980bc990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee05ed95d667159 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6917fa640 1 -- 172.21.15.8:0/11645764 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980caea0 con 0x55b20b9aed30 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6917fa640 1 -- 172.21.15.8:0/11645764 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fd6980bd240 con 0x55b20b9aed30 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b20b615860 con 0x55b20b9aed30 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.783+0000 7fd6917fa640 1 -- 172.21.15.8:0/11645764 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6980bc4a0 con 0x55b20b9aed30 2024-02-19T16:27:35.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 10 client.16728 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b20b8d6910 con 0x55b20b9aed30 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6917fa640 1 -- 172.21.15.8:0/11645764 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b20b615860 con 0x55b20b9aed30 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6917fa640 1 -- 172.21.15.8:0/11645764 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd6980dc070 con 0x55b20b9aed30 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 10 client.16728 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 20 client.16728 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20ba24250 0x55b20ba26640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 4 client.16728 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:27:35.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b20b666410 con 0x55b20ba24250 2024-02-19T16:27:35.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6ad851640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20ba24250 0x55b20ba26640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6917fa640 10 client.16728 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.784+0000 7fd6ad851640 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20ba24250 0x55b20ba26640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47e25b179c42443 server_cookie=3d52f1e12bc111e3 in_seq=0 out_seq=0 2024-02-19T16:27:35.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6917fa640 1 -- 172.21.15.8:0/11645764 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b20b666410 con 0x55b20ba24250 2024-02-19T16:27:35.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6917fa640 10 client.16728 handle_command_reply: tid=0 2024-02-19T16:27:35.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 client.16728 shutdown 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 msgr2=0x55b20b9bf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 0x55b20b9bf3d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ed950 tx=0x7fd6980bc990 comp rx=0 tx=0).stop 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 shutdown_connections 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55b20ba24250 0x55b20ba26640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b20b9b78b0 0x55b20b9b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b20b9aed30 0x55b20b9bf3d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 --2- 172.21.15.8:0/11645764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b20b9ae380 0x55b20b9bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6917fa640 0 client.16728 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 >> 172.21.15.8:0/11645764 conn(0x55b20b9a8840 msgr2=0x55b20b9a78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 shutdown_connections 2024-02-19T16:27:35.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 1 -- 172.21.15.8:0/11645764 wait complete. 2024-02-19T16:27:35.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.786+0000 7fd6b2769740 20 client.16728 trim_cache size 0 max 16384 2024-02-19T16:27:35.798 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:27:35.798 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:27:35.799 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "777f87e6-2387-450c-a077-5158be063a4b", 2024-02-19T16:27:35.799 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:27:35.799 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:27:35.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6940a2230 msgr2=0x7fd6940a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6940a2230 0x7fd6940a46f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009bd0 tx=0x7fd6a4005f90 comp rx=0 tx=0).stop 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 msgr2=0x7fd6a819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 0x7fd6a819a310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd698005940 tx=0x7fd69802c670 comp rx=0 tx=0).stop 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 shutdown_connections 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6a810de90 0x7fd6a819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a8100c90 0x7fd6a819a310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6940a2230 0x7fd6940a46f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 --2- 172.21.15.8:0/683204171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6a81002e0 0x7fd6a8199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 >> 172.21.15.8:0/683204171 conn(0x7fd6a8077550 msgr2=0x7fd6a810f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 shutdown_connections 2024-02-19T16:27:35.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.787+0000 7fd6afab5640 1 -- 172.21.15.8:0/683204171 wait complete. 2024-02-19T16:27:35.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:27:35.802 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:27:35.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/3213326875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 msgr2=0x7fb9c41078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/3213326875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c41078f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009930 tx=0x7fb9b402bc30 comp rx=0 tx=0).stop 2024-02-19T16:27:35.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/3213326875 shutdown_connections 2024-02-19T16:27:35.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/3213326875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 0x7fb9c410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/3213326875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 0x7fb9c40ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/3213326875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c41078f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/3213326875 >> 172.21.15.8:0/3213326875 conn(0x7fb9c40769b0 msgr2=0x7fb9c40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/3213326875 shutdown_connections 2024-02-19T16:27:35.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.907+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/3213326875 wait complete. 2024-02-19T16:27:35.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 Processor -- start 2024-02-19T16:27:35.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 -- start start 2024-02-19T16:27:35.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 0x7fb9c419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 0x7fb9c419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4197ec0 con 0x7fb9c4107510 2024-02-19T16:27:35.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4198030 con 0x7fb9c41003b0 2024-02-19T16:27:35.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.908+0000 7fb9c9d6c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41981f0 con 0x7fb9c40ff9f0 2024-02-19T16:27:35.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 0x7fb9c419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 0x7fb9c419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 0x7fb9c419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:48620/0 (socket says 172.21.15.8:48620) 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c37fe640 1 -- 172.21.15.8:0/887119064 learned_addr learned my addr 172.21.15.8:0/887119064 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c4197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46894/0 (socket says 172.21.15.8:46894) 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 0x7fb9c419dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:33016/0 (socket says 172.21.15.8:33016) 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 -- 172.21.15.8:0/887119064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 msgr2=0x7fb9c419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 0x7fb9c419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 -- 172.21.15.8:0/887119064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 msgr2=0x7fb9c419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 0x7fb9c419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 -- 172.21.15.8:0/887119064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b4009560 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c3fff640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c4197870 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009fd0 tx=0x7fb9bc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c0ff9640 1 -- 172.21.15.8:0/887119064 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00ce20 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c41984d0 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c0ff9640 1 -- 172.21.15.8:0/887119064 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc005020 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c0ff9640 1 -- 172.21.15.8:0/887119064 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc010690 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.909+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c41a4760 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9c0ff9640 1 -- 172.21.15.8:0/887119064 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb9bc005500 con 0x7fb9c4107510 2024-02-19T16:27:35.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 Processor -- start 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 -- start start 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c606fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c606febd0 0x559c60707270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c607077b0 0x559c60709ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6066d280 con 0x559c607077b0 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6066a7b0 con 0x559c606febd0 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.912+0000 7fb9cca21740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60665f90 con 0x559c606fe2b0 2024-02-19T16:27:35.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c0ff9640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb9b00a2230 0x7fb9b00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c606fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c606febd0 0x559c60707270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c606fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:48626/0 (socket says 172.21.15.8:48626) 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/1805465246 learned_addr learned my addr 172.21.15.8:0/1805465246 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c607077b0 0x559c60709ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c3fff640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb9b00a2230 0x7fb9b00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c0ff9640 1 -- 172.21.15.8:0/887119064 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb9bc014070 con 0x7fb9c4107510 2024-02-19T16:27:35.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/1805465246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c606febd0 msgr2=0x559c60707270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c606febd0 0x559c60707270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/1805465246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c607077b0 msgr2=0x559c60709ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c607077b0 0x559c60709ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/1805465246 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c60624570 con 0x559c606fe2b0 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c606febd0 0x559c60707270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c606fe690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0058c0 tx=0x7fb9ac005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2333b25b2136c6b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9c3fff640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb9b00a2230 0x7fb9b00a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0cc020 tx=0x7fb9bc06d1d0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9b9ffb640 1 -- 172.21.15.8:0/1805465246 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004030 con 0x559c606fe2b0 2024-02-19T16:27:35.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9b9ffb640 1 -- 172.21.15.8:0/1805465246 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac005020 con 0x559c606fe2b0 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.913+0000 7fb9b9ffb640 1 -- 172.21.15.8:0/1805465246 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004d40 con 0x559c606fe2b0 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 -- 172.21.15.8:0/1805465246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 msgr2=0x559c606fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c606fe690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0058c0 tx=0x7fb9ac005850 comp rx=0 tx=0).stop 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 -- 172.21.15.8:0/1805465246 shutdown_connections 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c607077b0 0x559c60709ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c606febd0 0x559c60707270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 --2- 172.21.15.8:0/1805465246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c606fe690 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 -- 172.21.15.8:0/1805465246 >> 172.21.15.8:0/1805465246 conn(0x559c606f8990 msgr2=0x559c606f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 -- 172.21.15.8:0/1805465246 shutdown_connections 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 -- 172.21.15.8:0/1805465246 wait complete. 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.914+0000 7fb9cca21740 1 -- start start 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 Processor -- start 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60704540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c60704b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60339430 con 0x559c606febd0 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c602f5e00 con 0x559c607077b0 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c605eafd0 con 0x559c606fe2b0 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9cca21740 10 client.0 _ll_register_callbacks cb 0x559c60640fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60704540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c60704b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c60704b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:33040/0 (socket says 172.21.15.8:33040) 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/247641428 learned_addr learned my addr 172.21.15.8:0/247641428 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60704540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46906/0 (socket says 172.21.15.8:46906) 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.915+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 msgr2=0x559c607071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/247641428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 msgr2=0x559c60704540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60704540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/247641428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac019040 con 0x559c607077b0 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60704540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:35.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c60704b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009800 tx=0x7fb9ac0048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735bb41845ac5728 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/247641428 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac005510 con 0x559c607077b0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c603657e0 con 0x559c607077b0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/247641428 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb9ac004020 con 0x559c607077b0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/247641428 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ac0041e0 con 0x559c607077b0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9cca21740 10 client.26507 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559c60664610 con 0x559c607077b0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.916+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/247641428 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559c603657e0 con 0x559c607077b0 2024-02-19T16:27:35.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/247641428 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb9ac01d070 con 0x559c607077b0 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9cca21740 10 client.26507 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9cca21740 10 client.26507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9cca21740 20 client.26507 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9cca21740 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60717800 0x559c60719bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9cca21740 4 client.26507 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c603a2720 con 0x559c60717800 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.917+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60717800 0x559c60719bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.918+0000 7fb9a2ffd640 10 client.26507 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.918+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60717800 0x559c60719bf0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db0304a1da63045 server_cookie=fd12ebc65622909b in_seq=0 out_seq=0 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.918+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/247641428 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559c603a2720 con 0x559c60717800 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.918+0000 7fb9a2ffd640 10 client.26507 handle_command_reply: tid=0 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 client.26507 shutdown 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 msgr2=0x559c60704b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c60704b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009800 tx=0x7fb9ac0048e0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 shutdown_connections 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c60704b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60704540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60717800 0x559c60719bf0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 --2- 172.21.15.8:0/247641428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9a2ffd640 0 client.26507 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 >> 172.21.15.8:0/247641428 conn(0x559c606f8990 msgr2=0x559c606fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 shutdown_connections 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 1 -- 172.21.15.8:0/247641428 wait complete. 2024-02-19T16:27:35.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.919+0000 7fb9cca21740 20 client.26507 trim_cache size 0 max 16384 2024-02-19T16:27:35.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.937+0000 7fb9cca21740 1 Processor -- start 2024-02-19T16:27:35.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.937+0000 7fb9cca21740 1 -- start start 2024-02-19T16:27:35.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c60703520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60706c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c607071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9cca21740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c606f8210 con 0x559c606febd0 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9cca21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60709f20 con 0x559c607077b0 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9cca21740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6071a250 con 0x559c606fe2b0 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60706c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60706c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46918/0 (socket says 172.21.15.8:46918) 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c607071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 -- 172.21.15.8:0/509324713 learned_addr learned my addr 172.21.15.8:0/509324713 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c60703520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 -- 172.21.15.8:0/509324713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 msgr2=0x559c60703520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c60703520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 -- 172.21.15.8:0/509324713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 msgr2=0x559c607071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c607071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 -- 172.21.15.8:0/509324713 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c60600ac0 con 0x559c606febd0 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c607071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60706c70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009430 tx=0x7fb9b40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359edc8b4ac7dd20 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.938+0000 7fb9b97fa640 1 -- 172.21.15.8:0/509324713 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4043040 con 0x559c606febd0 2024-02-19T16:27:35.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9b97fa640 1 -- 172.21.15.8:0/509324713 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4044040 con 0x559c606febd0 2024-02-19T16:27:35.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9b97fa640 1 -- 172.21.15.8:0/509324713 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b404e900 con 0x559c606febd0 2024-02-19T16:27:35.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 -- 172.21.15.8:0/509324713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 msgr2=0x559c60706c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60706c70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009430 tx=0x7fb9b40058d0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 -- 172.21.15.8:0/509324713 shutdown_connections 2024-02-19T16:27:35.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c607071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60706c70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 --2- 172.21.15.8:0/509324713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c60703520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 -- 172.21.15.8:0/509324713 >> 172.21.15.8:0/509324713 conn(0x559c606f8990 msgr2=0x559c60702e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 -- 172.21.15.8:0/509324713 shutdown_connections 2024-02-19T16:27:35.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.939+0000 7fb9cca21740 1 -- 172.21.15.8:0/509324713 wait complete. 2024-02-19T16:27:35.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 -- start start 2024-02-19T16:27:35.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 Processor -- start 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60701af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c6070d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6070def0 con 0x559c606febd0 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6070e060 con 0x559c607077b0 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6070e220 con 0x559c606fe2b0 2024-02-19T16:27:35.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9cca21740 10 client.0 _ll_register_callbacks cb 0x7fb9ac009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60701af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c6070d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c6070d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:33058/0 (socket says 172.21.15.8:33058) 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/3406062000 learned_addr learned my addr 172.21.15.8:0/3406062000 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60701af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46930/0 (socket says 172.21.15.8:46930) 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.940+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 msgr2=0x559c607015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/3406062000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 msgr2=0x559c60701af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60701af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c2ffd640 1 -- 172.21.15.8:0/3406062000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c60664610 con 0x559c607077b0 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60701af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9c2ffd640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c6070d9b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac004fd0 tx=0x7fb9ac0048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa861099e54ea5c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:27:35.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/3406062000 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac023040 con 0x559c607077b0 2024-02-19T16:27:35.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9ac005020 con 0x559c607077b0 2024-02-19T16:27:35.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/3406062000 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fb9ac000e40 con 0x559c607077b0 2024-02-19T16:27:35.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/3406062000 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ac011270 con 0x559c607077b0 2024-02-19T16:27:35.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9cca21740 10 client.26513 fetch_fsmap learned FSMap version 30 2024-02-19T16:27:35.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb9b4030070 con 0x559c607077b0 2024-02-19T16:27:35.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.941+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/3406062000 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac047020 con 0x559c607077b0 2024-02-19T16:27:35.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/3406062000 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb9ac01d070 con 0x559c607077b0 2024-02-19T16:27:35.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9cca21740 10 client.26513 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:27:35.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9cca21740 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:27:35.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9cca21740 20 client.26513 populate_metadata read hostname 'smithi008' 2024-02-19T16:27:35.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9cca21740 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60773fd0 0x559c607763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:27:35.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9cca21740 4 client.26513 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c603f1890 con 0x559c60773fd0 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.942+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60773fd0 0x559c607763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.943+0000 7fb9c37fe640 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60773fd0 0x559c607763f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f492156d764f7b server_cookie=984479170dc6ff9 in_seq=0 out_seq=0 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.943+0000 7fb9a2ffd640 10 client.26513 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.943+0000 7fb9a2ffd640 1 -- 172.21.15.8:0/3406062000 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x559c603f1890 con 0x559c60773fd0 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.943+0000 7fb9a2ffd640 10 client.26513 handle_command_reply: tid=0 2024-02-19T16:27:35.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.943+0000 7fb9cca21740 1 client.26513 shutdown 2024-02-19T16:27:35.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 msgr2=0x559c6070d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c6070d9b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac004fd0 tx=0x7fb9ac0048e0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 shutdown_connections 2024-02-19T16:27:35.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c607077b0 0x559c6070d9b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559c606febd0 0x559c60701af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559c60773fd0 0x559c607763f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 --2- 172.21.15.8:0/3406062000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559c606fe2b0 0x559c607015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 >> 172.21.15.8:0/3406062000 conn(0x559c606f8990 msgr2=0x559c607036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9a2ffd640 0 client.26513 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:27:35.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 shutdown_connections 2024-02-19T16:27:35.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 1 -- 172.21.15.8:0/3406062000 wait complete. 2024-02-19T16:27:35.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9cca21740 20 client.26513 trim_cache size 0 max 16384 2024-02-19T16:27:35.958 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:27:35.958 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2774 inodes in the stack)", 2024-02-19T16:27:35.958 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:27:35.958 INFO:teuthology.orchestra.run.smithi008.stdout: "777f87e6-2387-450c-a077-5158be063a4b": { 2024-02-19T16:27:35.959 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:27:35.959 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "777f87e6-2387-450c-a077-5158be063a4b", 2024-02-19T16:27:35.959 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:27:35.959 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:27:35.959 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:27:35.959 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:27:35.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb9b00a2230 msgr2=0x7fb9b00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb9b00a2230 0x7fb9b00a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0cc020 tx=0x7fb9bc06d1d0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 msgr2=0x7fb9c4197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:27:35.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c4197870 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009fd0 tx=0x7fb9bc00ef10 comp rx=0 tx=0).stop 2024-02-19T16:27:35.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 shutdown_connections 2024-02-19T16:27:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9c4107510 0x7fb9c4197870 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c41003b0 0x7fb9c419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb9b00a2230 0x7fb9b00a46f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 --2- 172.21.15.8:0/887119064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9c40ff9f0 0x7fb9c419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:27:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.944+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 >> 172.21.15.8:0/887119064 conn(0x7fb9c40769b0 msgr2=0x7fb9c40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:27:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.945+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 shutdown_connections 2024-02-19T16:27:35.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:27:35.945+0000 7fb9c9d6c640 1 -- 172.21.15.8:0/887119064 wait complete. 2024-02-19T16:27:35.963 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:35 smithi008 ceph-mon[32573]: pgmap v144: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:35.964 INFO:tasks.cephfs.filesystem:scrub status for tag:777f87e6-2387-450c-a077-5158be063a4b - {'path': '/', 'tag': '777f87e6-2387-450c-a077-5158be063a4b', 'options': 'recursive,force'} 2024-02-19T16:27:36.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:35 smithi156 ceph-mon[36502]: pgmap v144: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:36.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:35 smithi191 ceph-mon[37203]: pgmap v144: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:37.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:27:37.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:37.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:37.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:37.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:37.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:27:37.082 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:36 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:27:37.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:37.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:37.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:37.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:37.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:27:37.272 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:36 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:27:37.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:27:37.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:36 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:38.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:37 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:38.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:37 smithi156 ceph-mon[36502]: pgmap v145: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:38.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:37 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:38.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:37 smithi008 ceph-mon[32573]: pgmap v145: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:38.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:37 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:27:38.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:37 smithi191 ceph-mon[37203]: pgmap v145: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:40.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:39 smithi156 ceph-mon[36502]: pgmap v146: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:40.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:39 smithi008 ceph-mon[32573]: pgmap v146: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:40.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:39 smithi191 ceph-mon[37203]: pgmap v146: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:42.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:41 smithi156 ceph-mon[36502]: pgmap v147: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:42.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:41 smithi008 ceph-mon[32573]: pgmap v147: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:42.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:41 smithi191 ceph-mon[37203]: pgmap v147: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:44.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:43 smithi156 ceph-mon[36502]: pgmap v148: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:44.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:43 smithi008 ceph-mon[32573]: pgmap v148: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:44.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:43 smithi191 ceph-mon[37203]: pgmap v148: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:44 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:27:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:44 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:27:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:44 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:27:45.277 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:44 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:27:45.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:44 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:27:45.278 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:44 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:27:45.280 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:44 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:27:45.281 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:44 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:27:45.281 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:44 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:27:46.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:45 smithi156 ceph-mon[36502]: pgmap v149: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:46.171 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:45 smithi008 ceph-mon[32573]: pgmap v149: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:46.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:45 smithi191 ceph-mon[37203]: pgmap v149: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:48.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:47 smithi156 ceph-mon[36502]: pgmap v150: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:48.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:47 smithi008 ceph-mon[32573]: pgmap v150: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:48.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:47 smithi191 ceph-mon[37203]: pgmap v150: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:27:50.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:49 smithi156 ceph-mon[36502]: pgmap v151: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:50.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:49 smithi008 ceph-mon[32573]: pgmap v151: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:50.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:49 smithi191 ceph-mon[37203]: pgmap v151: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:27:52.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:51 smithi008 ceph-mon[32573]: pgmap v152: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:52.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:51 smithi191 ceph-mon[37203]: pgmap v152: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:52.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:51 smithi156 ceph-mon[36502]: pgmap v152: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:54.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:53 smithi008 ceph-mon[32573]: pgmap v153: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:54.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:53 smithi191 ceph-mon[37203]: pgmap v153: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:54.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:53 smithi156 ceph-mon[36502]: pgmap v153: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:56.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:55 smithi008 ceph-mon[32573]: pgmap v154: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:56.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:55 smithi191 ceph-mon[37203]: pgmap v154: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:56.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:55 smithi156 ceph-mon[36502]: pgmap v154: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:27:58.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:57 smithi008 ceph-mon[32573]: pgmap v155: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:58.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:57 smithi191 ceph-mon[37203]: pgmap v155: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:27:58.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:57 smithi156 ceph-mon[36502]: pgmap v155: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:00.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:27:59 smithi008 ceph-mon[32573]: pgmap v156: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:00.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:27:59 smithi191 ceph-mon[37203]: pgmap v156: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:00.329 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:27:59 smithi156 ceph-mon[36502]: pgmap v156: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:01 smithi008 ceph-mon[32573]: pgmap v157: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:01 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:28:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:01 smithi191 ceph-mon[37203]: pgmap v157: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:02.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:01 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:28:02.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:01 smithi156 ceph-mon[36502]: pgmap v157: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:02.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:01 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:28:03.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:02 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:28:03.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:02 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:28:03.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:02 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:28:03.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:02 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:28:03.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:02 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:28:03.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:02 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:28:03.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:02 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:28:03.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:02 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:28:03.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:02 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:28:04.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:03 smithi008 ceph-mon[32573]: pgmap v158: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:04.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:03 smithi191 ceph-mon[37203]: pgmap v158: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:04.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:03 smithi156 ceph-mon[36502]: pgmap v158: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:05.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:28:05.965 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:28:06.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 -- 172.21.15.8:0/1754465509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0100640 msgr2=0x7f11f0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 --2- 172.21.15.8:0/1754465509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0100640 0x7f11f0100ac0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009900 tx=0x7f11e402bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:06.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 -- 172.21.15.8:0/1754465509 shutdown_connections 2024-02-19T16:28:06.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 --2- 172.21.15.8:0/1754465509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f0101000 0x7f11f010d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 --2- 172.21.15.8:0/1754465509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0100640 0x7f11f0100ac0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 --2- 172.21.15.8:0/1754465509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11f0106480 0x7f11f0106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 -- 172.21.15.8:0/1754465509 >> 172.21.15.8:0/1754465509 conn(0x7f11f0077190 msgr2=0x7f11f00775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 -- 172.21.15.8:0/1754465509 shutdown_connections 2024-02-19T16:28:06.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.081+0000 7f11f5381640 1 -- 172.21.15.8:0/1754465509 wait complete. 2024-02-19T16:28:06.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 Processor -- start 2024-02-19T16:28:06.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 -- start start 2024-02-19T16:28:06.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11f0100640 0x7f11f010dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 0x7f11f010e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f0106480 0x7f11f010e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01a5eb0 con 0x7f11f0101000 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01a6020 con 0x7f11f0106480 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11f5381640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01a61e0 con 0x7f11f0100640 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 0x7f11f010e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 0x7f11f010e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38798/0 (socket says 172.21.15.8:38798) 2024-02-19T16:28:06.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 -- 172.21.15.8:0/1400083633 learned_addr learned my addr 172.21.15.8:0/1400083633 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f0106480 0x7f11f010e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11eeffd640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11f0100640 0x7f11f010dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 -- 172.21.15.8:0/1400083633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11f0100640 msgr2=0x7f11f010dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11f0100640 0x7f11f010dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 -- 172.21.15.8:0/1400083633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f0106480 msgr2=0x7f11f010e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f0106480 0x7f11f010e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.082+0000 7f11ee7fc640 1 -- 172.21.15.8:0/1400083633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4009560 con 0x7f11f0101000 2024-02-19T16:28:06.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.083+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 0x7f11f010e110 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f11e4005940 tx=0x7f11e402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.083+0000 7f11d7fff640 1 -- 172.21.15.8:0/1400083633 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4004350 con 0x7f11f0101000 2024-02-19T16:28:06.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.083+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f01a6460 con 0x7f11f0101000 2024-02-19T16:28:06.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.083+0000 7f11d7fff640 1 -- 172.21.15.8:0/1400083633 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e40044b0 con 0x7f11f0101000 2024-02-19T16:28:06.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.083+0000 7f11d7fff640 1 -- 172.21.15.8:0/1400083633 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4003d30 con 0x7f11f0101000 2024-02-19T16:28:06.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.083+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f01a6970 con 0x7f11f0101000 2024-02-19T16:28:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11d7fff640 1 -- 172.21.15.8:0/1400083633 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f11e4004df0 con 0x7f11f0101000 2024-02-19T16:28:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 Processor -- start 2024-02-19T16:28:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 -- start start 2024-02-19T16:28:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a5153690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a515c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a515eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a50c2280 con 0x5605a515c7b0 2024-02-19T16:28:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a50bf7b0 con 0x5605a51532b0 2024-02-19T16:28:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a50baf90 con 0x5605a5153bd0 2024-02-19T16:28:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11ee7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a515eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a5153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11eeffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a515c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a5153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:43608/0 (socket says 172.21.15.8:43608) 2024-02-19T16:28:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11ee7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a515eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38814/0 (socket says 172.21.15.8:38814) 2024-02-19T16:28:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11ef7fe640 1 -- 172.21.15.8:0/3837731185 learned_addr learned my addr 172.21.15.8:0/3837731185 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.086+0000 7f11d7fff640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11d80a2230 0x7f11d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11d80a2230 0x7f11d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11eeffd640 1 -- 172.21.15.8:0/3837731185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 msgr2=0x5605a5153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11eeffd640 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a5153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11eeffd640 1 -- 172.21.15.8:0/3837731185 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 msgr2=0x5605a515eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11eeffd640 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a515eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11eeffd640 1 -- 172.21.15.8:0/3837731185 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a5079570 con 0x5605a5153bd0 2024-02-19T16:28:06.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a515eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11d7fff640 1 -- 172.21.15.8:0/1400083633 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f11e40f0400 con 0x7f11f0101000 2024-02-19T16:28:06.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11eeffd640 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a515c270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009c20 tx=0x7f11e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2c6d022c9fd8c39 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11d80a2230 0x7f11d80a46f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009fd0 tx=0x7f11e8004ab0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11d57fa640 1 -- 172.21.15.8:0/3837731185 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0004050 con 0x5605a5153bd0 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11d57fa640 1 -- 172.21.15.8:0/3837731185 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e0005500 con 0x5605a5153bd0 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.087+0000 7f11d57fa640 1 -- 172.21.15.8:0/3837731185 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0004c30 con 0x5605a5153bd0 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- 172.21.15.8:0/3837731185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 msgr2=0x5605a515c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a515c270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009c20 tx=0x7f11e0011810 comp rx=0 tx=0).stop 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- 172.21.15.8:0/3837731185 shutdown_connections 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a515eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a515c270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/3837731185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a5153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- 172.21.15.8:0/3837731185 >> 172.21.15.8:0/3837731185 conn(0x5605a514d990 msgr2=0x5605a514dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- 172.21.15.8:0/3837731185 shutdown_connections 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- 172.21.15.8:0/3837731185 wait complete. 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- start start 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 Processor -- start 2024-02-19T16:28:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a515c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a5159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a4d8e430 con 0x5605a515c7b0 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a4d4ae00 con 0x5605a51532b0 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.088+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a503ffd0 con 0x5605a5153bd0 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11f7bfd740 10 client.0 _ll_register_callbacks cb 0x5605a5095fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ee7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a5159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38822/0 (socket says 172.21.15.8:38822) 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ee7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a5159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:46604/0 (socket says 172.21.15.8:46604) 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 -- 172.21.15.8:0/4212025378 learned_addr learned my addr 172.21.15.8:0/4212025378 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11eeffd640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a515c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 -- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 msgr2=0x5605a5159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a5159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 -- 172.21.15.8:0/4212025378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 msgr2=0x5605a515c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a515c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 -- 172.21.15.8:0/4212025378 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e0009850 con 0x5605a515c7b0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a5159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5159b60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11e8004200 tx=0x7f11e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d128d10c5b239b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11d2ff5640 1 -- 172.21.15.8:0/4212025378 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8004d20 con 0x5605a515c7b0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11d2ff5640 1 -- 172.21.15.8:0/4212025378 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f11e8010940 con 0x5605a515c7b0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605a4dba7e0 con 0x5605a515c7b0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11d2ff5640 1 -- 172.21.15.8:0/4212025378 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e8004e80 con 0x5605a515c7b0 2024-02-19T16:28:06.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11f7bfd740 10 client.16758 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.089+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5605a50b9610 con 0x5605a515c7b0 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11d2ff5640 1 -- 172.21.15.8:0/4212025378 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5605a4dba7e0 con 0x5605a515c7b0 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11d2ff5640 1 -- 172.21.15.8:0/4212025378 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11e8022480 con 0x5605a515c7b0 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11f7bfd740 10 client.16758 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11f7bfd740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11f7bfd740 20 client.16758 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a516c960 0x5605a516ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11f7bfd740 4 client.16758 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:06.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605a4df7720 con 0x5605a516c960 2024-02-19T16:28:06.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11eeffd640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a516c960 0x5605a516ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11d2ff5640 10 client.16758 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.090+0000 7f11eeffd640 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a516c960 0x5605a516ed80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4f4ce9034b4b4b server_cookie=a1c5a062e4357719 in_seq=0 out_seq=0 2024-02-19T16:28:06.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.091+0000 7f11d2ff5640 1 -- 172.21.15.8:0/4212025378 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5605a4df7720 con 0x5605a516c960 2024-02-19T16:28:06.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.091+0000 7f11d2ff5640 10 client.16758 handle_command_reply: tid=0 2024-02-19T16:28:06.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 client.16758 shutdown 2024-02-19T16:28:06.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 msgr2=0x5605a5159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5159b60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11e8004200 tx=0x7f11e8004a40 comp rx=0 tx=0).stop 2024-02-19T16:28:06.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 shutdown_connections 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5159b60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a516c960 0x5605a516ed80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a5153bd0 0x5605a5159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11d2ff5640 0 client.16758 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/4212025378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a51532b0 0x5605a515c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 >> 172.21.15.8:0/4212025378 conn(0x5605a514d990 msgr2=0x5605a514f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 shutdown_connections 2024-02-19T16:28:06.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 1 -- 172.21.15.8:0/4212025378 wait complete. 2024-02-19T16:28:06.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.092+0000 7f11f7bfd740 20 client.16758 trim_cache size 0 max 16384 2024-02-19T16:28:06.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 Processor -- start 2024-02-19T16:28:06.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 -- start start 2024-02-19T16:28:06.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a51532b0 0x5605a5159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a515b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5167b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a515a9f0 con 0x5605a515c7b0 2024-02-19T16:28:06.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a514e590 con 0x5605a5153bd0 2024-02-19T16:28:06.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a514ea10 con 0x5605a51532b0 2024-02-19T16:28:06.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5167b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38832/0 (socket says 172.21.15.8:38832) 2024-02-19T16:28:06.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11ef7fe640 1 -- 172.21.15.8:0/1177055907 learned_addr learned my addr 172.21.15.8:0/1177055907 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11eeffd640 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a51532b0 0x5605a5159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a515b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.108+0000 7f11ef7fe640 1 -- 172.21.15.8:0/1177055907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a51532b0 msgr2=0x5605a5159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a51532b0 0x5605a5159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11ef7fe640 1 -- 172.21.15.8:0/1177055907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 msgr2=0x5605a515b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a515b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11ef7fe640 1 -- 172.21.15.8:0/1177055907 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a5055ac0 con 0x5605a515c7b0 2024-02-19T16:28:06.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5167b30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f11e8013f80 tx=0x7f11e8003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5eee79a57bcc49 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11d4ff9640 1 -- 172.21.15.8:0/1177055907 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8039070 con 0x5605a515c7b0 2024-02-19T16:28:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11d4ff9640 1 -- 172.21.15.8:0/1177055907 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e8010430 con 0x5605a515c7b0 2024-02-19T16:28:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11d4ff9640 1 -- 172.21.15.8:0/1177055907 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8035430 con 0x5605a515c7b0 2024-02-19T16:28:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 -- 172.21.15.8:0/1177055907 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 msgr2=0x5605a5167b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5167b30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f11e8013f80 tx=0x7f11e8003990 comp rx=0 tx=0).stop 2024-02-19T16:28:06.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 -- 172.21.15.8:0/1177055907 shutdown_connections 2024-02-19T16:28:06.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a515c7b0 0x5605a5167b30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a515b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/1177055907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a51532b0 0x5605a5159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 -- 172.21.15.8:0/1177055907 >> 172.21.15.8:0/1177055907 conn(0x5605a514d990 msgr2=0x5605a515ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 -- 172.21.15.8:0/1177055907 shutdown_connections 2024-02-19T16:28:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.109+0000 7f11f7bfd740 1 -- 172.21.15.8:0/1177055907 wait complete. 2024-02-19T16:28:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 -- start start 2024-02-19T16:28:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 Processor -- start 2024-02-19T16:28:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 0x5605a5164080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a51645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a515c7b0 0x5605a515da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a515e160 con 0x5605a51532b0 2024-02-19T16:28:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a515e2d0 con 0x5605a5153bd0 2024-02-19T16:28:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a515e490 con 0x5605a515c7b0 2024-02-19T16:28:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11f7bfd740 10 client.0 _ll_register_callbacks cb 0x7f11e8013e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 0x5605a5164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 0x5605a5164080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38846/0 (socket says 172.21.15.8:38846) 2024-02-19T16:28:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 -- 172.21.15.8:0/2206395317 learned_addr learned my addr 172.21.15.8:0/2206395317 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11ef7fe640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a515c7b0 0x5605a515da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a51645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 -- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a515c7b0 msgr2=0x5605a515da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a515c7b0 0x5605a515da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 -- 172.21.15.8:0/2206395317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 msgr2=0x5605a51645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a51645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.110+0000 7f11eeffd640 1 -- 172.21.15.8:0/2206395317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605a50b9610 con 0x5605a51532b0 2024-02-19T16:28:06.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11eeffd640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 0x5605a5164080 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009c20 tx=0x7f11e0005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8852f749dcbb359 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11d2ff5640 1 -- 172.21.15.8:0/2206395317 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e002a070 con 0x5605a51532b0 2024-02-19T16:28:06.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11d2ff5640 1 -- 172.21.15.8:0/2206395317 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f11e00044e0 con 0x5605a51532b0 2024-02-19T16:28:06.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605a4e46890 con 0x5605a51532b0 2024-02-19T16:28:06.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11d2ff5640 1 -- 172.21.15.8:0/2206395317 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e0009d10 con 0x5605a51532b0 2024-02-19T16:28:06.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 10 client.16764 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5605a507b6a0 con 0x5605a51532b0 2024-02-19T16:28:06.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11d2ff5640 1 -- 172.21.15.8:0/2206395317 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5605a4e46890 con 0x5605a51532b0 2024-02-19T16:28:06.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11d2ff5640 1 -- 172.21.15.8:0/2206395317 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11e0021040 con 0x5605a51532b0 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 10 client.16764 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 20 client.16764 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a51c95f0 0x5605a51cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 4 client.16764 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.111+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605a4da94e0 con 0x5605a51c95f0 2024-02-19T16:28:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a51c95f0 0x5605a51cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11ee7fc640 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a51c95f0 0x5605a51cba10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1971d2ad19b5c8c2 server_cookie=bbcbb02a040937f6 in_seq=0 out_seq=0 2024-02-19T16:28:06.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11d2ff5640 10 client.16764 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11d2ff5640 1 -- 172.21.15.8:0/2206395317 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5605a4da94e0 con 0x5605a51c95f0 2024-02-19T16:28:06.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11d2ff5640 10 client.16764 handle_command_reply: tid=0 2024-02-19T16:28:06.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 client.16764 shutdown 2024-02-19T16:28:06.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 msgr2=0x5605a5164080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 0x5605a5164080 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009c20 tx=0x7f11e0005700 comp rx=0 tx=0).stop 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 shutdown_connections 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5605a51c95f0 0x5605a51cba10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605a515c7b0 0x5605a515da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11d2ff5640 0 client.16764 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605a5153bd0 0x5605a51645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 --2- 172.21.15.8:0/2206395317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5605a51532b0 0x5605a5164080 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 >> 172.21.15.8:0/2206395317 conn(0x5605a514d990 msgr2=0x5605a514ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 shutdown_connections 2024-02-19T16:28:06.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.112+0000 7f11f7bfd740 1 -- 172.21.15.8:0/2206395317 wait complete. 2024-02-19T16:28:06.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f7bfd740 20 client.16764 trim_cache size 0 max 16384 2024-02-19T16:28:06.136 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:28:06.137 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:28:06.137 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:28:06.137 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:28:06.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11d80a2230 msgr2=0x7f11d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11d80a2230 0x7f11d80a46f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009fd0 tx=0x7f11e8004ab0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 msgr2=0x7f11f010e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 0x7f11f010e110 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f11e4005940 tx=0x7f11e402c670 comp rx=0 tx=0).stop 2024-02-19T16:28:06.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 shutdown_connections 2024-02-19T16:28:06.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f0106480 0x7f11f010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11f0101000 0x7f11f010e110 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11d80a2230 0x7f11d80a46f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 --2- 172.21.15.8:0/1400083633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11f0100640 0x7f11f010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 >> 172.21.15.8:0/1400083633 conn(0x7f11f0077190 msgr2=0x7f11f00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 shutdown_connections 2024-02-19T16:28:06.140 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.113+0000 7f11f5381640 1 -- 172.21.15.8:0/1400083633 wait complete. 2024-02-19T16:28:06.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:28:06.140 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:28:06.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.242+0000 7f66e029d640 1 -- 172.21.15.8:0/4209509420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d8103010 msgr2=0x7f66d810b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.242+0000 7f66e029d640 1 --2- 172.21.15.8:0/4209509420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d8103010 0x7f66d810b4f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009860 tx=0x7f66cc02bc00 comp rx=0 tx=0).stop 2024-02-19T16:28:06.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- 172.21.15.8:0/4209509420 shutdown_connections 2024-02-19T16:28:06.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 --2- 172.21.15.8:0/4209509420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66d810bac0 0x7f66d810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 --2- 172.21.15.8:0/4209509420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d8103010 0x7f66d810b4f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 --2- 172.21.15.8:0/4209509420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d81026f0 0x7f66d8102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- 172.21.15.8:0/4209509420 >> 172.21.15.8:0/4209509420 conn(0x7f66d80771a0 msgr2=0x7f66d80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- 172.21.15.8:0/4209509420 shutdown_connections 2024-02-19T16:28:06.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- 172.21.15.8:0/4209509420 wait complete. 2024-02-19T16:28:06.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 Processor -- start 2024-02-19T16:28:06.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- start start 2024-02-19T16:28:06.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 0x7f66d819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66d8103010 0x7f66d819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d810bac0 0x7f66d81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8198020 con 0x7f66d81026f0 2024-02-19T16:28:06.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8198190 con 0x7f66d8103010 2024-02-19T16:28:06.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66e029d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8198350 con 0x7f66d810bac0 2024-02-19T16:28:06.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66de039640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 0x7f66d819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66de039640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 0x7f66d819d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38872/0 (socket says 172.21.15.8:38872) 2024-02-19T16:28:06.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66de039640 1 -- 172.21.15.8:0/580827556 learned_addr learned my addr 172.21.15.8:0/580827556 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.243+0000 7f66dd838640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66d8103010 0x7f66d819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de83a640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d810bac0 0x7f66d81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de039640 1 -- 172.21.15.8:0/580827556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d810bac0 msgr2=0x7f66d81979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de039640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d810bac0 0x7f66d81979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de039640 1 -- 172.21.15.8:0/580827556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66d8103010 msgr2=0x7f66d819dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de039640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66d8103010 0x7f66d819dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de039640 1 -- 172.21.15.8:0/580827556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8009490 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66de039640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 0x7f66d819d820 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c420 tx=0x7f66d000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004b20 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66d0003e50 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc009490 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000b570 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.244+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d81a46e0 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.245+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d8067a50 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.246+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f66d000a040 con 0x7f66d81026f0 2024-02-19T16:28:06.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.247+0000 7f66d77fe640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f66c00a2230 0x7f66c00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.247+0000 7f66dd838640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f66c00a2230 0x7f66c00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.247+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f66d00ce600 con 0x7f66d81026f0 2024-02-19T16:28:06.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.248+0000 7f66dd838640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f66c00a2230 0x7f66c00a46f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f66cc02c210 tx=0x7f66cc0058d0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.252+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f66d0096e80 con 0x7f66d81026f0 2024-02-19T16:28:06.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:05 smithi008 ceph-mon[32573]: pgmap v159: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:06.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:05 smithi191 ceph-mon[37203]: pgmap v159: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:06.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:05 smithi156 ceph-mon[36502]: pgmap v159: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:06.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.478+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66d8106bb0 con 0x7f66d81026f0 2024-02-19T16:28:06.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.479+0000 7f66d77fe640 1 -- 172.21.15.8:0/580827556 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f66d001a090 con 0x7f66d81026f0 2024-02-19T16:28:06.481 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:28:06.481 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:28:06.481 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:28:06.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f66c00a2230 msgr2=0x7f66c00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f66c00a2230 0x7f66c00a46f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f66cc02c210 tx=0x7f66cc0058d0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 msgr2=0x7f66d819d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 0x7f66d819d820 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c420 tx=0x7f66d000c820 comp rx=0 tx=0).stop 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 shutdown_connections 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f66c00a2230 0x7f66c00a46f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d810bac0 0x7f66d81979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66d8103010 0x7f66d819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 --2- 172.21.15.8:0/580827556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66d81026f0 0x7f66d819d820 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 >> 172.21.15.8:0/580827556 conn(0x7f66d80771a0 msgr2=0x7f66d80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 shutdown_connections 2024-02-19T16:28:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.481+0000 7f66e029d640 1 -- 172.21.15.8:0/580827556 wait complete. 2024-02-19T16:28:06.493 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:28:06.493 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:28:06.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 -- 172.21.15.8:0/4188032259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a601096a0 msgr2=0x7f4a60109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 --2- 172.21.15.8:0/4188032259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a601096a0 0x7f4a60109a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009930 tx=0x7f4a5002bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:06.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 -- 172.21.15.8:0/4188032259 shutdown_connections 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 --2- 172.21.15.8:0/4188032259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a60104050 0x7f4a60110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 --2- 172.21.15.8:0/4188032259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a60103690 0x7f4a60103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 --2- 172.21.15.8:0/4188032259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a601096a0 0x7f4a60109a80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 -- 172.21.15.8:0/4188032259 >> 172.21.15.8:0/4188032259 conn(0x7f4a600ff1a0 msgr2=0x7f4a601015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 -- 172.21.15.8:0/4188032259 shutdown_connections 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 -- 172.21.15.8:0/4188032259 wait complete. 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.597+0000 7f4a655e2640 1 Processor -- start 2024-02-19T16:28:06.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 -- start start 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 0x7f4a60199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a60104050 0x7f4a6019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a601096a0 0x7f4a6019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6019ec50 con 0x7f4a60103690 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6019edc0 con 0x7f4a601096a0 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a655e2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6019ef80 con 0x7f4a60104050 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 0x7f4a60199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 0x7f4a60199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38894/0 (socket says 172.21.15.8:38894) 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 -- 172.21.15.8:0/816141010 learned_addr learned my addr 172.21.15.8:0/816141010 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a567fc640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a60104050 0x7f4a6019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a601096a0 0x7f4a6019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 -- 172.21.15.8:0/816141010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a60104050 msgr2=0x7f4a6019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a60104050 0x7f4a6019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 -- 172.21.15.8:0/816141010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a601096a0 msgr2=0x7f4a6019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a601096a0 0x7f4a6019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 -- 172.21.15.8:0/816141010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50009560 con 0x7f4a60103690 2024-02-19T16:28:06.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a601096a0 0x7f4a6019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:28:06.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5effd640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 0x7f4a60199d90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005940 tx=0x7f4a500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.598+0000 7f4a5cff9640 1 -- 172.21.15.8:0/816141010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500044e0 con 0x7f4a60103690 2024-02-19T16:28:06.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.599+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6019f200 con 0x7f4a60103690 2024-02-19T16:28:06.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.599+0000 7f4a5cff9640 1 -- 172.21.15.8:0/816141010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a5002cbc0 con 0x7f4a60103690 2024-02-19T16:28:06.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.599+0000 7f4a5cff9640 1 -- 172.21.15.8:0/816141010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50003e20 con 0x7f4a60103690 2024-02-19T16:28:06.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.599+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601a6ae0 con 0x7f4a60103690 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a5cff9640 1 -- 172.21.15.8:0/816141010 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4a500035f0 con 0x7f4a60103690 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 Processor -- start 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 -- start start 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919413630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x56191941c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191941eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619193ce960 con 0x561919413250 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561919382310 con 0x561919413b70 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a681bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191937f840 con 0x56191941c750 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.601+0000 7f4a5cff9640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4a4c0a2230 0x7f4a4c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919413630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919413630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38906/0 (socket says 172.21.15.8:38906) 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5f7fe640 1 -- 172.21.15.8:0/2573302809 learned_addr learned my addr 172.21.15.8:0/2573302809 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191941eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4a4c0a2230 0x7f4a4c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5cff9640 1 -- 172.21.15.8:0/816141010 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4a500f06f0 con 0x7f4a60103690 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5effd640 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x56191941c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 -- 172.21.15.8:0/2573302809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 msgr2=0x56191941c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x56191941c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 -- 172.21.15.8:0/2573302809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 msgr2=0x561919413630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919413630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 -- 172.21.15.8:0/2573302809 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56191933d060 con 0x56191941c750 2024-02-19T16:28:06.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919413630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191941eb40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4a480058c0 tx=0x7f4a4800cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c537c5488c2e549 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4a4c0a2230 0x7f4a4c0a46f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a58005fd0 tx=0x7f4a58005c80 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a557fa640 1 -- 172.21.15.8:0/2573302809 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800eec0 con 0x56191941c750 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a557fa640 1 -- 172.21.15.8:0/2573302809 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a480035f0 con 0x56191941c750 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.602+0000 7f4a557fa640 1 -- 172.21.15.8:0/2573302809 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48010650 con 0x56191941c750 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 -- 172.21.15.8:0/2573302809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 msgr2=0x56191941eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191941eb40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4a480058c0 tx=0x7f4a4800cee0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 -- 172.21.15.8:0/2573302809 shutdown_connections 2024-02-19T16:28:06.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191941eb40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x56191941c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2573302809 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919413630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 -- 172.21.15.8:0/2573302809 >> 172.21.15.8:0/2573302809 conn(0x56191940d980 msgr2=0x56191940dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 -- 172.21.15.8:0/2573302809 shutdown_connections 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 -- 172.21.15.8:0/2573302809 wait complete. 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 -- start start 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.603+0000 7f4a681bf740 1 Processor -- start 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919413250 0x5619194196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 0x561919419c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56191941c750 0x56191941a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191937b020 con 0x561919413b70 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191904e430 con 0x56191941c750 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191900ae00 con 0x561919413250 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a681bf740 10 client.0 _ll_register_callbacks cb 0x561919322cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 0x561919419c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919413250 0x5619194196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 0x561919419c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38922/0 (socket says 172.21.15.8:38922) 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a567fc640 1 -- 172.21.15.8:0/2169267509 learned_addr learned my addr 172.21.15.8:0/2169267509 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56191941c750 0x56191941a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 -- 172.21.15.8:0/2169267509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56191941c750 msgr2=0x56191941a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56191941c750 0x56191941a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 -- 172.21.15.8:0/2169267509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 msgr2=0x561919419c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 0x561919419c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 -- 172.21.15.8:0/2169267509 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4800c770 con 0x561919413250 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 0x561919419c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a5effd640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919413250 0x5619194196d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4a500edba0 tx=0x7f4a500bc820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ddc0457d7d2fbc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a467fc640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.604+0000 7f4a467fc640 1 -- 172.21.15.8:0/2169267509 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500bce40 con 0x561919413250 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56191907a7e0 con 0x561919413250 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a467fc640 1 -- 172.21.15.8:0/2169267509 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4a500c0f90 con 0x561919413250 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a467fc640 1 -- 172.21.15.8:0/2169267509 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a500c55e0 con 0x561919413250 2024-02-19T16:28:06.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 10 client.26647 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619193796a0 con 0x561919413250 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a467fc640 1 -- 172.21.15.8:0/2169267509 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56191907a7e0 con 0x561919413250 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a467fc640 1 -- 172.21.15.8:0/2169267509 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a500d7020 con 0x561919413250 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 10 client.26647 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 10 client.26647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 20 client.26647 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56191942c8e0 0x56191942ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 4 client.26647 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619190b7720 con 0x56191942c8e0 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.605+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56191942c8e0 0x56191942ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a467fc640 10 client.26647 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a567fc640 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56191942c8e0 0x56191942ecf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7edd4a38e8f469da server_cookie=bd8b339105a2ea87 in_seq=0 out_seq=0 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a467fc640 1 -- 172.21.15.8:0/2169267509 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5619190b7720 con 0x56191942c8e0 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a467fc640 10 client.26647 handle_command_reply: tid=0 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 client.26647 shutdown 2024-02-19T16:28:06.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919413250 msgr2=0x5619194196d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919413250 0x5619194196d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4a500edba0 tx=0x7f4a500bc820 comp rx=0 tx=0).stop 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 shutdown_connections 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56191942c8e0 0x56191942ecf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56191941c750 0x56191941a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413b70 0x561919419c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a467fc640 0 client.26647 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 --2- 172.21.15.8:0/2169267509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919413250 0x5619194196d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 >> 172.21.15.8:0/2169267509 conn(0x56191940d980 msgr2=0x56191940f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 shutdown_connections 2024-02-19T16:28:06.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.606+0000 7f4a681bf740 1 -- 172.21.15.8:0/2169267509 wait complete. 2024-02-19T16:28:06.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.607+0000 7f4a681bf740 20 client.26647 trim_cache size 0 max 16384 2024-02-19T16:28:06.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 Processor -- start 2024-02-19T16:28:06.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 -- start start 2024-02-19T16:28:06.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x56191940f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x561919419f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191940ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561919300130 con 0x561919413250 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619194295a0 con 0x561919413b70 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a681bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619194129f0 con 0x56191941c750 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x56191940f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x56191940f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38932/0 (socket says 172.21.15.8:38932) 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 -- 172.21.15.8:0/1569987303 learned_addr learned my addr 172.21.15.8:0/1569987303 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a567fc640 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x561919419f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191940ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 -- 172.21.15.8:0/1569987303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 msgr2=0x56191940ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191940ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 -- 172.21.15.8:0/1569987303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 msgr2=0x561919419f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x561919419f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.619+0000 7f4a5effd640 1 -- 172.21.15.8:0/1569987303 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561919315c20 con 0x561919413250 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a5effd640 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x56191940f640 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4a500ee090 tx=0x7f4a50003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5325367013d8e02 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a54ff9640 1 -- 172.21.15.8:0/1569987303 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500bc980 con 0x561919413250 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a54ff9640 1 -- 172.21.15.8:0/1569987303 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a500d0e90 con 0x561919413250 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a54ff9640 1 -- 172.21.15.8:0/1569987303 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500c55e0 con 0x561919413250 2024-02-19T16:28:06.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a681bf740 1 -- 172.21.15.8:0/1569987303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 msgr2=0x56191940f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a681bf740 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x56191940f640 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4a500ee090 tx=0x7f4a50003920 comp rx=0 tx=0).stop 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a681bf740 1 -- 172.21.15.8:0/1569987303 shutdown_connections 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.620+0000 7f4a681bf740 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x56191940ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x561919419f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 --2- 172.21.15.8:0/1569987303 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x56191940f640 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- 172.21.15.8:0/1569987303 >> 172.21.15.8:0/1569987303 conn(0x56191940d980 msgr2=0x56191940c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- 172.21.15.8:0/1569987303 shutdown_connections 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- 172.21.15.8:0/1569987303 wait complete. 2024-02-19T16:28:06.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- start start 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 Processor -- start 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919424060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x5619194166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x561919416c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561919424960 con 0x561919413250 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561919417170 con 0x561919413b70 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561919417330 con 0x56191941c750 2024-02-19T16:28:06.632 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a681bf740 10 client.0 _ll_register_callbacks cb 0x7f4a50009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919424060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919424060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38940/0 (socket says 172.21.15.8:38940) 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a5effd640 1 -- 172.21.15.8:0/889521262 learned_addr learned my addr 172.21.15.8:0/889521262 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a567fc640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x5619194166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a5f7fe640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x561919416c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.621+0000 7f4a5effd640 1 -- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 msgr2=0x561919416c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a5effd640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x561919416c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a5effd640 1 -- 172.21.15.8:0/889521262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 msgr2=0x5619194166f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a5effd640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x5619194166f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a5effd640 1 -- 172.21.15.8:0/889521262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619193796a0 con 0x561919413250 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a567fc640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x5619194166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a5effd640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919424060 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4a500db440 tx=0x7f4a500b90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15da1a4aef5bf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a467fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a467fc640 1 -- 172.21.15.8:0/889521262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500bc980 con 0x561919413250 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a467fc640 1 -- 172.21.15.8:0/889521262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4a500bd540 con 0x561919413250 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a480035f0 con 0x561919413250 2024-02-19T16:28:06.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a467fc640 1 -- 172.21.15.8:0/889521262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a500c55e0 con 0x561919413250 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 10 client.16788 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56191933b8c0 con 0x561919413250 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a467fc640 1 -- 172.21.15.8:0/889521262 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a480035f0 con 0x561919413250 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a467fc640 1 -- 172.21.15.8:0/889521262 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a5003c070 con 0x561919413250 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 10 client.16788 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 10 client.16788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 20 client.16788 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x561919488fc0 0x56191948b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 4 client.16788 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.622+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561919106890 con 0x561919488fc0 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a567fc640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x561919488fc0 0x56191948b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a567fc640 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x561919488fc0 0x56191948b3b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a944dceaaaf6961a server_cookie=6238f5df8a5982 in_seq=0 out_seq=0 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a467fc640 10 client.16788 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a467fc640 1 -- 172.21.15.8:0/889521262 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561919106890 con 0x561919488fc0 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a467fc640 10 client.16788 handle_command_reply: tid=0 2024-02-19T16:28:06.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 client.16788 shutdown 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 msgr2=0x561919424060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919424060 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4a500db440 tx=0x7f4a500b90e0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 shutdown_connections 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x561919488fc0 0x56191948b3b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56191941c750 0x561919416c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a467fc640 0 client.16788 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561919413b70 0x5619194166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 --2- 172.21.15.8:0/889521262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561919413250 0x561919424060 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 >> 172.21.15.8:0/889521262 conn(0x56191940d980 msgr2=0x56191940cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 shutdown_connections 2024-02-19T16:28:06.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 1 -- 172.21.15.8:0/889521262 wait complete. 2024-02-19T16:28:06.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.623+0000 7f4a681bf740 20 client.16788 trim_cache size 0 max 16384 2024-02-19T16:28:06.638 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:28:06.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4a4c0a2230 msgr2=0x7f4a4c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4a4c0a2230 0x7f4a4c0a46f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a58005fd0 tx=0x7f4a58005c80 comp rx=0 tx=0).stop 2024-02-19T16:28:06.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 msgr2=0x7f4a60199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 0x7f4a60199d90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005940 tx=0x7f4a500042f0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 shutdown_connections 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a601096a0 0x7f4a6019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4a4c0a2230 0x7f4a4c0a46f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a60104050 0x7f4a6019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 --2- 172.21.15.8:0/816141010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a60103690 0x7f4a60199d90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 >> 172.21.15.8:0/816141010 conn(0x7f4a600ff1a0 msgr2=0x7f4a600ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 shutdown_connections 2024-02-19T16:28:06.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.624+0000 7f4a655e2640 1 -- 172.21.15.8:0/816141010 wait complete. 2024-02-19T16:28:06.642 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:28:06.642 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:28:06.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.744+0000 7f1375247640 1 -- 172.21.15.8:0/3251191226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f137010bab0 msgr2=0x7f137010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.744+0000 7f1375247640 1 --2- 172.21.15.8:0/3251191226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f137010bab0 0x7f137010df20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1368009ca0 tx=0x7f136802c180 comp rx=0 tx=0).stop 2024-02-19T16:28:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 -- 172.21.15.8:0/3251191226 shutdown_connections 2024-02-19T16:28:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 --2- 172.21.15.8:0/3251191226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f137010bab0 0x7f137010df20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 --2- 172.21.15.8:0/3251191226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1370103090 0x7f137010b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 --2- 172.21.15.8:0/3251191226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 0x7f1370102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 -- 172.21.15.8:0/3251191226 >> 172.21.15.8:0/3251191226 conn(0x7f1370077550 msgr2=0x7f1370077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 -- 172.21.15.8:0/3251191226 shutdown_connections 2024-02-19T16:28:06.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 -- 172.21.15.8:0/3251191226 wait complete. 2024-02-19T16:28:06.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.745+0000 7f1375247640 1 Processor -- start 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 -- start start 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 0x7f137019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 0x7f137019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f137010bab0 0x7f13701978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1370197ef0 con 0x7f1370103090 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1370198060 con 0x7f137010bab0 2024-02-19T16:28:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1375247640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1370198220 con 0x7f1370102770 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 0x7f137019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 0x7f137019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38964/0 (socket says 172.21.15.8:38964) 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 -- 172.21.15.8:0/3377087959 learned_addr learned my addr 172.21.15.8:0/3377087959 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136ed9d640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 0x7f137019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136f59e640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f137010bab0 0x7f13701978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 -- 172.21.15.8:0/3377087959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 msgr2=0x7f137019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 0x7f137019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 -- 172.21.15.8:0/3377087959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f137010bab0 msgr2=0x7f13701978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f137010bab0 0x7f13701978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 -- 172.21.15.8:0/3377087959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1368009900 con 0x7f1370103090 2024-02-19T16:28:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136ed9d640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 0x7f137019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f136e59c640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 0x7f137019dc30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f136400e9f0 tx=0x7f136400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.746+0000 7f1357fff640 1 -- 172.21.15.8:0/3377087959 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364009700 con 0x7f1370103090 2024-02-19T16:28:06.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.747+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1370198500 con 0x7f1370103090 2024-02-19T16:28:06.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.747+0000 7f1357fff640 1 -- 172.21.15.8:0/3377087959 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f136400cd80 con 0x7f1370103090 2024-02-19T16:28:06.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.747+0000 7f1357fff640 1 -- 172.21.15.8:0/3377087959 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364018920 con 0x7f1370103090 2024-02-19T16:28:06.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.747+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13701a4790 con 0x7f1370103090 2024-02-19T16:28:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1357fff640 1 -- 172.21.15.8:0/3377087959 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f1364018a80 con 0x7f1370103090 2024-02-19T16:28:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 Processor -- start 2024-02-19T16:28:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 -- start start 2024-02-19T16:28:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1357fff640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f13580a2230 0x7f13580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79250 0x5601f4d79630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4d34960 con 0x5601f4d79b70 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4ce8310 con 0x5601f4d82750 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1377e1d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4ce5840 con 0x5601f4d79250 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f1357fff640 1 -- 172.21.15.8:0/3377087959 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f1364014070 con 0x7f1370103090 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f136f59e640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f13580a2230 0x7f13580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f136e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f136e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d82210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38976/0 (socket says 172.21.15.8:38976) 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f136e59c640 1 -- 172.21.15.8:0/905444986 learned_addr learned my addr 172.21.15.8:0/905444986 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f136ed9d640 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79250 0x5601f4d79630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.749+0000 7f136f59e640 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136e59c640 1 -- 172.21.15.8:0/905444986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79250 msgr2=0x5601f4d79630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136e59c640 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79250 0x5601f4d79630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136e59c640 1 -- 172.21.15.8:0/905444986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 msgr2=0x5601f4d84b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136e59c640 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d84b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136e59c640 1 -- 172.21.15.8:0/905444986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f4ca3060 con 0x5601f4d79b70 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136ed9d640 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79250 0x5601f4d79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136f59e640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f13580a2230 0x7f13580a46f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1368004af0 tx=0x7f1368005fa0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f136e59c640 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d82210 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1364097810 tx=0x7f136409aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d920e8109a4f6be server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f13557fa640 1 -- 172.21.15.8:0/905444986 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364019680 con 0x5601f4d79b70 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f13557fa640 1 -- 172.21.15.8:0/905444986 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13640217a0 con 0x5601f4d79b70 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.750+0000 7f13557fa640 1 -- 172.21.15.8:0/905444986 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364005520 con 0x5601f4d79b70 2024-02-19T16:28:06.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- 172.21.15.8:0/905444986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 msgr2=0x5601f4d82210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d82210 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1364097810 tx=0x7f136409aa40 comp rx=0 tx=0).stop 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- 172.21.15.8:0/905444986 shutdown_connections 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d84b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d82210 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- 172.21.15.8:0/905444986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79250 0x5601f4d79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- 172.21.15.8:0/905444986 >> 172.21.15.8:0/905444986 conn(0x5601f4d73980 msgr2=0x5601f4d73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- 172.21.15.8:0/905444986 shutdown_connections 2024-02-19T16:28:06.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- 172.21.15.8:0/905444986 wait complete. 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- start start 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 Processor -- start 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79250 0x5601f4d7f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d7fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d80250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4ce1020 con 0x5601f4d79b70 2024-02-19T16:28:06.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f49b4430 con 0x5601f4d79250 2024-02-19T16:28:06.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4970e00 con 0x5601f4d82750 2024-02-19T16:28:06.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f1377e1d740 10 client.0 _ll_register_callbacks cb 0x5601f4c88cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f136e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d7fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f136e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d7fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38988/0 (socket says 172.21.15.8:38988) 2024-02-19T16:28:06.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f136e59c640 1 -- 172.21.15.8:0/1057094244 learned_addr learned my addr 172.21.15.8:0/1057094244 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f136ed9d640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79250 0x5601f4d7f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.751+0000 7f136f59e640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d80250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f136e59c640 1 -- 172.21.15.8:0/1057094244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 msgr2=0x5601f4d80250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f136e59c640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d80250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f136e59c640 1 -- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79250 msgr2=0x5601f4d7f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f136e59c640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79250 0x5601f4d7f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f136e59c640 1 -- 172.21.15.8:0/1057094244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13701a4790 con 0x5601f4d79b70 2024-02-19T16:28:06.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f136e59c640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d7fd10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f136409add0 tx=0x7f1364010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fd48cd417a9a53 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1352ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1352ff5640 1 -- 172.21.15.8:0/1057094244 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13640215f0 con 0x5601f4d79b70 2024-02-19T16:28:06.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1352ff5640 1 -- 172.21.15.8:0/1057094244 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f136409ae40 con 0x5601f4d79b70 2024-02-19T16:28:06.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601f49e07e0 con 0x5601f4d79b70 2024-02-19T16:28:06.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1352ff5640 1 -- 172.21.15.8:0/1057094244 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f136409eee0 con 0x5601f4d79b70 2024-02-19T16:28:06.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1377e1d740 10 client.16800 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5601f4cdf6a0 con 0x5601f4d79b70 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1352ff5640 1 -- 172.21.15.8:0/1057094244 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13640a9020 con 0x5601f4d79b70 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1352ff5640 1 -- 172.21.15.8:0/1057094244 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13640ce070 con 0x5601f4d79b70 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1377e1d740 10 client.16800 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.752+0000 7f1377e1d740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f1377e1d740 20 client.16800 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f1377e1d740 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4d929a0 0x5601f4d94d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f1377e1d740 4 client.16800 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601f4a1d720 con 0x5601f4d929a0 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f136ed9d640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4d929a0 0x5601f4d94d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f136ed9d640 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4d929a0 0x5601f4d94d90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ead971b72828aa2 server_cookie=9b512cf36f7a03bf in_seq=0 out_seq=0 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.753+0000 7f1352ff5640 10 client.16800 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1352ff5640 1 -- 172.21.15.8:0/1057094244 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5601f4a1d720 con 0x5601f4d929a0 2024-02-19T16:28:06.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1352ff5640 10 client.16800 handle_command_reply: tid=0 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 client.16800 shutdown 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 msgr2=0x5601f4d7fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d7fd10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f136409add0 tx=0x7f1364010db0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 shutdown_connections 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4d929a0 0x5601f4d94d90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d80250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1352ff5640 0 client.16800 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:06.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79b70 0x5601f4d7fd10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 --2- 172.21.15.8:0/1057094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79250 0x5601f4d7f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 >> 172.21.15.8:0/1057094244 conn(0x5601f4d73980 msgr2=0x5601f4d756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 shutdown_connections 2024-02-19T16:28:06.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 1 -- 172.21.15.8:0/1057094244 wait complete. 2024-02-19T16:28:06.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.754+0000 7f1377e1d740 20 client.16800 trim_cache size 0 max 16384 2024-02-19T16:28:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.766+0000 7f1377e1d740 1 Processor -- start 2024-02-19T16:28:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.766+0000 7f1377e1d740 1 -- start start 2024-02-19T16:28:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79b70 0x5601f4d6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d80070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1377e1d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4c66130 con 0x5601f4d79250 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1377e1d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4d785e0 con 0x5601f4d82750 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1377e1d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4d8f640 con 0x5601f4d79b70 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d81f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38990/0 (socket says 172.21.15.8:38990) 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 -- 172.21.15.8:0/584291333 learned_addr learned my addr 172.21.15.8:0/584291333 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136e59c640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79b70 0x5601f4d6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136f59e640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d80070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 -- 172.21.15.8:0/584291333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79b70 msgr2=0x5601f4d6ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79b70 0x5601f4d6ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 -- 172.21.15.8:0/584291333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 msgr2=0x5601f4d80070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d80070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 -- 172.21.15.8:0/584291333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f4c7bc20 con 0x5601f4d79250 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136e59c640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79b70 0x5601f4d6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136f59e640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d80070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f136ed9d640 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d81f00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x5601f4d95600 tx=0x7f1360005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78be5d23d0b21c81 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1354ff9640 1 -- 172.21.15.8:0/584291333 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360017030 con 0x5601f4d79250 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1354ff9640 1 -- 172.21.15.8:0/584291333 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1360018040 con 0x5601f4d79250 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.767+0000 7f1354ff9640 1 -- 172.21.15.8:0/584291333 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360010b00 con 0x5601f4d79250 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 -- 172.21.15.8:0/584291333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 msgr2=0x5601f4d81f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d81f00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x5601f4d95600 tx=0x7f1360005f90 comp rx=0 tx=0).stop 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 -- 172.21.15.8:0/584291333 shutdown_connections 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d82750 0x5601f4d80070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d79b70 0x5601f4d6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 --2- 172.21.15.8:0/584291333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d81f00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 -- 172.21.15.8:0/584291333 >> 172.21.15.8:0/584291333 conn(0x5601f4d73980 msgr2=0x5601f4d72b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 -- 172.21.15.8:0/584291333 shutdown_connections 2024-02-19T16:28:06.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 -- 172.21.15.8:0/584291333 wait complete. 2024-02-19T16:28:06.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 -- start start 2024-02-19T16:28:06.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.768+0000 7f1377e1d740 1 Processor -- start 2024-02-19T16:28:06.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79b70 0x5601f4d86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d86b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4d87110 con 0x5601f4d79250 2024-02-19T16:28:06.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4d87280 con 0x5601f4d79b70 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f4d87440 con 0x5601f4d82750 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 10 client.0 _ll_register_callbacks cb 0x7f1360009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d8a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38992/0 (socket says 172.21.15.8:38992) 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 -- 172.21.15.8:0/2896431335 learned_addr learned my addr 172.21.15.8:0/2896431335 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136e59c640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79b70 0x5601f4d86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136f59e640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d86b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 -- 172.21.15.8:0/2896431335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 msgr2=0x5601f4d86b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d86b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 -- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79b70 msgr2=0x5601f4d86610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79b70 0x5601f4d86610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 -- 172.21.15.8:0/2896431335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1360012040 con 0x5601f4d79250 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f136ed9d640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d8a290 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1360014010 tx=0x7f1360020180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f674d22b055f974 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1352ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1352ff5640 1 -- 172.21.15.8:0/2896431335 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360017040 con 0x5601f4d79250 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1352ff5640 1 -- 172.21.15.8:0/2896431335 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f1360010c90 con 0x5601f4d79250 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601f4d6f0f0 con 0x5601f4d79250 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1352ff5640 1 -- 172.21.15.8:0/2896431335 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1360010e90 con 0x5601f4d79250 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 10 client.16806 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.769+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5601f4ca18c0 con 0x5601f4d79250 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1352ff5640 1 -- 172.21.15.8:0/2896431335 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5601f4d6f0f0 con 0x5601f4d79250 2024-02-19T16:28:06.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1352ff5640 1 -- 172.21.15.8:0/2896431335 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1360004700 con 0x5601f4d79250 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 10 client.16806 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 10 client.16806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 20 client.16806 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4def7c0 0x5601f4df1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 4 client.16806 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f13640217a0 con 0x5601f4def7c0 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f136e59c640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4def7c0 0x5601f4df1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f136e59c640 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4def7c0 0x5601f4df1bb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17d85b281b302bac server_cookie=7063da0d9573abee in_seq=0 out_seq=0 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1352ff5640 10 client.16806 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1352ff5640 1 -- 172.21.15.8:0/2896431335 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f13640217a0 con 0x5601f4def7c0 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1352ff5640 10 client.16806 handle_command_reply: tid=0 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.770+0000 7f1377e1d740 1 client.16806 shutdown 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 msgr2=0x5601f4d8a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d8a290 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1360014010 tx=0x7f1360020180 comp rx=0 tx=0).stop 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 shutdown_connections 2024-02-19T16:28:06.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5601f4def7c0 0x5601f4df1bb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f4d82750 0x5601f4d86b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5601f4d79b70 0x5601f4d86610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 --2- 172.21.15.8:0/2896431335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5601f4d79250 0x5601f4d8a290 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1352ff5640 0 client.16806 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 >> 172.21.15.8:0/2896431335 conn(0x5601f4d73980 msgr2=0x5601f4d99650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 shutdown_connections 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 1 -- 172.21.15.8:0/2896431335 wait complete. 2024-02-19T16:28:06.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1377e1d740 20 client.16806 trim_cache size 0 max 16384 2024-02-19T16:28:06.794 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:28:06.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f13580a2230 msgr2=0x7f13580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f13580a2230 0x7f13580a46f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1368004af0 tx=0x7f1368005fa0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 msgr2=0x7f137019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 0x7f137019dc30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f136400e9f0 tx=0x7f136400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 shutdown_connections 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f137010bab0 0x7f13701978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1370103090 0x7f137019dc30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f13580a2230 0x7f13580a46f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 --2- 172.21.15.8:0/3377087959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1370102770 0x7f137019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 >> 172.21.15.8:0/3377087959 conn(0x7f1370077550 msgr2=0x7f13700fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 shutdown_connections 2024-02-19T16:28:06.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.771+0000 7f1375247640 1 -- 172.21.15.8:0/3377087959 wait complete. 2024-02-19T16:28:06.797 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:28:06.797 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:28:06.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.900+0000 7f79a8615640 1 -- 172.21.15.8:0/987997745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0100ac0 msgr2=0x7f79a0100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.900+0000 7f79a8615640 1 --2- 172.21.15.8:0/987997745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0100ac0 0x7f79a0100f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7994009890 tx=0x7f799402bb70 comp rx=0 tx=0).stop 2024-02-19T16:28:06.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- 172.21.15.8:0/987997745 shutdown_connections 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 --2- 172.21.15.8:0/987997745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 --2- 172.21.15.8:0/987997745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0100ac0 0x7f79a0100f40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 --2- 172.21.15.8:0/987997745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79a0106680 0x7f79a0106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- 172.21.15.8:0/987997745 >> 172.21.15.8:0/987997745 conn(0x7f79a0076910 msgr2=0x7f79a00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- 172.21.15.8:0/987997745 shutdown_connections 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- 172.21.15.8:0/987997745 wait complete. 2024-02-19T16:28:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 Processor -- start 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- start start 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79a0100ac0 0x7f79a019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0106680 0x7f79a0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01982a0 con 0x7f79a0101480 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0198410 con 0x7f79a0106680 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a8615640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01985d0 con 0x7f79a0100ac0 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39014/0 (socket says 172.21.15.8:39014) 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a5bb0640 1 -- 172.21.15.8:0/320950072 learned_addr learned my addr 172.21.15.8:0/320950072 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.901+0000 7f79a63b1640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79a0100ac0 0x7f79a019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0106680 0x7f79a0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a5bb0640 1 -- 172.21.15.8:0/320950072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79a0100ac0 msgr2=0x7f79a019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79a0100ac0 0x7f79a019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a5bb0640 1 -- 172.21.15.8:0/320950072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0106680 msgr2=0x7f79a0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0106680 0x7f79a0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a5bb0640 1 -- 172.21.15.8:0/320950072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994009490 con 0x7f79a0101480 2024-02-19T16:28:06.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a019df90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7994009860 tx=0x7f7994004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f799f7fe640 1 -- 172.21.15.8:0/320950072 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994034950 con 0x7f79a0101480 2024-02-19T16:28:06.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a0198850 con 0x7f79a0101480 2024-02-19T16:28:06.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f799f7fe640 1 -- 172.21.15.8:0/320950072 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799402c9f0 con 0x7f79a0101480 2024-02-19T16:28:06.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f799f7fe640 1 -- 172.21.15.8:0/320950072 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994003e50 con 0x7f79a0101480 2024-02-19T16:28:06.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.902+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01a49d0 con 0x7f79a0101480 2024-02-19T16:28:06.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.904+0000 7f799f7fe640 1 -- 172.21.15.8:0/320950072 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f799403b040 con 0x7f79a0101480 2024-02-19T16:28:06.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 Processor -- start 2024-02-19T16:28:06.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 -- start start 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f799f7fe640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f79880a2230 0x7f79880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a90973630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a9097c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a9097eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9092e960 con 0x563a9097c750 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a908e2310 con 0x563a90973b70 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a908df840 con 0x563a90973250 2024-02-19T16:28:06.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f79880a2230 0x7f79880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79a63b1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a90973630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f799f7fe640 1 -- 172.21.15.8:0/320950072 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f79940efe80 con 0x7f79a0101480 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79a63b1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a90973630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:46774/0 (socket says 172.21.15.8:46774) 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a9097c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79a63b1640 1 -- 172.21.15.8:0/2354558625 learned_addr learned my addr 172.21.15.8:0/2354558625 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.905+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a9097eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2354558625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 msgr2=0x563a90973630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a90973630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2354558625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 msgr2=0x563a9097eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a9097eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2354558625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a9089d060 con 0x563a90973b70 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f79880a2230 0x7f79880a46f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7990005fd0 tx=0x7f7990009f10 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a9097c210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79940ed330 tx=0x7f7994005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7287d4069cfa355 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f799cbfe640 1 -- 172.21.15.8:0/2354558625 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940bc110 con 0x563a90973b70 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f799cbfe640 1 -- 172.21.15.8:0/2354558625 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79940b8040 con 0x563a90973b70 2024-02-19T16:28:06.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.906+0000 7f799cbfe640 1 -- 172.21.15.8:0/2354558625 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940035f0 con 0x563a90973b70 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2354558625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 msgr2=0x563a9097c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a9097c210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79940ed330 tx=0x7f7994005b20 comp rx=0 tx=0).stop 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2354558625 shutdown_connections 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a9097eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a9097c210 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2354558625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a90973630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2354558625 >> 172.21.15.8:0/2354558625 conn(0x563a9096d980 msgr2=0x563a9096dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2354558625 shutdown_connections 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2354558625 wait complete. 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 -- start start 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 Processor -- start 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.907+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90979b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a9097a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a908db020 con 0x563a90973b70 2024-02-19T16:28:06.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a905ae430 con 0x563a90973250 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9056ae00 con 0x563a9097c750 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 10 client.0 _ll_register_callbacks cb 0x563a90882cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90979b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a63b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90979b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39034/0 (socket says 172.21.15.8:39034) 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2656168968 learned_addr learned my addr 172.21.15.8:0/2656168968 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a9097a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 msgr2=0x563a9097a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a9097a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2656168968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 msgr2=0x563a909795f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909795f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 -- 172.21.15.8:0/2656168968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01a49d0 con 0x563a90973b70 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a63b1640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90979b30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f79940ed820 tx=0x7f7994005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0c6e580e104104 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f7985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f7985ffb640 1 -- 172.21.15.8:0/2656168968 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940bc110 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a905da7e0 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f7985ffb640 1 -- 172.21.15.8:0/2656168968 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f79940bc5d0 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f7985ffb640 1 -- 172.21.15.8:0/2656168968 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79940bbf30 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 10 client.16818 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.908+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563a908d96a0 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f7985ffb640 1 -- 172.21.15.8:0/2656168968 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563a905da7e0 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f7985ffb640 1 -- 172.21.15.8:0/2656168968 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79940eb410 con 0x563a90973b70 2024-02-19T16:28:06.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79ab1f4740 10 client.16818 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79ab1f4740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79ab1f4740 20 client.16818 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a9098c1d0 0x563a9098e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79ab1f4740 4 client.16818 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a90617720 con 0x563a9098c1d0 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79a63b1640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a9098c1d0 0x563a9098e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f79a63b1640 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a9098c1d0 0x563a9098e5e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156e24c58ea80939 server_cookie=c8cf7de9396bfed2 in_seq=0 out_seq=0 2024-02-19T16:28:06.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.909+0000 7f7985ffb640 10 client.16818 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f7985ffb640 1 -- 172.21.15.8:0/2656168968 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563a90617720 con 0x563a9098c1d0 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f7985ffb640 10 client.16818 handle_command_reply: tid=0 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 client.16818 shutdown 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 msgr2=0x563a90979b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90979b30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f79940ed820 tx=0x7f7994005d20 comp rx=0 tx=0).stop 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 shutdown_connections 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a9098c1d0 0x563a9098e5e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a9097a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90979b30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/2656168968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f7985ffb640 0 client.16818 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 >> 172.21.15.8:0/2656168968 conn(0x563a9096d980 msgr2=0x563a9096f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 shutdown_connections 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 1 -- 172.21.15.8:0/2656168968 wait complete. 2024-02-19T16:28:06.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.910+0000 7f79ab1f4740 20 client.16818 trim_cache size 0 max 16384 2024-02-19T16:28:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 Processor -- start 2024-02-19T16:28:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 -- start start 2024-02-19T16:28:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a9097bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a90979e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a909877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90860130 con 0x563a9097c750 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9098ec90 con 0x563a90973b70 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90972c90 con 0x563a90973250 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a909877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a909877a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39048/0 (socket says 172.21.15.8:39048) 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 -- 172.21.15.8:0/3346703712 learned_addr learned my addr 172.21.15.8:0/3346703712 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a63b1640 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a9097bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a90979e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 -- 172.21.15.8:0/3346703712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 msgr2=0x563a9097bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a9097bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 -- 172.21.15.8:0/3346703712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 msgr2=0x563a90979e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a90979e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 -- 172.21.15.8:0/3346703712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a90875c20 con 0x563a9097c750 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.923+0000 7f79a6bb2640 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a909877a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7990011a30 tx=0x7f7990021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e487bcdb85ff40 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f7987fff640 1 -- 172.21.15.8:0/3346703712 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990016040 con 0x563a9097c750 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f7987fff640 1 -- 172.21.15.8:0/3346703712 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7990009b40 con 0x563a9097c750 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f7987fff640 1 -- 172.21.15.8:0/3346703712 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79900203f0 con 0x563a9097c750 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3346703712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 msgr2=0x563a909877a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a909877a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7990011a30 tx=0x7f7990021000 comp rx=0 tx=0).stop 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3346703712 shutdown_connections 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a9097c750 0x563a909877a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973b70 0x563a90979e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3346703712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a90973250 0x563a9097bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3346703712 >> 172.21.15.8:0/3346703712 conn(0x563a9096d980 msgr2=0x563a9096cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3346703712 shutdown_connections 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3346703712 wait complete. 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.924+0000 7f79ab1f4740 1 -- start start 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 Processor -- start 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90983ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a90976400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a909769e0 con 0x563a90973b70 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90976b50 con 0x563a90973250 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90976d10 con 0x563a9097c750 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79ab1f4740 10 client.0 _ll_register_callbacks cb 0x7f799402cb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90983ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a63b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90983ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39062/0 (socket says 172.21.15.8:39062) 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a6bb2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a90976400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 -- 172.21.15.8:0/3381540283 learned_addr learned my addr 172.21.15.8:0/3381540283 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 -- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 msgr2=0x563a90976400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a90976400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 -- 172.21.15.8:0/3381540283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 msgr2=0x563a909839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909839a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 -- 172.21.15.8:0/3381540283 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7990019040 con 0x563a90973b70 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f79a5bb0640 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90983ee0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7994000c00 tx=0x7f7994005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aec570138d74834 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f7985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.925+0000 7f7985ffb640 1 -- 172.21.15.8:0/3381540283 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940eb5c0 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f7985ffb640 1 -- 172.21.15.8:0/3381540283 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f79940bccd0 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79940b8040 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f7985ffb640 1 -- 172.21.15.8:0/3381540283 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79940bc110 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 10 client.16824 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563a9089b8c0 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f7985ffb640 1 -- 172.21.15.8:0/3381540283 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79940b8040 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f7985ffb640 1 -- 172.21.15.8:0/3381540283 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79940bba20 con 0x563a90973b70 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 10 client.16824 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 20 client.16824 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a909e8680 0x563a909eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 4 client.16824 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a90666890 con 0x563a909e8680 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.926+0000 7f79a63b1640 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a909e8680 0x563a909eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f7985ffb640 10 client.16824 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a63b1640 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a909e8680 0x563a909eaa70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f14b185f2d5410b server_cookie=7d5b6cff8536092f in_seq=0 out_seq=0 2024-02-19T16:28:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f7985ffb640 1 -- 172.21.15.8:0/3381540283 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a90666890 con 0x563a909e8680 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f7985ffb640 10 client.16824 handle_command_reply: tid=0 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 client.16824 shutdown 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 msgr2=0x563a90983ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90983ee0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7994000c00 tx=0x7f7994005d90 comp rx=0 tx=0).stop 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 shutdown_connections 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563a909e8680 0x563a909eaa70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a9097c750 0x563a90976400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563a90973b70 0x563a90983ee0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f7985ffb640 0 client.16824 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 --2- 172.21.15.8:0/3381540283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a90973250 0x563a909839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 >> 172.21.15.8:0/3381540283 conn(0x563a9096d980 msgr2=0x563a9096eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 shutdown_connections 2024-02-19T16:28:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 1 -- 172.21.15.8:0/3381540283 wait complete. 2024-02-19T16:28:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79ab1f4740 20 client.16824 trim_cache size 0 max 16384 2024-02-19T16:28:06.942 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:28:06.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f79880a2230 msgr2=0x7f79880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f79880a2230 0x7f79880a46f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7990005fd0 tx=0x7f7990009f10 comp rx=0 tx=0).stop 2024-02-19T16:28:06.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 msgr2=0x7f79a019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:06.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a019df90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7994009860 tx=0x7f7994004530 comp rx=0 tx=0).stop 2024-02-19T16:28:06.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 shutdown_connections 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a0106680 0x7f79a0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79a0101480 0x7f79a019df90 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f79880a2230 0x7f79880a46f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.927+0000 7f79a8615640 1 --2- 172.21.15.8:0/320950072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79a0100ac0 0x7f79a019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.928+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 >> 172.21.15.8:0/320950072 conn(0x7f79a0076910 msgr2=0x7f79a00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.928+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 shutdown_connections 2024-02-19T16:28:06.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:06.928+0000 7f79a8615640 1 -- 172.21.15.8:0/320950072 wait complete. 2024-02-19T16:28:07.269 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:07 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/580827556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:28:07.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:07 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/580827556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:28:07.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:07 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/580827556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:28:07.945 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:28:07.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:28:07.946 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:28:08.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 -- 172.21.15.8:0/1305376295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8090100f80 msgr2=0x7f8090104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 --2- 172.21.15.8:0/1305376295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8090100f80 0x7f8090104fc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8080009930 tx=0x7f808002bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:08.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 -- 172.21.15.8:0/1305376295 shutdown_connections 2024-02-19T16:28:08.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 --2- 172.21.15.8:0/1305376295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901056a0 0x7f809010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 --2- 172.21.15.8:0/1305376295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8090100f80 0x7f8090104fc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 --2- 172.21.15.8:0/1305376295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80901005d0 0x7f80901009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 -- 172.21.15.8:0/1305376295 >> 172.21.15.8:0/1305376295 conn(0x7f80900774f0 msgr2=0x7f8090077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 -- 172.21.15.8:0/1305376295 shutdown_connections 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.064+0000 7f80981bd640 1 -- 172.21.15.8:0/1305376295 wait complete. 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 Processor -- start 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 -- start start 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 0x7f8090199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80901056a0 0x7f809019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809019e600 0x7f809019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809019eff0 con 0x7f80901005d0 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809019f160 con 0x7f80901056a0 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f80981bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809019f2d0 con 0x7f809019e600 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 0x7f8090199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 0x7f8090199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39070/0 (socket says 172.21.15.8:39070) 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 -- 172.21.15.8:0/3511466290 learned_addr learned my addr 172.21.15.8:0/3511466290 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095758640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80901056a0 0x7f809019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f809675a640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809019e600 0x7f809019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 -- 172.21.15.8:0/3511466290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809019e600 msgr2=0x7f809019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809019e600 0x7f809019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 -- 172.21.15.8:0/3511466290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80901056a0 msgr2=0x7f809019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80901056a0 0x7f809019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.065+0000 7f8095f59640 1 -- 172.21.15.8:0/3511466290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009560 con 0x7f80901005d0 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.066+0000 7f8095f59640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 0x7f8090199d40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9f0 tx=0x7f808c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.066+0000 7f8086ffd640 1 -- 172.21.15.8:0/3511466290 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c009700 con 0x7f80901005d0 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.066+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80901a68b0 con 0x7f80901005d0 2024-02-19T16:28:08.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.066+0000 7f8086ffd640 1 -- 172.21.15.8:0/3511466290 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f808c00cd80 con 0x7f80901005d0 2024-02-19T16:28:08.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.066+0000 7f8086ffd640 1 -- 172.21.15.8:0/3511466290 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c018920 con 0x7f80901005d0 2024-02-19T16:28:08.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.066+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80901a6d30 con 0x7f80901005d0 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.068+0000 7f8086ffd640 1 -- 172.21.15.8:0/3511466290 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f808c018a80 con 0x7f80901005d0 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.068+0000 7f809ad8f740 1 Processor -- start 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 -- start start 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 0x55dbd8969760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969d30 0x55dbd89722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f8086ffd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f807c0a2230 0x7f807c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd89728b0 0x55dbd8974d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd88d7f90 con 0x55dbd8969380 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd88d54c0 con 0x55dbd8969d30 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809ad8f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd88d0ca0 con 0x55dbd89728b0 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 0x55dbd8969760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f8095758640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f807c0a2230 0x7f807c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 0x55dbd8969760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39078/0 (socket says 172.21.15.8:39078) 2024-02-19T16:28:08.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 -- 172.21.15.8:0/3551783211 learned_addr learned my addr 172.21.15.8:0/3551783211 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f8095f59640 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969d30 0x55dbd89722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f8086ffd640 1 -- 172.21.15.8:0/3511466290 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f808c014070 con 0x7f80901005d0 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd89728b0 0x55dbd8974d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 -- 172.21.15.8:0/3551783211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd89728b0 msgr2=0x55dbd8974d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd89728b0 0x55dbd8974d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 -- 172.21.15.8:0/3551783211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969d30 msgr2=0x55dbd89722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969d30 0x55dbd89722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f809675a640 1 -- 172.21.15.8:0/3551783211 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd88930b0 con 0x55dbd8969380 2024-02-19T16:28:08.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.069+0000 7f8095f59640 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969d30 0x55dbd89722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f8095758640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f807c0a2230 0x7f807c0a46f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8080000c00 tx=0x7f8080005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f809675a640 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 0x55dbd8969760 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8088009fd0 tx=0x7f8088005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847db2b11ec88d9c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f807bfff640 1 -- 172.21.15.8:0/3551783211 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088013130 con 0x55dbd8969380 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f807bfff640 1 -- 172.21.15.8:0/3551783211 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8088013290 con 0x55dbd8969380 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f807bfff640 1 -- 172.21.15.8:0/3551783211 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088005310 con 0x55dbd8969380 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f809ad8f740 1 -- 172.21.15.8:0/3551783211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 msgr2=0x55dbd8969760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.070+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 0x55dbd8969760 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8088009fd0 tx=0x7f8088005bd0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 -- 172.21.15.8:0/3551783211 shutdown_connections 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd89728b0 0x55dbd8974d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969d30 0x55dbd89722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3551783211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd8969380 0x55dbd8969760 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 -- 172.21.15.8:0/3551783211 >> 172.21.15.8:0/3551783211 conn(0x55dbd8963840 msgr2=0x55dbd8963c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 -- 172.21.15.8:0/3551783211 shutdown_connections 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 -- 172.21.15.8:0/3551783211 wait complete. 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 -- start start 2024-02-19T16:28:08.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 Processor -- start 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd896f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.071+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd8875f40 con 0x55dbd89728b0 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd85a44b0 con 0x55dbd8969380 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd8856670 con 0x55dbd8969d30 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 10 client.0 _ll_register_callbacks cb 0x55dbd88c3f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:08.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f8095f59640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd896f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39090/0 (socket says 172.21.15.8:39090) 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 -- 172.21.15.8:0/3204353555 learned_addr learned my addr 172.21.15.8:0/3204353555 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f8095f59640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd896f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:43892/0 (socket says 172.21.15.8:43892) 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f8095758640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 -- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 msgr2=0x55dbd896fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 -- 172.21.15.8:0/3204353555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 msgr2=0x55dbd896f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd896f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 -- 172.21.15.8:0/3204353555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808801b040 con 0x55dbd89728b0 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f8095f59640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd896f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809675a640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970250 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8088004cf0 tx=0x7f8088004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579ee97225667313 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f80797fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f80797fa640 1 -- 172.21.15.8:0/3204353555 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088003b80 con 0x55dbd89728b0 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f80797fa640 1 -- 172.21.15.8:0/3204353555 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8088003ce0 con 0x55dbd89728b0 2024-02-19T16:28:08.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbd88aaf40 con 0x55dbd89728b0 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f80797fa640 1 -- 172.21.15.8:0/3204353555 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f808800f850 con 0x55dbd89728b0 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 10 client.16836 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.072+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dbd88cf320 con 0x55dbd89728b0 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f80797fa640 1 -- 172.21.15.8:0/3204353555 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbd88aaf40 con 0x55dbd89728b0 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f80797fa640 1 -- 172.21.15.8:0/3204353555 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8088005090 con 0x55dbd89728b0 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f809ad8f740 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f809ad8f740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:08.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f809ad8f740 20 client.16836 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd8982bf0 0x55dbd8984fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f809ad8f740 4 client.16836 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbd88c7310 con 0x55dbd8982bf0 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f8095f59640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd8982bf0 0x55dbd8984fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f8095f59640 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd8982bf0 0x55dbd8984fe0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb31a1f3823b3817 server_cookie=3ae3326fc87ca414 in_seq=0 out_seq=0 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.073+0000 7f80797fa640 10 client.16836 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.074+0000 7f80797fa640 1 -- 172.21.15.8:0/3204353555 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dbd88c7310 con 0x55dbd8982bf0 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.074+0000 7f80797fa640 10 client.16836 handle_command_reply: tid=0 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.074+0000 7f809ad8f740 1 client.16836 shutdown 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 msgr2=0x55dbd8970250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970250 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8088004cf0 tx=0x7f8088004d20 comp rx=0 tx=0).stop 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 shutdown_connections 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970250 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd8982bf0 0x55dbd8984fe0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 --2- 172.21.15.8:0/3204353555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd896f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 >> 172.21.15.8:0/3204353555 conn(0x55dbd8963840 msgr2=0x55dbd89657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f80797fa640 0 client.16836 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 shutdown_connections 2024-02-19T16:28:08.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 1 -- 172.21.15.8:0/3204353555 wait complete. 2024-02-19T16:28:08.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.075+0000 7f809ad8f740 20 client.16836 trim_cache size 0 max 16384 2024-02-19T16:28:08.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 Processor -- start 2024-02-19T16:28:08.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 -- start start 2024-02-19T16:28:08.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8971f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd896b000 con 0x55dbd89728b0 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd897f860 con 0x55dbd8969380 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.091+0000 7f809ad8f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd89829f0 con 0x55dbd8969d30 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f8095758640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39104/0 (socket says 172.21.15.8:39104) 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f8095f59640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8971f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 -- 172.21.15.8:0/39718338 learned_addr learned my addr 172.21.15.8:0/39718338 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 -- 172.21.15.8:0/39718338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 msgr2=0x55dbd896fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 -- 172.21.15.8:0/39718338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 msgr2=0x55dbd8971f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8971f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 -- 172.21.15.8:0/39718338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd886bc00 con 0x55dbd89728b0 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f809675a640 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8088005cc0 tx=0x7f8088013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b992c810c0d15375 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f807b7fe640 1 -- 172.21.15.8:0/39718338 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088004d90 con 0x55dbd89728b0 2024-02-19T16:28:08.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f807b7fe640 1 -- 172.21.15.8:0/39718338 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8088005990 con 0x55dbd89728b0 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.092+0000 7f807b7fe640 1 -- 172.21.15.8:0/39718338 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088005140 con 0x55dbd89728b0 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 -- 172.21.15.8:0/39718338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 msgr2=0x55dbd8970260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8088005cc0 tx=0x7f8088013a70 comp rx=0 tx=0).stop 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 -- 172.21.15.8:0/39718338 shutdown_connections 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8970260 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd896fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 --2- 172.21.15.8:0/39718338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8971f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 -- 172.21.15.8:0/39718338 >> 172.21.15.8:0/39718338 conn(0x55dbd8963840 msgr2=0x55dbd89628e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 -- 172.21.15.8:0/39718338 shutdown_connections 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 -- 172.21.15.8:0/39718338 wait complete. 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 -- start start 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 Processor -- start 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8975da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.093+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd89762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809ad8f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8976820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809ad8f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd89dd440 con 0x55dbd89728b0 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809ad8f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd89dd5b0 con 0x55dbd8969380 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809ad8f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd89dd770 con 0x55dbd8969d30 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809ad8f740 10 client.0 _ll_register_callbacks cb 0x7f808c010e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8976820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8976820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39108/0 (socket says 172.21.15.8:39108) 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f8095758640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd89762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 -- 172.21.15.8:0/1773683869 learned_addr learned my addr 172.21.15.8:0/1773683869 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f8095f59640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8975da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 -- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 msgr2=0x55dbd89762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd89762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 -- 172.21.15.8:0/1773683869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 msgr2=0x55dbd8975da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8975da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 -- 172.21.15.8:0/1773683869 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbd88cf320 con 0x55dbd89728b0 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f8095758640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd89762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f8095f59640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8975da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:08.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809675a640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8976820 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8088006010 tx=0x7f80880097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2e3e376337d6ef server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f80797fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f80797fa640 1 -- 172.21.15.8:0/1773683869 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088004c50 con 0x55dbd89728b0 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f80797fa640 1 -- 172.21.15.8:0/1773683869 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f8088005000 con 0x55dbd89728b0 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.094+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbd88f6900 con 0x55dbd89728b0 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f80797fa640 1 -- 172.21.15.8:0/1773683869 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80880051c0 con 0x55dbd89728b0 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 10 client.16842 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:08.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dbd8891910 con 0x55dbd89728b0 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f80797fa640 1 -- 172.21.15.8:0/1773683869 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbd88f6900 con 0x55dbd89728b0 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f80797fa640 1 -- 172.21.15.8:0/1773683869 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8088010a30 con 0x55dbd89728b0 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 10 client.16842 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 20 client.16842 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd89dead0 0x55dbd89e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 4 client.16842 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dbd85d0860 con 0x55dbd89dead0 2024-02-19T16:28:08.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f8095f59640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd89dead0 0x55dbd89e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f80797fa640 10 client.16842 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.095+0000 7f8095f59640 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd89dead0 0x55dbd89e0ef0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd1526fb1b1e60f server_cookie=ebbf93ee2e1750c0 in_seq=0 out_seq=0 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.096+0000 7f80797fa640 1 -- 172.21.15.8:0/1773683869 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dbd85d0860 con 0x55dbd89dead0 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.096+0000 7f80797fa640 10 client.16842 handle_command_reply: tid=0 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.096+0000 7f809ad8f740 1 client.16842 shutdown 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 msgr2=0x55dbd8976820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8976820 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8088006010 tx=0x7f80880097b0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 shutdown_connections 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55dbd89728b0 0x55dbd8976820 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55dbd89dead0 0x55dbd89e0ef0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dbd8969d30 0x55dbd89762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 --2- 172.21.15.8:0/1773683869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbd8969380 0x55dbd8975da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80797fa640 0 client.16842 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 >> 172.21.15.8:0/1773683869 conn(0x55dbd8963840 msgr2=0x55dbd8989870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 shutdown_connections 2024-02-19T16:28:08.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 1 -- 172.21.15.8:0/1773683869 wait complete. 2024-02-19T16:28:08.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f809ad8f740 20 client.16842 trim_cache size 0 max 16384 2024-02-19T16:28:08.110 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:28:08.110 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:28:08.110 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "0b30836e-f81c-457d-a682-b6c9c4e2c92a", 2024-02-19T16:28:08.110 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:28:08.111 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:28:08.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f807c0a2230 msgr2=0x7f807c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f807c0a2230 0x7f807c0a46f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8080000c00 tx=0x7f8080005fd0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 msgr2=0x7f8090199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 0x7f8090199d40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9f0 tx=0x7f808c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 shutdown_connections 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f807c0a2230 0x7f807c0a46f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809019e600 0x7f809019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80901056a0 0x7f809019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 --2- 172.21.15.8:0/3511466290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80901005d0 0x7f8090199d40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.097+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 >> 172.21.15.8:0/3511466290 conn(0x7f80900774f0 msgr2=0x7f80900686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.098+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 shutdown_connections 2024-02-19T16:28:08.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.098+0000 7f80981bd640 1 -- 172.21.15.8:0/3511466290 wait complete. 2024-02-19T16:28:08.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:28:08.114 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:28:08.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 -- 172.21.15.8:0/1496109001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0100640 msgr2=0x7fa8f0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/1496109001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0100640 0x7fa8f0100ac0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009930 tx=0x7fa8e002bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:08.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 -- 172.21.15.8:0/1496109001 shutdown_connections 2024-02-19T16:28:08.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/1496109001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f0101000 0x7fa8f010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/1496109001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0100640 0x7fa8f0100ac0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/1496109001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8f0106510 0x7fa8f01068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 -- 172.21.15.8:0/1496109001 >> 172.21.15.8:0/1496109001 conn(0x7fa8f0077180 msgr2=0x7fa8f0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 -- 172.21.15.8:0/1496109001 shutdown_connections 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.218+0000 7fa8f79d0640 1 -- 172.21.15.8:0/1496109001 wait complete. 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 Processor -- start 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 -- start start 2024-02-19T16:28:08.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f0100640 0x7fa8f019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 0x7fa8f019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8f0106510 0x7fa8f01978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0197ef0 con 0x7fa8f0101000 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0198060 con 0x7fa8f0100640 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f79d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0198220 con 0x7fa8f0106510 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 0x7fa8f019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 0x7fa8f019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39134/0 (socket says 172.21.15.8:39134) 2024-02-19T16:28:08.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/2541420349 learned_addr learned my addr 172.21.15.8:0/2541420349 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f576c640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f0100640 0x7fa8f019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8f0106510 0x7fa8f01978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/2541420349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8f0106510 msgr2=0x7fa8f01978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8f0106510 0x7fa8f01978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/2541420349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f0100640 msgr2=0x7fa8f019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f0100640 0x7fa8f019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.219+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/2541420349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009560 con 0x7fa8f0101000 2024-02-19T16:28:08.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.220+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 0x7fa8f019dc30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009230 tx=0x7fa8e002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.220+0000 7fa8e67fc640 1 -- 172.21.15.8:0/2541420349 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004350 con 0x7fa8f0101000 2024-02-19T16:28:08.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.220+0000 7fa8e67fc640 1 -- 172.21.15.8:0/2541420349 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8e00044b0 con 0x7fa8f0101000 2024-02-19T16:28:08.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.220+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f01984a0 con 0x7fa8f0101000 2024-02-19T16:28:08.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.220+0000 7fa8e67fc640 1 -- 172.21.15.8:0/2541420349 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0003d00 con 0x7fa8f0101000 2024-02-19T16:28:08.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.220+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f01a4790 con 0x7fa8f0101000 2024-02-19T16:28:08.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8e67fc640 1 -- 172.21.15.8:0/2541420349 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fa8e0004df0 con 0x7fa8f0101000 2024-02-19T16:28:08.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 Processor -- start 2024-02-19T16:28:08.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 -- start start 2024-02-19T16:28:08.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f10690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f19270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f197b0 0x562438f1bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438e7f280 con 0x562438f197b0 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438e7c7b0 con 0x562438f102b0 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438e77f90 con 0x562438f10bd0 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f197b0 0x562438f1bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8f576c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f19270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8f5f6d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f10690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8f576c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f19270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:46900/0 (socket says 172.21.15.8:46900) 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.223+0000 7fa8f576c640 1 -- 172.21.15.8:0/3647743526 learned_addr learned my addr 172.21.15.8:0/3647743526 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8e67fc640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa8dc0a2230 0x7fa8dc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3647743526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 msgr2=0x562438f19270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f19270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3647743526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 msgr2=0x562438f10690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f10690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3647743526 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562438e36570 con 0x562438f197b0 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa8dc0a2230 0x7fa8dc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8e67fc640 1 -- 172.21.15.8:0/2541420349 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa8e00f0310 con 0x7fa8f0101000 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f197b0 0x562438f1bba0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00ed7c0 tx=0x7fa8e0087b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a9783b2f84a4a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8db7fe640 1 -- 172.21.15.8:0/3647743526 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00bc5a0 con 0x562438f197b0 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa8dc0a2230 0x7fa8dc0a46f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005fd0 tx=0x7fa8e8004ab0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8db7fe640 1 -- 172.21.15.8:0/3647743526 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8e00bc3c0 con 0x562438f197b0 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.224+0000 7fa8db7fe640 1 -- 172.21.15.8:0/3647743526 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00035f0 con 0x562438f197b0 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3647743526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f197b0 msgr2=0x562438f1bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f197b0 0x562438f1bba0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00ed7c0 tx=0x7fa8e0087b60 comp rx=0 tx=0).stop 2024-02-19T16:28:08.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3647743526 shutdown_connections 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f197b0 0x562438f1bba0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f19270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3647743526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f10690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3647743526 >> 172.21.15.8:0/3647743526 conn(0x562438f0a990 msgr2=0x562438f0ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3647743526 shutdown_connections 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3647743526 wait complete. 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.225+0000 7fa8fa5ac740 1 -- start start 2024-02-19T16:28:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 Processor -- start 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 0x562438f191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f16560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f197b0 0x562438f16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438b4b430 con 0x562438f102b0 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438b07e00 con 0x562438f197b0 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438dfcfd0 con 0x562438f10bd0 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8fa5ac740 10 client.0 _ll_register_callbacks cb 0x562438e52fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 0x562438f191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 0x562438f191d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39160/0 (socket says 172.21.15.8:39160) 2024-02-19T16:28:08.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 -- 172.21.15.8:0/2107241088 learned_addr learned my addr 172.21.15.8:0/2107241088 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f197b0 0x562438f16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f16560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 -- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 msgr2=0x562438f16560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f16560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 -- 172.21.15.8:0/2107241088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f197b0 msgr2=0x562438f16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f197b0 0x562438f16b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.226+0000 7fa8f576c640 1 -- 172.21.15.8:0/2107241088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f01a4790 con 0x562438f102b0 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8f576c640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 0x562438f191d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec012740 tx=0x7fa8ec012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eee46f8ff32a26a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/2107241088 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec017070 con 0x562438f102b0 2024-02-19T16:28:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/2107241088 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa8ec004050 con 0x562438f102b0 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562438b777e0 con 0x562438f102b0 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/2107241088 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8ec004250 con 0x562438f102b0 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562438e76610 con 0x562438f102b0 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/2107241088 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562438b777e0 con 0x562438f102b0 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/2107241088 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa8ec003a60 con 0x562438f102b0 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 20 client.16854 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f29730 0x562438f2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 4 client.16854 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.227+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562438bb4720 con 0x562438f29730 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.228+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f29730 0x562438f2bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.228+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f29730 0x562438f2bb50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeedee7451cf4bcd server_cookie=91fe7c996d54f22b in_seq=0 out_seq=0 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.228+0000 7fa8d8ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/2107241088 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562438bb4720 con 0x562438f29730 2024-02-19T16:28:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8d8ff9640 10 client.16854 handle_command_reply: tid=0 2024-02-19T16:28:08.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 client.16854 shutdown 2024-02-19T16:28:08.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 msgr2=0x562438f191d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 0x562438f191d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec012740 tx=0x7fa8ec012c00 comp rx=0 tx=0).stop 2024-02-19T16:28:08.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 shutdown_connections 2024-02-19T16:28:08.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f197b0 0x562438f16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f29730 0x562438f2bb50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f10bd0 0x562438f16560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/2107241088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f102b0 0x562438f191d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8d8ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 >> 172.21.15.8:0/2107241088 conn(0x562438f0a990 msgr2=0x562438f0c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 shutdown_connections 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/2107241088 wait complete. 2024-02-19T16:28:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.229+0000 7fa8fa5ac740 20 client.16854 trim_cache size 0 max 16384 2024-02-19T16:28:08.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 Processor -- start 2024-02-19T16:28:08.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 -- start start 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f16500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f18c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438f0a220 con 0x562438f10bd0 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438f178b0 con 0x562438f102b0 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438f0ff10 con 0x562438f197b0 2024-02-19T16:28:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f18c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f576c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f16500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f18c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39172/0 (socket says 172.21.15.8:39172) 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3714377318 learned_addr learned my addr 172.21.15.8:0/3714377318 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3714377318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 msgr2=0x562438f191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3714377318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 msgr2=0x562438f16500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f16500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3714377318 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562438e12ac0 con 0x562438f10bd0 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.248+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.249+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f18c90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00ed870 tx=0x7fa8e00037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215556ce4e542f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.249+0000 7fa8daffd640 1 -- 172.21.15.8:0/3714377318 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00f05d0 con 0x562438f10bd0 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.249+0000 7fa8daffd640 1 -- 172.21.15.8:0/3714377318 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8e00035f0 con 0x562438f10bd0 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.249+0000 7fa8daffd640 1 -- 172.21.15.8:0/3714377318 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00c5200 con 0x562438f10bd0 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3714377318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 msgr2=0x562438f18c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f18c90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00ed870 tx=0x7fa8e00037d0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3714377318 shutdown_connections 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f18c90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3714377318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f16500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3714377318 >> 172.21.15.8:0/3714377318 conn(0x562438f0a990 msgr2=0x562438f15df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3714377318 shutdown_connections 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3714377318 wait complete. 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 -- start start 2024-02-19T16:28:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.250+0000 7fa8fa5ac740 1 Processor -- start 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f1a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f1a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438f20970 con 0x562438f10bd0 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438f20ae0 con 0x562438f102b0 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562438f20ca0 con 0x562438f197b0 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8fa5ac740 10 client.0 _ll_register_callbacks cb 0x7fa8ec009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:08.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f1a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f576c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f1a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f1a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39178/0 (socket says 172.21.15.8:39178) 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3409519983 learned_addr learned my addr 172.21.15.8:0/3409519983 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f5f6d640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 msgr2=0x562438f20430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f20430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3409519983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 msgr2=0x562438f1a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:08.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f1a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 -- 172.21.15.8:0/3409519983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562438e36570 con 0x562438f10bd0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f576c640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f1a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8f4f6b640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f1a8c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0000c00 tx=0x7fa8e002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7502bd9a39d4286 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.251+0000 7fa8d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/3409519983 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00f05d0 con 0x562438f10bd0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/3409519983 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fa8e00bd160 con 0x562438f10bd0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8e00bc3c0 con 0x562438f10bd0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/3409519983 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8e00bca60 con 0x562438f10bd0 2024-02-19T16:28:08.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 10 client.16860 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562438e386a0 con 0x562438f10bd0 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/3409519983 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8e00bc3c0 con 0x562438f10bd0 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/3409519983 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa8e003c070 con 0x562438f10bd0 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 20 client.16860 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f856d0 0x562438f87af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:08.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 4 client.16860 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562438c03890 con 0x562438f856d0 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.252+0000 7fa8f576c640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f856d0 0x562438f87af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8f576c640 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f856d0 0x562438f87af0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293baccd74ae8e8 server_cookie=defc7980272b55dd in_seq=0 out_seq=0 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8d8ff9640 10 client.16860 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8d8ff9640 1 -- 172.21.15.8:0/3409519983 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x562438c03890 con 0x562438f856d0 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8d8ff9640 10 client.16860 handle_command_reply: tid=0 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 client.16860 shutdown 2024-02-19T16:28:08.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 msgr2=0x562438f1a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f1a8c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0000c00 tx=0x7fa8e002bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 shutdown_connections 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562438f856d0 0x562438f87af0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562438f197b0 0x562438f20430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8d8ff9640 0 client.16860 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562438f10bd0 0x562438f1a8c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 --2- 172.21.15.8:0/3409519983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562438f102b0 0x562438f1a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.253+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 >> 172.21.15.8:0/3409519983 conn(0x562438f0a990 msgr2=0x562438f152e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 shutdown_connections 2024-02-19T16:28:08.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8fa5ac740 1 -- 172.21.15.8:0/3409519983 wait complete. 2024-02-19T16:28:08.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8fa5ac740 20 client.16860 trim_cache size 0 max 16384 2024-02-19T16:28:08.266 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:28:08.266 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2868 inodes in the stack)", 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: "0b30836e-f81c-457d-a682-b6c9c4e2c92a": { 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "0b30836e-f81c-457d-a682-b6c9c4e2c92a", 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:28:08.267 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:28:08.268 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:28:08.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa8dc0a2230 msgr2=0x7fa8dc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa8dc0a2230 0x7fa8dc0a46f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005fd0 tx=0x7fa8e8004ab0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 msgr2=0x7fa8f019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 0x7fa8f019dc30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009230 tx=0x7fa8e002c650 comp rx=0 tx=0).stop 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 shutdown_connections 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa8dc0a2230 0x7fa8dc0a46f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8f0106510 0x7fa8f01978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f0101000 0x7fa8f019dc30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 --2- 172.21.15.8:0/2541420349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f0100640 0x7fa8f019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:08.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 >> 172.21.15.8:0/2541420349 conn(0x7fa8f0077180 msgr2=0x7fa8f0069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:08.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 shutdown_connections 2024-02-19T16:28:08.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:08.254+0000 7fa8f79d0640 1 -- 172.21.15.8:0/2541420349 wait complete. 2024-02-19T16:28:08.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:08 smithi008 ceph-mon[32573]: pgmap v160: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:28:08.271 INFO:tasks.cephfs.filesystem:scrub status for tag:0b30836e-f81c-457d-a682-b6c9c4e2c92a - {'path': '/', 'tag': '0b30836e-f81c-457d-a682-b6c9c4e2c92a', 'options': 'recursive,force'} 2024-02-19T16:28:08.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:08 smithi191 ceph-mon[37203]: pgmap v160: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:28:08.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:08 smithi156 ceph-mon[36502]: pgmap v160: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-02-19T16:28:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:09 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:09 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:09 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:09.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:09 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:09 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:09.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:09 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:09 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:09.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:09 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:09 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:09.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:09 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:09 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:09.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:09 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:09 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:09 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:09 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:09 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:09.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:09 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:09.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:09 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:28:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:10.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:10.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.271 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:10 smithi008 ceph-mon[32573]: pgmap v161: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:10.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:28:10.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:10.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:10.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:10 smithi191 ceph-mon[37203]: pgmap v161: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:10.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:28:10.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:10.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:10.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:10.331 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:10 smithi156 ceph-mon[36502]: pgmap v161: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:12.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:12 smithi008 ceph-mon[32573]: pgmap v162: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:28:12.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:12 smithi191 ceph-mon[37203]: pgmap v162: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:28:12.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:12 smithi156 ceph-mon[36502]: pgmap v162: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:28:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:14 smithi008 ceph-mon[32573]: pgmap v163: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:28:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:14 smithi191 ceph-mon[37203]: pgmap v163: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:28:14.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:14 smithi156 ceph-mon[36502]: pgmap v163: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-02-19T16:28:15.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:15 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:28:15.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:15 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:28:15.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:15 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:28:15.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:15 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:28:15.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:15 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:28:15.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:15 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:28:15.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:15 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:28:15.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:15 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:28:15.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:15 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:28:16.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:16 smithi008 ceph-mon[32573]: pgmap v164: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:16.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:16 smithi191 ceph-mon[37203]: pgmap v164: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:16.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:16 smithi156 ceph-mon[36502]: pgmap v164: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-02-19T16:28:18.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:18 smithi191 ceph-mon[37203]: pgmap v165: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:28:18.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:18 smithi156 ceph-mon[36502]: pgmap v165: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:28:18.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:18 smithi008 ceph-mon[32573]: pgmap v165: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-19T16:28:20.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:20 smithi191 ceph-mon[37203]: pgmap v166: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:28:20.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:20 smithi156 ceph-mon[36502]: pgmap v166: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:28:20.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:20 smithi008 ceph-mon[32573]: pgmap v166: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-02-19T16:28:22.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:22 smithi191 ceph-mon[37203]: pgmap v167: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:28:22.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:22 smithi156 ceph-mon[36502]: pgmap v167: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:28:22.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:22 smithi008 ceph-mon[32573]: pgmap v167: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:28:24.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:24 smithi156 ceph-mon[36502]: pgmap v168: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:28:24.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:24 smithi008 ceph-mon[32573]: pgmap v168: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:28:24.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:24 smithi191 ceph-mon[37203]: pgmap v168: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-19T16:28:26.330 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:26 smithi156 ceph-mon[36502]: pgmap v169: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:28:26.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:26 smithi008 ceph-mon[32573]: pgmap v169: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:28:26.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:26 smithi191 ceph-mon[37203]: pgmap v169: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:28:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:28 smithi008 ceph-mon[32573]: pgmap v170: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:28 smithi191 ceph-mon[37203]: pgmap v170: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:28 smithi156 ceph-mon[36502]: pgmap v170: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:30.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:30 smithi008 ceph-mon[32573]: pgmap v171: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:30 smithi191 ceph-mon[37203]: pgmap v171: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:30 smithi156 ceph-mon[36502]: pgmap v171: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:32.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:32 smithi008 ceph-mon[32573]: pgmap v172: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:32.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:32 smithi191 ceph-mon[37203]: pgmap v172: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:32.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:32 smithi156 ceph-mon[36502]: pgmap v172: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:34 smithi008 ceph-mon[32573]: pgmap v173: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:34 smithi191 ceph-mon[37203]: pgmap v173: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:34 smithi156 ceph-mon[36502]: pgmap v173: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:36.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:36 smithi008 ceph-mon[32573]: pgmap v174: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:36.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:36 smithi191 ceph-mon[37203]: pgmap v174: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:36 smithi156 ceph-mon[36502]: pgmap v174: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:38.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:28:38.273 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:28:38.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/1881376519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 msgr2=0x7f7a38069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/1881376519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a38069fc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009930 tx=0x7f7a2c02bc20 comp rx=0 tx=0).stop 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/1881376519 shutdown_connections 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/1881376519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 0x7f7a3810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/1881376519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a38069fc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/1881376519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 0x7f7a38107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.390+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/1881376519 >> 172.21.15.8:0/1881376519 conn(0x7f7a38077190 msgr2=0x7f7a380775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/1881376519 shutdown_connections 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/1881376519 wait complete. 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 Processor -- start 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 -- start start 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a3819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 0x7f7a3819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 0x7f7a38199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3819a110 con 0x7f7a38069b60 2024-02-19T16:28:38.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3819a280 con 0x7f7a38107640 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3ff7a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3819a440 con 0x7f7a3806a500 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3dd16640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a3819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3dd16640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a3819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47392/0 (socket says 172.21.15.8:47392) 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3dd16640 1 -- 172.21.15.8:0/2787335869 learned_addr learned my addr 172.21.15.8:0/2787335869 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.391+0000 7f7a3d515640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 0x7f7a3819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 0x7f7a38199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3d515640 1 -- 172.21.15.8:0/2787335869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 msgr2=0x7f7a38199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3d515640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 0x7f7a38199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3d515640 1 -- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 msgr2=0x7f7a3819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3d515640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a3819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3d515640 1 -- 172.21.15.8:0/2787335869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009560 con 0x7f7a3806a500 2024-02-19T16:28:38.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 0x7f7a38199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:38.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3d515640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 0x7f7a3819fe50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009230 tx=0x7f7a2c02c5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a2affd640 1 -- 172.21.15.8:0/2787335869 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0042d0 con 0x7f7a3806a500 2024-02-19T16:28:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3819a6c0 con 0x7f7a3806a500 2024-02-19T16:28:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a2affd640 1 -- 172.21.15.8:0/2787335869 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a2c004430 con 0x7f7a3806a500 2024-02-19T16:28:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a2affd640 1 -- 172.21.15.8:0/2787335869 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c003bd0 con 0x7f7a3806a500 2024-02-19T16:28:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.392+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a381a69b0 con 0x7f7a3806a500 2024-02-19T16:28:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.395+0000 7f7a2affd640 1 -- 172.21.15.8:0/2787335869 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f7a2c003d30 con 0x7f7a3806a500 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.395+0000 7f7a42c32740 1 Processor -- start 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.395+0000 7f7a42c32740 1 -- start start 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedc6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedd1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a42c32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ded35280 con 0x55c5dedcf7b0 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a42c32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ded327b0 con 0x55c5dedc62b0 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a42c32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ded2df90 con 0x55c5dedc6bd0 2024-02-19T16:28:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3d515640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedd1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedc6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedc6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:58896/0 (socket says 172.21.15.8:58896) 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 -- 172.21.15.8:0/3973454604 learned_addr learned my addr 172.21.15.8:0/3973454604 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a2affd640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7a240a2230 0x7f7a240a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 -- 172.21.15.8:0/3973454604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 msgr2=0x55c5dedcf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 -- 172.21.15.8:0/3973454604 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 msgr2=0x55c5dedd1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedd1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 -- 172.21.15.8:0/3973454604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5decec570 con 0x55c5dedc62b0 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3d515640 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedd1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a2affd640 1 -- 172.21.15.8:0/2787335869 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f7a2c0f02b0 con 0x7f7a3806a500 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.396+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7a240a2230 0x7f7a240a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a3e517640 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedc6690 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a300117e0 tx=0x7f7a30011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76717c4fcf6ca0dd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a23fff640 1 -- 172.21.15.8:0/3973454604 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30021070 con 0x55c5dedc62b0 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a23fff640 1 -- 172.21.15.8:0/3973454604 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a30010b20 con 0x55c5dedc62b0 2024-02-19T16:28:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a23fff640 1 -- 172.21.15.8:0/3973454604 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3001c9a0 con 0x55c5dedc62b0 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 -- 172.21.15.8:0/3973454604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 msgr2=0x55c5dedc6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedc6690 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a300117e0 tx=0x7f7a30011ca0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7a240a2230 0x7f7a240a46f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a30011490 tx=0x7f7a30021000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 -- 172.21.15.8:0/3973454604 shutdown_connections 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedd1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3973454604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedc6690 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.397+0000 7f7a42c32740 1 -- 172.21.15.8:0/3973454604 >> 172.21.15.8:0/3973454604 conn(0x55c5dedc0990 msgr2=0x55c5dedc0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 -- 172.21.15.8:0/3973454604 shutdown_connections 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 -- 172.21.15.8:0/3973454604 wait complete. 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 -- start start 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 Processor -- start 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedcf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dea01430 con 0x55c5dedcf7b0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de9bde00 con 0x55c5dedc62b0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5decb2fd0 con 0x55c5dedc6bd0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.398+0000 7f7a42c32740 10 client.0 _ll_register_callbacks cb 0x55c5ded08fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47406/0 (socket says 172.21.15.8:47406) 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 -- 172.21.15.8:0/2037271213 learned_addr learned my addr 172.21.15.8:0/2037271213 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3d515640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedcf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 -- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 msgr2=0x55c5dedcc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 -- 172.21.15.8:0/2037271213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 msgr2=0x55c5dedcf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedcf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 -- 172.21.15.8:0/2037271213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a300114c0 con 0x55c5dedcf7b0 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a3e517640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedccb30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7a30010ec0 tx=0x7f7a3001c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2cf028c1e32af4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a217fa640 1 -- 172.21.15.8:0/2037271213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30011090 con 0x55c5dedcf7b0 2024-02-19T16:28:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5dea2d7e0 con 0x55c5dedcf7b0 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a217fa640 1 -- 172.21.15.8:0/2037271213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f7a30010850 con 0x55c5dedcf7b0 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a217fa640 1 -- 172.21.15.8:0/2037271213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a300049e0 con 0x55c5dedcf7b0 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a42c32740 10 client.16872 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.399+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c5ded2c610 con 0x55c5dedcf7b0 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a217fa640 1 -- 172.21.15.8:0/2037271213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5dea2d7e0 con 0x55c5dedcf7b0 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a217fa640 1 -- 172.21.15.8:0/2037271213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a30004020 con 0x55c5dedcf7b0 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a42c32740 10 client.16872 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a42c32740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a42c32740 20 client.16872 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a42c32740 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5deddf940 0x55c5dede1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a42c32740 4 client.16872 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5dea6a720 con 0x55c5deddf940 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5deddf940 0x55c5dede1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5deddf940 0x55c5dede1d60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a74463ba58d5e22 server_cookie=fc94895a853cd8e1 in_seq=0 out_seq=0 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.400+0000 7f7a217fa640 10 client.16872 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.401+0000 7f7a217fa640 1 -- 172.21.15.8:0/2037271213 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5dea6a720 con 0x55c5deddf940 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.401+0000 7f7a217fa640 10 client.16872 handle_command_reply: tid=0 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.401+0000 7f7a42c32740 1 client.16872 shutdown 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.401+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 msgr2=0x55c5dedccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.401+0000 7f7a42c32740 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedccb30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7a30010ec0 tx=0x7f7a3001c6c0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.401+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 shutdown_connections 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedcf7b0 0x55c5dedccb30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5deddf940 0x55c5dede1d60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 --2- 172.21.15.8:0/2037271213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc62b0 0x55c5dedcf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a217fa640 0 client.16872 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 >> 172.21.15.8:0/2037271213 conn(0x55c5dedc0990 msgr2=0x55c5dedc2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 shutdown_connections 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 1 -- 172.21.15.8:0/2037271213 wait complete. 2024-02-19T16:28:38.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.402+0000 7f7a42c32740 20 client.16872 trim_cache size 0 max 16384 2024-02-19T16:28:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 Processor -- start 2024-02-19T16:28:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 -- start start 2024-02-19T16:28:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedc2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dedc01d0 con 0x55c5dedc62b0 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dedc5630 con 0x55c5dedc6bd0 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a42c32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dedbae20 con 0x55c5dedcf7b0 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedc2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedc2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47408/0 (socket says 172.21.15.8:47408) 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 -- 172.21.15.8:0/3691646058 learned_addr learned my addr 172.21.15.8:0/3691646058 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3d515640 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3e517640 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 -- 172.21.15.8:0/3691646058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedcf7b0 msgr2=0x55c5dedcec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 -- 172.21.15.8:0/3691646058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc6bd0 msgr2=0x55c5dedcc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.418+0000 7f7a3dd16640 1 -- 172.21.15.8:0/3691646058 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5decc8ac0 con 0x55c5dedc62b0 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedc2520 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7a34004700 tx=0x7f7a34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dc3e46598addaa server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a237fe640 1 -- 172.21.15.8:0/3691646058 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34017030 con 0x55c5dedc62b0 2024-02-19T16:28:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a237fe640 1 -- 172.21.15.8:0/3691646058 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a34018040 con 0x55c5dedc62b0 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a237fe640 1 -- 172.21.15.8:0/3691646058 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a340124b0 con 0x55c5dedc62b0 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 -- 172.21.15.8:0/3691646058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 msgr2=0x55c5dedc2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedc2520 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7a34004700 tx=0x7f7a34005f90 comp rx=0 tx=0).stop 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 -- 172.21.15.8:0/3691646058 shutdown_connections 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 --2- 172.21.15.8:0/3691646058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedc2520 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 -- 172.21.15.8:0/3691646058 >> 172.21.15.8:0/3691646058 conn(0x55c5dedc0990 msgr2=0x55c5dedc1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 -- 172.21.15.8:0/3691646058 shutdown_connections 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.419+0000 7f7a42c32740 1 -- 172.21.15.8:0/3691646058 wait complete. 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 -- start start 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 Processor -- start 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedcb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dedd6820 con 0x55c5dedc62b0 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dedd6990 con 0x55c5dedcf7b0 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dedd6b50 con 0x55c5dedc6bd0 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a42c32740 10 client.0 _ll_register_callbacks cb 0x7f7a3001f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a3dd16640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedcb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a3dd16640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedcb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47422/0 (socket says 172.21.15.8:47422) 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a3dd16640 1 -- 172.21.15.8:0/134632262 learned_addr learned my addr 172.21.15.8:0/134632262 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a3d515640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.420+0000 7f7a3e517640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a3dd16640 1 -- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 msgr2=0x55c5dedcbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a3dd16640 1 -- 172.21.15.8:0/134632262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedcf7b0 msgr2=0x55c5dedcc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a3dd16640 1 -- 172.21.15.8:0/134632262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34009c80 con 0x55c5dedc62b0 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a3dd16640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedcb6a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7a34003f00 tx=0x7f7a34003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51d21624284db89 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a217fa640 1 -- 172.21.15.8:0/134632262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34017040 con 0x55c5dedc62b0 2024-02-19T16:28:38.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a217fa640 1 -- 172.21.15.8:0/134632262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f7a34022980 con 0x55c5dedc62b0 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a30010b20 con 0x55c5dedc62b0 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a217fa640 1 -- 172.21.15.8:0/134632262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a34022b40 con 0x55c5dedc62b0 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 10 client.16878 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c5decee6a0 con 0x55c5dedc62b0 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a217fa640 1 -- 172.21.15.8:0/134632262 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a30010b20 con 0x55c5dedc62b0 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a217fa640 1 -- 172.21.15.8:0/134632262 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a34029070 con 0x55c5dedc62b0 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 10 client.16878 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:38.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 20 client.16878 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5dee3be30 0x55c5dee3e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 4 client.16878 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.421+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c5deab9890 con 0x55c5dee3be30 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a3d515640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5dee3be30 0x55c5dee3e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a3d515640 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5dee3be30 0x55c5dee3e250 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5445eefc258365e2 server_cookie=d8aecad3145408be in_seq=0 out_seq=0 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a217fa640 10 client.16878 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a217fa640 1 -- 172.21.15.8:0/134632262 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c5deab9890 con 0x55c5dee3be30 2024-02-19T16:28:38.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a217fa640 10 client.16878 handle_command_reply: tid=0 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 client.16878 shutdown 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 msgr2=0x55c5dedcb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedcb6a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7a34003f00 tx=0x7f7a34003c40 comp rx=0 tx=0).stop 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 shutdown_connections 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5dedcf7b0 0x55c5dedcc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c5dee3be30 0x55c5dee3e250 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c5dedc6bd0 0x55c5dedcbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 --2- 172.21.15.8:0/134632262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c5dedc62b0 0x55c5dedcb6a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a217fa640 0 client.16878 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:38.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 >> 172.21.15.8:0/134632262 conn(0x55c5dedc0990 msgr2=0x55c5dedbf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 shutdown_connections 2024-02-19T16:28:38.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.422+0000 7f7a42c32740 1 -- 172.21.15.8:0/134632262 wait complete. 2024-02-19T16:28:38.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a42c32740 20 client.16878 trim_cache size 0 max 16384 2024-02-19T16:28:38.434 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:28:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:28:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:28:38.435 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:28:38.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7a240a2230 msgr2=0x7f7a240a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7a240a2230 0x7f7a240a46f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a30011490 tx=0x7f7a30021000 comp rx=0 tx=0).stop 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 msgr2=0x7f7a3819fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 0x7f7a3819fe50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009230 tx=0x7f7a2c02c5c0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 shutdown_connections 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a38107640 0x7f7a38199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7a240a2230 0x7f7a240a46f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a3806a500 0x7f7a3819fe50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 --2- 172.21.15.8:0/2787335869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a38069b60 0x7f7a3819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 >> 172.21.15.8:0/2787335869 conn(0x7f7a38077190 msgr2=0x7f7a3810c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 shutdown_connections 2024-02-19T16:28:38.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.423+0000 7f7a3ff7a640 1 -- 172.21.15.8:0/2787335869 wait complete. 2024-02-19T16:28:38.438 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:28:38.438 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:28:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2560221221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e781056f0 msgr2=0x7f9e7810d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2560221221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e781056f0 0x7f9e7810d2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009be0 tx=0x7f9e7002bef0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2560221221 shutdown_connections 2024-02-19T16:28:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2560221221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e781056f0 0x7f9e7810d2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2560221221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78100fd0 0x7f9e78105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2560221221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e78100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2560221221 >> 172.21.15.8:0/2560221221 conn(0x7f9e78077490 msgr2=0x7f9e780778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2560221221 shutdown_connections 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.541+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2560221221 wait complete. 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 Processor -- start 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 -- start start 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e7819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e781056f0 0x7f9e7819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e78197930 0x7f9e78197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78198320 con 0x7f9e78100620 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78198490 con 0x7f9e781056f0 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7d0a4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78198650 con 0x7f9e78197930 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e76d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e7819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e6ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e781056f0 0x7f9e7819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e76d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e7819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47438/0 (socket says 172.21.15.8:47438) 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e76d9d640 1 -- 172.21.15.8:0/2277317985 learned_addr learned my addr 172.21.15.8:0/2277317985 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.542+0000 7f9e7759e640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e78197930 0x7f9e78197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e76d9d640 1 -- 172.21.15.8:0/2277317985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e78197930 msgr2=0x7f9e78197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e76d9d640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e78197930 0x7f9e78197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e76d9d640 1 -- 172.21.15.8:0/2277317985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e781056f0 msgr2=0x7f9e7819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e76d9d640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e781056f0 0x7f9e7819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e76d9d640 1 -- 172.21.15.8:0/2277317985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68009560 con 0x7f9e78100620 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e76d9d640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e7819d780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800e9c0 tx=0x7f9e6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800cde0 con 0x7f9e78100620 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e70009840 con 0x7f9e78100620 2024-02-19T16:28:38.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e680035f0 con 0x7f9e78100620 2024-02-19T16:28:38.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68010650 con 0x7f9e78100620 2024-02-19T16:28:38.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.543+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e781a4a10 con 0x7f9e78100620 2024-02-19T16:28:38.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.544+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e78068730 con 0x7f9e78100620 2024-02-19T16:28:38.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.545+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f9e68003ad0 con 0x7f9e78100620 2024-02-19T16:28:38.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.546+0000 7f9e74d99640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9e640a2230 0x7f9e640a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.546+0000 7f9e6ffff640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9e640a2230 0x7f9e640a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.546+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f9e68014070 con 0x7f9e78100620 2024-02-19T16:28:38.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.546+0000 7f9e6ffff640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9e640a2230 0x7f9e640a46f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9e7806ac00 tx=0x7f9e60005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.550+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9e68097170 con 0x7f9e78100620 2024-02-19T16:28:38.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:38 smithi008 ceph-mon[32573]: pgmap v175: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:38.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.772+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e78198e60 con 0x7f9e78100620 2024-02-19T16:28:38.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.773+0000 7f9e74d99640 1 -- 172.21.15.8:0/2277317985 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f9e68003760 con 0x7f9e78100620 2024-02-19T16:28:38.774 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:28:38.774 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:28:38.775 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:28:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.774+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9e640a2230 msgr2=0x7f9e640a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.774+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9e640a2230 0x7f9e640a46f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9e7806ac00 tx=0x7f9e60005f90 comp rx=0 tx=0).stop 2024-02-19T16:28:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 msgr2=0x7f9e7819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e7819d780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800e9c0 tx=0x7f9e6800ee80 comp rx=0 tx=0).stop 2024-02-19T16:28:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 shutdown_connections 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9e640a2230 0x7f9e640a46f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e78197930 0x7f9e78197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e781056f0 0x7f9e7819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 --2- 172.21.15.8:0/2277317985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e78100620 0x7f9e7819d780 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 >> 172.21.15.8:0/2277317985 conn(0x7f9e78077490 msgr2=0x7f9e781042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 shutdown_connections 2024-02-19T16:28:38.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.775+0000 7f9e7d0a4640 1 -- 172.21.15.8:0/2277317985 wait complete. 2024-02-19T16:28:38.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:38 smithi191 ceph-mon[37203]: pgmap v175: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:38.787 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:28:38.787 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:28:38.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:38 smithi156 ceph-mon[36502]: pgmap v175: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:38.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 -- 172.21.15.8:0/3082738078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 msgr2=0x7f48a41009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 --2- 172.21.15.8:0/3082738078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a41009b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4894009930 tx=0x7f489402bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 -- 172.21.15.8:0/3082738078 shutdown_connections 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 --2- 172.21.15.8:0/3082738078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48a41056a0 0x7f48a410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 --2- 172.21.15.8:0/3082738078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a4100f80 0x7f48a4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 --2- 172.21.15.8:0/3082738078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a41009b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 -- 172.21.15.8:0/3082738078 >> 172.21.15.8:0/3082738078 conn(0x7f48a40774f0 msgr2=0x7f48a4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.894+0000 7f48a91ae640 1 -- 172.21.15.8:0/3082738078 shutdown_connections 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.895+0000 7f48a91ae640 1 -- 172.21.15.8:0/3082738078 wait complete. 2024-02-19T16:28:38.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.895+0000 7f48a91ae640 1 Processor -- start 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.895+0000 7f48a91ae640 1 -- start start 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a4102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 0x7f48a41025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48a4102ae0 0x7f48a41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a41a6300 con 0x7f48a41056a0 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a41a6470 con 0x7f48a41005d0 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a41a6630 con 0x7f48a4102ae0 2024-02-19T16:28:38.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 0x7f48a41025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a4102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 0x7f48a41025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47452/0 (socket says 172.21.15.8:47452) 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a4102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:58920/0 (socket says 172.21.15.8:58920) 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 -- 172.21.15.8:0/1614885326 learned_addr learned my addr 172.21.15.8:0/1614885326 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a359e640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48a4102ae0 0x7f48a41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 -- 172.21.15.8:0/1614885326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48a4102ae0 msgr2=0x7f48a41a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48a4102ae0 0x7f48a41a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 -- 172.21.15.8:0/1614885326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 msgr2=0x7f48a4102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a4102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 -- 172.21.15.8:0/1614885326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009490 con 0x7f48a41056a0 2024-02-19T16:28:38.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a4102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:38.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a259c640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 0x7f48a41025a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9a0 tx=0x7f489800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f488bfff640 1 -- 172.21.15.8:0/1614885326 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800cd80 con 0x7f48a41056a0 2024-02-19T16:28:38.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4894009560 con 0x7f48a41056a0 2024-02-19T16:28:38.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f488bfff640 1 -- 172.21.15.8:0/1614885326 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48980035f0 con 0x7f48a41056a0 2024-02-19T16:28:38.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f488bfff640 1 -- 172.21.15.8:0/1614885326 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898010690 con 0x7f48a41056a0 2024-02-19T16:28:38.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.896+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a41a6c90 con 0x7f48a41056a0 2024-02-19T16:28:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f488bfff640 1 -- 172.21.15.8:0/1614885326 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4898003ad0 con 0x7f48a41056a0 2024-02-19T16:28:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 Processor -- start 2024-02-19T16:28:38.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 -- start start 2024-02-19T16:28:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e7f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 0x55a07e800210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e802b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e7b2960 con 0x55a07e7f7b70 2024-02-19T16:28:38.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e766310 con 0x55a07e800750 2024-02-19T16:28:38.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48abd82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e763840 con 0x55a07e7f7250 2024-02-19T16:28:38.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 0x55a07e800210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 0x55a07e800210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47468/0 (socket says 172.21.15.8:47468) 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1506777539 learned_addr learned my addr 172.21.15.8:0/1506777539 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a359e640 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e7f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a259c640 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e802b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f488bfff640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f488c0a2260 0x7f488c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a359e640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f488c0a2260 0x7f488c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1506777539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 msgr2=0x55a07e7f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e7f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1506777539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 msgr2=0x55a07e802b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.899+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e802b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1506777539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07e721060 con 0x55a07e7f7b70 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48a359e640 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e7f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f488bfff640 1 -- 172.21.15.8:0/1614885326 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4898014070 con 0x7f48a41056a0 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 0x55a07e800210 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4894036040 tx=0x7f489402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392fdf89073a768f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48a359e640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f488c0a2260 0x7f488c0a4720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fd0 tx=0x7f489c004ab0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48897fa640 1 -- 172.21.15.8:0/1506777539 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489402c7e0 con 0x55a07e7f7b70 2024-02-19T16:28:38.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48897fa640 1 -- 172.21.15.8:0/1506777539 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4894004510 con 0x55a07e7f7b70 2024-02-19T16:28:38.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.900+0000 7f48897fa640 1 -- 172.21.15.8:0/1506777539 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894005440 con 0x55a07e7f7b70 2024-02-19T16:28:38.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- 172.21.15.8:0/1506777539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 msgr2=0x55a07e800210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 0x55a07e800210 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4894036040 tx=0x7f489402c660 comp rx=0 tx=0).stop 2024-02-19T16:28:38.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- 172.21.15.8:0/1506777539 shutdown_connections 2024-02-19T16:28:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e802b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7b70 0x55a07e800210 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- 172.21.15.8:0/1506777539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e7f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- 172.21.15.8:0/1506777539 >> 172.21.15.8:0/1506777539 conn(0x55a07e7f1980 msgr2=0x55a07e7f1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- 172.21.15.8:0/1506777539 shutdown_connections 2024-02-19T16:28:38.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- 172.21.15.8:0/1506777539 wait complete. 2024-02-19T16:28:38.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- start start 2024-02-19T16:28:38.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 Processor -- start 2024-02-19T16:28:38.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e7fdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e800750 0x55a07e7fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e75f020 con 0x55a07e7f7250 2024-02-19T16:28:38.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e432430 con 0x55a07e7f7b70 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e3eee00 con 0x55a07e800750 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48abd82740 10 client.0 _ll_register_callbacks cb 0x55a07e706cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7fd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47480/0 (socket says 172.21.15.8:47480) 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1166291915 learned_addr learned my addr 172.21.15.8:0/1166291915 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48a259c640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e7fdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.901+0000 7f48a359e640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e800750 0x55a07e7fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e800750 msgr2=0x55a07e7fe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e800750 0x55a07e7fe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1166291915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 msgr2=0x55a07e7fdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e7fdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48a2d9d640 1 -- 172.21.15.8:0/1166291915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a410da80 con 0x55a07e7f7250 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7fd7a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f489402c170 tx=0x7f4894004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a02332d250c1c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f4886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:38.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f4886ff5640 1 -- 172.21.15.8:0/1166291915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894005610 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f4886ff5640 1 -- 172.21.15.8:0/1166291915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f489404d070 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a07e45e7e0 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f4886ff5640 1 -- 172.21.15.8:0/1166291915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489403f440 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48abd82740 10 client.16896 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a07e75d6a0 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f4886ff5640 1 -- 172.21.15.8:0/1166291915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a07e45e7e0 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.902+0000 7f4886ff5640 1 -- 172.21.15.8:0/1166291915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f489403c050 con 0x55a07e7f7250 2024-02-19T16:28:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48abd82740 10 client.16896 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48abd82740 10 client.16896 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48abd82740 20 client.16896 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48abd82740 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e810780 0x55a07e812b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48abd82740 4 client.16896 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a07e49b720 con 0x55a07e810780 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48a259c640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e810780 0x55a07e812b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f4886ff5640 10 client.16896 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.903+0000 7f48a259c640 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e810780 0x55a07e812b70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f906191c1aa067aa server_cookie=89abc82c3318b97a in_seq=0 out_seq=0 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f4886ff5640 1 -- 172.21.15.8:0/1166291915 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a07e49b720 con 0x55a07e810780 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f4886ff5640 10 client.16896 handle_command_reply: tid=0 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 client.16896 shutdown 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 msgr2=0x55a07e7fd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7fd7a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f489402c170 tx=0x7f4894004fe0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 shutdown_connections 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e810780 0x55a07e812b70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e800750 0x55a07e7fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e7fdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f4886ff5640 0 client.16896 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 --2- 172.21.15.8:0/1166291915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7fd7a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 >> 172.21.15.8:0/1166291915 conn(0x55a07e7f1980 msgr2=0x55a07e7f36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 shutdown_connections 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 1 -- 172.21.15.8:0/1166291915 wait complete. 2024-02-19T16:28:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.904+0000 7f48abd82740 20 client.16896 trim_cache size 0 max 16384 2024-02-19T16:28:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 Processor -- start 2024-02-19T16:28:38.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 -- start start 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7b70 0x55a07e7fe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e80ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e6e4130 con 0x55a07e7f7250 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e7fbcf0 con 0x55a07e800750 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48abd82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e7fc6e0 con 0x55a07e7f7b70 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.916+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47482/0 (socket says 172.21.15.8:47482) 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 -- 172.21.15.8:0/824994761 learned_addr learned my addr 172.21.15.8:0/824994761 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a259c640 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7b70 0x55a07e7fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a359e640 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e80ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 -- 172.21.15.8:0/824994761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7b70 msgr2=0x55a07e7fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7b70 0x55a07e7fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 -- 172.21.15.8:0/824994761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 msgr2=0x55a07e80ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e80ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 -- 172.21.15.8:0/824994761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07e6f9c20 con 0x55a07e7f7250 2024-02-19T16:28:38.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7ffed0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4894009800 tx=0x7f489402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b0876a9856651a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f4888ff9640 1 -- 172.21.15.8:0/824994761 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894043070 con 0x55a07e7f7250 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f4888ff9640 1 -- 172.21.15.8:0/824994761 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4894034a60 con 0x55a07e7f7250 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.917+0000 7f4888ff9640 1 -- 172.21.15.8:0/824994761 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489403fb80 con 0x55a07e7f7250 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- 172.21.15.8:0/824994761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 msgr2=0x55a07e7ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7ffed0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4894009800 tx=0x7f489402c660 comp rx=0 tx=0).stop 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- 172.21.15.8:0/824994761 shutdown_connections 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e800750 0x55a07e80ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7b70 0x55a07e7fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- 172.21.15.8:0/824994761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e7f7250 0x55a07e7ffed0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- 172.21.15.8:0/824994761 >> 172.21.15.8:0/824994761 conn(0x55a07e7f1980 msgr2=0x55a07e7f0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- 172.21.15.8:0/824994761 shutdown_connections 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- 172.21.15.8:0/824994761 wait complete. 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- start start 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 Processor -- start 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e804080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e8045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 0x55a07e86b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e804d20 con 0x55a07e800750 2024-02-19T16:28:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e86b580 con 0x55a07e7f7b70 2024-02-19T16:28:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07e86b740 con 0x55a07e7f7250 2024-02-19T16:28:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48abd82740 10 client.0 _ll_register_callbacks cb 0x7f489809ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48a359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 0x55a07e86b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48a359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 0x55a07e86b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47490/0 (socket says 172.21.15.8:47490) 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48a359e640 1 -- 172.21.15.8:0/4043627723 learned_addr learned my addr 172.21.15.8:0/4043627723 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48a259c640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e8045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.918+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e804080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48a359e640 1 -- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 msgr2=0x55a07e804080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48a359e640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e804080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48a359e640 1 -- 172.21.15.8:0/4043627723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 msgr2=0x55a07e8045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48a359e640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e8045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48a359e640 1 -- 172.21.15.8:0/4043627723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a07e75d6a0 con 0x55a07e800750 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48a359e640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 0x55a07e86b040 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f489c003bf0 tx=0x7f489c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e45bf27b11fe6e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f4886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f4886ff5640 1 -- 172.21.15.8:0/4043627723 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c027070 con 0x55a07e800750 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f4886ff5640 1 -- 172.21.15.8:0/4043627723 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f489c004250 con 0x55a07e800750 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4894004510 con 0x55a07e800750 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f4886ff5640 1 -- 172.21.15.8:0/4043627723 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489c004410 con 0x55a07e800750 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 10 client.16902 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:38.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a07e71f8c0 con 0x55a07e800750 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f4886ff5640 1 -- 172.21.15.8:0/4043627723 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4894004510 con 0x55a07e800750 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f4886ff5640 1 -- 172.21.15.8:0/4043627723 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f489c021c20 con 0x55a07e800750 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 10 client.16902 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 10 client.16902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 20 client.16902 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e86d040 0x55a07e86f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 4 client.16902 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.919+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a07e4ea890 con 0x55a07e86d040 2024-02-19T16:28:38.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e86d040 0x55a07e86f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48a2d9d640 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e86d040 0x55a07e86f430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cd2431dad20825c server_cookie=ef177de122384d6c in_seq=0 out_seq=0 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f4886ff5640 10 client.16902 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f4886ff5640 1 -- 172.21.15.8:0/4043627723 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a07e4ea890 con 0x55a07e86d040 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f4886ff5640 10 client.16902 handle_command_reply: tid=0 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 client.16902 shutdown 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 msgr2=0x55a07e86b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 0x55a07e86b040 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f489c003bf0 tx=0x7f489c004a40 comp rx=0 tx=0).stop 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 shutdown_connections 2024-02-19T16:28:38.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55a07e86d040 0x55a07e86f430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a07e800750 0x55a07e86b040 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a07e7f7b70 0x55a07e8045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 --2- 172.21.15.8:0/4043627723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a07e7f7250 0x55a07e804080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f4886ff5640 0 client.16902 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 >> 172.21.15.8:0/4043627723 conn(0x55a07e7f1980 msgr2=0x55a07e817470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 shutdown_connections 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 1 -- 172.21.15.8:0/4043627723 wait complete. 2024-02-19T16:28:38.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.920+0000 7f48abd82740 20 client.16902 trim_cache size 0 max 16384 2024-02-19T16:28:38.944 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:28:38.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f488c0a2260 msgr2=0x7f488c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f488c0a2260 0x7f488c0a4720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fd0 tx=0x7f489c004ab0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 msgr2=0x7f48a41025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 0x7f48a41025a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9a0 tx=0x7f489800ee60 comp rx=0 tx=0).stop 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 shutdown_connections 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f488c0a2260 0x7f488c0a4720 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48a4102ae0 0x7f48a41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48a41056a0 0x7f48a41025a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 --2- 172.21.15.8:0/1614885326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a41005d0 0x7f48a4102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 >> 172.21.15.8:0/1614885326 conn(0x7f48a40774f0 msgr2=0x7f48a40689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:38.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 shutdown_connections 2024-02-19T16:28:38.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:38.921+0000 7f48a91ae640 1 -- 172.21.15.8:0/1614885326 wait complete. 2024-02-19T16:28:38.947 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:28:38.947 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:28:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 -- 172.21.15.8:0/3059728515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58100500 msgr2=0x7fde5810c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 --2- 172.21.15.8:0/3059728515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58100500 0x7fde5810c9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fde50009ca0 tx=0x7fde5002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 -- 172.21.15.8:0/3059728515 shutdown_connections 2024-02-19T16:28:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 --2- 172.21.15.8:0/3059728515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58100500 0x7fde5810c9c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 --2- 172.21.15.8:0/3059728515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde580ffb40 0x7fde580fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 --2- 172.21.15.8:0/3059728515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde58105c00 0x7fde58105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 -- 172.21.15.8:0/3059728515 >> 172.21.15.8:0/3059728515 conn(0x7fde580769b0 msgr2=0x7fde580fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 -- 172.21.15.8:0/3059728515 shutdown_connections 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 -- 172.21.15.8:0/3059728515 wait complete. 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 Processor -- start 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.051+0000 7fde5fde4640 1 -- start start 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde580ffb40 0x7fde5819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde58100500 0x7fde5819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 0x7fde58196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58196e20 con 0x7fde58105c00 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58196f90 con 0x7fde58100500 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde58197150 con 0x7fde580ffb40 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 0x7fde58196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 0x7fde58196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47500/0 (socket says 172.21.15.8:47500) 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 -- 172.21.15.8:0/2971009857 learned_addr learned my addr 172.21.15.8:0/2971009857 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5db80640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde580ffb40 0x7fde5819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5d37f640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde58100500 0x7fde5819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 -- 172.21.15.8:0/2971009857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde580ffb40 msgr2=0x7fde5819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde580ffb40 0x7fde5819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 -- 172.21.15.8:0/2971009857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde58100500 msgr2=0x7fde5819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde58100500 0x7fde5819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 -- 172.21.15.8:0/2971009857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48009490 con 0x7fde58105c00 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5e381640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 0x7fde58196780 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fde50005e30 tx=0x7fde50035aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde4effd640 1 -- 172.21.15.8:0/2971009857 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50035ea0 con 0x7fde58105c00 2024-02-19T16:28:39.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde4effd640 1 -- 172.21.15.8:0/2971009857 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde50003e50 con 0x7fde58105c00 2024-02-19T16:28:39.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde54009490 con 0x7fde58105c00 2024-02-19T16:28:39.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde4effd640 1 -- 172.21.15.8:0/2971009857 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50004830 con 0x7fde58105c00 2024-02-19T16:28:39.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.052+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde50009900 con 0x7fde58105c00 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde4effd640 1 -- 172.21.15.8:0/2971009857 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fde5003b040 con 0x7fde58105c00 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 Processor -- start 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 -- start start 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbeda1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde4effd640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fde440a2230 0x7fde440a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedaed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbed5c960 con 0x55ccbeda1600 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbed10310 con 0x55ccbeda0c50 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.055+0000 7fde629b6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbed0d840 con 0x55ccbedac960 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5d37f640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fde440a2230 0x7fde440a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47512/0 (socket says 172.21.15.8:47512) 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5e381640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedaed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 -- 172.21.15.8:0/2715783749 learned_addr learned my addr 172.21.15.8:0/2715783749 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde4effd640 1 -- 172.21.15.8:0/2971009857 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fde500f0c90 con 0x7fde58105c00 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5d37f640 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbeda1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 -- 172.21.15.8:0/2715783749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 msgr2=0x55ccbedaed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedaed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 -- 172.21.15.8:0/2715783749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 msgr2=0x55ccbeda1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbeda1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 -- 172.21.15.8:0/2715783749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccbeccb060 con 0x55ccbeda1600 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5db80640 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedac420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde54009e80 tx=0x7fde54011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6526d0b81cf9da server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde5d37f640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fde440a2230 0x7fde440a46f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde58104ae0 tx=0x7fde48009c50 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde43fff640 1 -- 172.21.15.8:0/2715783749 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54019630 con 0x55ccbeda1600 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde43fff640 1 -- 172.21.15.8:0/2715783749 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde54019c70 con 0x55ccbeda1600 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.056+0000 7fde43fff640 1 -- 172.21.15.8:0/2715783749 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5400ebd0 con 0x55ccbeda1600 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 -- 172.21.15.8:0/2715783749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 msgr2=0x55ccbedac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedac420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde54009e80 tx=0x7fde54011810 comp rx=0 tx=0).stop 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 -- 172.21.15.8:0/2715783749 shutdown_connections 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedaed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedac420 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 --2- 172.21.15.8:0/2715783749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbeda1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 -- 172.21.15.8:0/2715783749 >> 172.21.15.8:0/2715783749 conn(0x55ccbed9b980 msgr2=0x55ccbed9bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 -- 172.21.15.8:0/2715783749 shutdown_connections 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 -- 172.21.15.8:0/2715783749 wait complete. 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 -- start start 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.057+0000 7fde629b6740 1 Processor -- start 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 0x55ccbeda9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbedac960 0x55ccbedaa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbed09020 con 0x55ccbeda0c50 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbe9dc430 con 0x55ccbedac960 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbe998e00 con 0x55ccbeda1600 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde629b6740 10 client.0 _ll_register_callbacks cb 0x55ccbecb0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 0x55ccbeda9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 0x55ccbeda9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47522/0 (socket says 172.21.15.8:47522) 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 -- 172.21.15.8:0/255823352 learned_addr learned my addr 172.21.15.8:0/255823352 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5d37f640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5e381640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbedac960 0x55ccbedaa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 -- 172.21.15.8:0/255823352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda1600 msgr2=0x55ccbeda9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 -- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbedac960 msgr2=0x55ccbedaa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbedac960 0x55ccbedaa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 -- 172.21.15.8:0/255823352 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde54009ae0 con 0x55ccbeda0c50 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde5db80640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 0x55ccbeda9920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fde54000c00 tx=0x7fde54005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b055efa1f279ece5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.058+0000 7fde417fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:39.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde417fa640 1 -- 172.21.15.8:0/255823352 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54005040 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccbea087e0 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde417fa640 1 -- 172.21.15.8:0/255823352 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fde540051a0 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde417fa640 1 -- 172.21.15.8:0/255823352 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde54005360 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 10 client.16914 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ccbed076a0 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde417fa640 1 -- 172.21.15.8:0/255823352 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ccbea087e0 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde417fa640 1 -- 172.21.15.8:0/255823352 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fde54004930 con 0x55ccbeda0c50 2024-02-19T16:28:39.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 10 client.16914 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 20 client.16914 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbedbcc70 0x55ccbedbf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 4 client.16914 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccbea45720 con 0x55ccbedbcc70 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.059+0000 7fde5d37f640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbedbcc70 0x55ccbedbf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.060+0000 7fde5d37f640 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbedbcc70 0x55ccbedbf0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b53130ca8c4ef9 server_cookie=1182c1a278e27416 in_seq=0 out_seq=0 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.060+0000 7fde417fa640 10 client.16914 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:39.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde417fa640 1 -- 172.21.15.8:0/255823352 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ccbea45720 con 0x55ccbedbcc70 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde417fa640 10 client.16914 handle_command_reply: tid=0 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 client.16914 shutdown 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 msgr2=0x55ccbeda9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 0x55ccbeda9920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fde54000c00 tx=0x7fde54005dc0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 shutdown_connections 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbedbcc70 0x55ccbedbf0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbedac960 0x55ccbedaa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde417fa640 0 client.16914 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:39.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 --2- 172.21.15.8:0/255823352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda0c50 0x55ccbeda9920 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 >> 172.21.15.8:0/255823352 conn(0x55ccbed9b980 msgr2=0x55ccbed9d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 shutdown_connections 2024-02-19T16:28:39.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 1 -- 172.21.15.8:0/255823352 wait complete. 2024-02-19T16:28:39.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.061+0000 7fde629b6740 20 client.16914 trim_cache size 0 max 16384 2024-02-19T16:28:39.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 Processor -- start 2024-02-19T16:28:39.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 -- start start 2024-02-19T16:28:39.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 0x55ccbed9a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbedac960 0x55ccbedaa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbec8e130 con 0x55ccbedac960 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbedaa8f0 con 0x55ccbeda1600 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.073+0000 7fde629b6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbeda5550 con 0x55ccbeda0c50 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 0x55ccbed9a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5e381640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbedac960 0x55ccbedaa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 0x55ccbed9a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51532/0 (socket says 172.21.15.8:51532) 2024-02-19T16:28:39.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 -- 172.21.15.8:0/1577324351 learned_addr learned my addr 172.21.15.8:0/1577324351 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5d37f640 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 -- 172.21.15.8:0/1577324351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda1600 msgr2=0x55ccbeda9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 -- 172.21.15.8:0/1577324351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbedac960 msgr2=0x55ccbedaa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbedac960 0x55ccbedaa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 -- 172.21.15.8:0/1577324351 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccbeca3c20 con 0x55ccbeda0c50 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5e381640 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbedac960 0x55ccbedaa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:39.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde5db80640 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 0x55ccbed9a2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde54011d10 tx=0x7fde54004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db6d386a518a0c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde437fe640 1 -- 172.21.15.8:0/1577324351 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5400eda0 con 0x55ccbeda0c50 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde437fe640 1 -- 172.21.15.8:0/1577324351 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde54005520 con 0x55ccbeda0c50 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.074+0000 7fde437fe640 1 -- 172.21.15.8:0/1577324351 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54019770 con 0x55ccbeda0c50 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 -- 172.21.15.8:0/1577324351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 msgr2=0x55ccbed9a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 0x55ccbed9a2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde54011d10 tx=0x7fde54004e60 comp rx=0 tx=0).stop 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 -- 172.21.15.8:0/1577324351 shutdown_connections 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbedac960 0x55ccbedaa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda1600 0x55ccbeda9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 --2- 172.21.15.8:0/1577324351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbeda0c50 0x55ccbed9a2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 -- 172.21.15.8:0/1577324351 >> 172.21.15.8:0/1577324351 conn(0x55ccbed9b980 msgr2=0x55ccbedc4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 -- 172.21.15.8:0/1577324351 shutdown_connections 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 -- 172.21.15.8:0/1577324351 wait complete. 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 -- start start 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.075+0000 7fde629b6740 1 Processor -- start 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbedb3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedb4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedb03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbedb0a30 con 0x55ccbeda1600 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbedb0ba0 con 0x55ccbeda0c50 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbedb0d60 con 0x55ccbedac960 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 10 client.0 _ll_register_callbacks cb 0x7fde5401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedb4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedb4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47548/0 (socket says 172.21.15.8:47548) 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 -- 172.21.15.8:0/4149402343 learned_addr learned my addr 172.21.15.8:0/4149402343 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5db80640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbedb3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5e381640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedb03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 -- 172.21.15.8:0/4149402343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 msgr2=0x55ccbedb03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedb03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 -- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 msgr2=0x55ccbedb3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbedb3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 -- 172.21.15.8:0/4149402343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde50009900 con 0x55ccbeda1600 2024-02-19T16:28:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde5d37f640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedb4440 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55ccbedbf920 tx=0x7fde48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac7b81270ed09f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde417fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde417fa640 1 -- 172.21.15.8:0/4149402343 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48025040 con 0x55ccbeda1600 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde54019c70 con 0x55ccbeda1600 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.076+0000 7fde417fa640 1 -- 172.21.15.8:0/4149402343 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7fde48004480 con 0x55ccbeda1600 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde417fa640 1 -- 172.21.15.8:0/4149402343 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde480113d0 con 0x55ccbeda1600 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 10 client.16920 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ccbed076a0 con 0x55ccbeda1600 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde417fa640 1 -- 172.21.15.8:0/4149402343 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde54019c70 con 0x55ccbeda1600 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde417fa640 1 -- 172.21.15.8:0/4149402343 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fde48003890 con 0x55ccbeda1600 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 10 client.16920 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 10 client.16920 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 20 client.16920 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbee196b0 0x55ccbee1bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 4 client.16920 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:28:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ccbea94890 con 0x55ccbee196b0 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde5db80640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbee196b0 0x55ccbee1bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde417fa640 10 client.16920 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.077+0000 7fde5db80640 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbee196b0 0x55ccbee1bad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acb2f0087c907a4b server_cookie=caafd8f512dd7637 in_seq=0 out_seq=0 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde417fa640 1 -- 172.21.15.8:0/4149402343 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ccbea94890 con 0x55ccbee196b0 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde417fa640 10 client.16920 handle_command_reply: tid=0 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 client.16920 shutdown 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 msgr2=0x55ccbedb4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedb4440 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55ccbedbf920 tx=0x7fde48005f90 comp rx=0 tx=0).stop 2024-02-19T16:28:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 shutdown_connections 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ccbee196b0 0x55ccbee1bad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ccbedac960 0x55ccbedb03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ccbeda1600 0x55ccbedb4440 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 --2- 172.21.15.8:0/4149402343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccbeda0c50 0x55ccbedb3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde417fa640 0 client.16920 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 >> 172.21.15.8:0/4149402343 conn(0x55ccbed9b980 msgr2=0x55ccbed9c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 shutdown_connections 2024-02-19T16:28:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 1 -- 172.21.15.8:0/4149402343 wait complete. 2024-02-19T16:28:39.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde629b6740 20 client.16920 trim_cache size 0 max 16384 2024-02-19T16:28:39.094 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:28:39.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fde440a2230 msgr2=0x7fde440a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fde440a2230 0x7fde440a46f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde58104ae0 tx=0x7fde48009c50 comp rx=0 tx=0).stop 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 msgr2=0x7fde58196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 0x7fde58196780 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fde50005e30 tx=0x7fde50035aa0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 shutdown_connections 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde58105c00 0x7fde58196780 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde58100500 0x7fde5819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fde440a2230 0x7fde440a46f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 --2- 172.21.15.8:0/2971009857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde580ffb40 0x7fde5819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.078+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 >> 172.21.15.8:0/2971009857 conn(0x7fde580769b0 msgr2=0x7fde581016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.079+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 shutdown_connections 2024-02-19T16:28:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.079+0000 7fde5fde4640 1 -- 172.21.15.8:0/2971009857 wait complete. 2024-02-19T16:28:39.098 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:28:39.098 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:28:39.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3462332370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381014e0 msgr2=0x7f4e3810d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3462332370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381014e0 0x7f4e3810d210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009c10 tx=0x7f4e3002bf30 comp rx=0 tx=0).stop 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3462332370 shutdown_connections 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3462332370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381014e0 0x7f4e3810d210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3462332370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e38100b40 0x7f4e38100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3462332370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e381053a0 0x7f4e38105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3462332370 >> 172.21.15.8:0/3462332370 conn(0x7f4e380769b0 msgr2=0x7f4e380fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3462332370 shutdown_connections 2024-02-19T16:28:39.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3462332370 wait complete. 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.201+0000 7f4e3fe12640 1 Processor -- start 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 -- start start 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 0x7f4e3819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 0x7f4e3819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381053a0 0x7f4e381979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38198010 con 0x7f4e381014e0 2024-02-19T16:28:39.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38198180 con 0x7f4e38100b40 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3fe12640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38198340 con 0x7f4e381053a0 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3d3ad640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 0x7f4e3819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 0x7f4e3819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381053a0 0x7f4e381979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 0x7f4e3819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:59062/0 (socket says 172.21.15.8:59062) 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3d3ad640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 0x7f4e3819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47560/0 (socket says 172.21.15.8:47560) 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3dbae640 1 -- 172.21.15.8:0/3106358855 learned_addr learned my addr 172.21.15.8:0/3106358855 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 -- 172.21.15.8:0/3106358855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 msgr2=0x7f4e3819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 0x7f4e3819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 -- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 msgr2=0x7f4e3819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 0x7f4e3819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 -- 172.21.15.8:0/3106358855 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e28009560 con 0x7f4e381053a0 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 0x7f4e3819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 0x7f4e3819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.202+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381053a0 0x7f4e381979c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e3002c470 tx=0x7f4e300335f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.203+0000 7f4e2effd640 1 -- 172.21.15.8:0/3106358855 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e300337e0 con 0x7f4e381053a0 2024-02-19T16:28:39.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.203+0000 7f4e2effd640 1 -- 172.21.15.8:0/3106358855 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e30003b80 con 0x7f4e381053a0 2024-02-19T16:28:39.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.203+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e30009840 con 0x7f4e381053a0 2024-02-19T16:28:39.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.203+0000 7f4e2effd640 1 -- 172.21.15.8:0/3106358855 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e300347b0 con 0x7f4e381053a0 2024-02-19T16:28:39.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.203+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e381a4680 con 0x7f4e381053a0 2024-02-19T16:28:39.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.205+0000 7f4e2effd640 1 -- 172.21.15.8:0/3106358855 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4e3002dd50 con 0x7f4e381053a0 2024-02-19T16:28:39.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.205+0000 7f4e429e2740 1 Processor -- start 2024-02-19T16:28:39.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.205+0000 7f4e429e2740 1 -- start start 2024-02-19T16:28:39.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e2effd640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e240a2230 0x7f4e240a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f4505630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f4510b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e429e2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f44c0960 con 0x5594f4505b70 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e429e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f4474310 con 0x5594f450e750 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e429e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f4471840 con 0x5594f4505250 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e240a2230 0x7f4e240a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e2effd640 1 -- 172.21.15.8:0/3106358855 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4e300f0800 con 0x7f4e381053a0 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f4505630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f4505630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51570/0 (socket says 172.21.15.8:51570) 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3e3af640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f4510b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 -- 172.21.15.8:0/4217934150 learned_addr learned my addr 172.21.15.8:0/4217934150 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 -- 172.21.15.8:0/4217934150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 msgr2=0x5594f4510b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f4510b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 -- 172.21.15.8:0/4217934150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 msgr2=0x5594f450e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 -- 172.21.15.8:0/4217934150 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594f442f060 con 0x5594f4505250 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f4510b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:39.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e240a2230 0x7f4e240a46f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009fd0 tx=0x7f4e28011570 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f4505630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e340115f0 tx=0x7f4e34011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe9b0f73f1beaad4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.206+0000 7f4e23fff640 1 -- 172.21.15.8:0/4217934150 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34005020 con 0x5594f4505250 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e23fff640 1 -- 172.21.15.8:0/4217934150 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e34005660 con 0x5594f4505250 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e23fff640 1 -- 172.21.15.8:0/4217934150 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400f920 con 0x5594f4505250 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 -- 172.21.15.8:0/4217934150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 msgr2=0x5594f4505630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f4505630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e340115f0 tx=0x7f4e34011ab0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 -- 172.21.15.8:0/4217934150 shutdown_connections 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f4510b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 --2- 172.21.15.8:0/4217934150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f4505630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 -- 172.21.15.8:0/4217934150 >> 172.21.15.8:0/4217934150 conn(0x5594f44ff980 msgr2=0x5594f44ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 -- 172.21.15.8:0/4217934150 shutdown_connections 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.207+0000 7f4e429e2740 1 -- 172.21.15.8:0/4217934150 wait complete. 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 -- start start 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 Processor -- start 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f450b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f450c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f446d020 con 0x5594f4505b70 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f4140430 con 0x5594f450e750 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f40fce00 con 0x5594f4505250 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e429e2740 10 client.0 _ll_register_callbacks cb 0x5594f4414cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47580/0 (socket says 172.21.15.8:47580) 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f450b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 -- 172.21.15.8:0/207286005 learned_addr learned my addr 172.21.15.8:0/207286005 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f450c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 -- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 msgr2=0x5594f450b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f450b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 -- 172.21.15.8:0/207286005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 msgr2=0x5594f450c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f450c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3d3ad640 1 -- 172.21.15.8:0/207286005 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e34009dc0 con 0x5594f4505b70 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.208+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f450b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450bcb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4e28005fd0 tx=0x7f4e2801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a189f20248b9ad77 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e217fa640 1 -- 172.21.15.8:0/207286005 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28016040 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e217fa640 1 -- 172.21.15.8:0/207286005 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4e2801f450 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594f416c7e0 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e217fa640 1 -- 172.21.15.8:0/207286005 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e2801f610 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 10 client.16932 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5594f446b6a0 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e217fa640 1 -- 172.21.15.8:0/207286005 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5594f416c7e0 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e217fa640 1 -- 172.21.15.8:0/207286005 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e28026030 con 0x5594f4505b70 2024-02-19T16:28:39.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 10 client.16932 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 20 client.16932 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f451ec20 0x5594f4521040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 4 client.16932 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.209+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594f41a9720 con 0x5594f451ec20 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.210+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f451ec20 0x5594f4521040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.210+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f451ec20 0x5594f4521040 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1331f5299ce56fd6 server_cookie=9517bb763dcf4d in_seq=0 out_seq=0 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.210+0000 7f4e217fa640 10 client.16932 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:39.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e217fa640 1 -- 172.21.15.8:0/207286005 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5594f41a9720 con 0x5594f451ec20 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e217fa640 10 client.16932 handle_command_reply: tid=0 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 client.16932 shutdown 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 msgr2=0x5594f450bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450bcb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4e28005fd0 tx=0x7f4e2801d000 comp rx=0 tx=0).stop 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 shutdown_connections 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f450e750 0x5594f450c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505b70 0x5594f450bcb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f451ec20 0x5594f4521040 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 --2- 172.21.15.8:0/207286005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505250 0x5594f450b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e217fa640 0 client.16932 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:39.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 >> 172.21.15.8:0/207286005 conn(0x5594f44ff980 msgr2=0x5594f45016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 shutdown_connections 2024-02-19T16:28:39.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 1 -- 172.21.15.8:0/207286005 wait complete. 2024-02-19T16:28:39.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.211+0000 7f4e429e2740 20 client.16932 trim_cache size 0 max 16384 2024-02-19T16:28:39.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.223+0000 7f4e429e2740 1 Processor -- start 2024-02-19T16:28:39.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.223+0000 7f4e429e2740 1 -- start start 2024-02-19T16:28:39.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.223+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 0x5594f450c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.223+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505b70 0x5594f45012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f450e750 0x5594f4501800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e429e2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f43f2130 con 0x5594f4505250 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e429e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f451b640 con 0x5594f4505b70 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e429e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f45216a0 con 0x5594f450e750 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 0x5594f450c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 0x5594f450c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47582/0 (socket says 172.21.15.8:47582) 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 -- 172.21.15.8:0/2278296190 learned_addr learned my addr 172.21.15.8:0/2278296190 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505b70 0x5594f45012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f450e750 0x5594f4501800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 -- 172.21.15.8:0/2278296190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f450e750 msgr2=0x5594f4501800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f450e750 0x5594f4501800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 -- 172.21.15.8:0/2278296190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505b70 msgr2=0x5594f45012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505b70 0x5594f45012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 -- 172.21.15.8:0/2278296190 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594f4407c20 con 0x5594f4505250 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 0x5594f450c010 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4e34004760 tx=0x7f4e34004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c9a33e10d54444 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e237fe640 1 -- 172.21.15.8:0/2278296190 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34027070 con 0x5594f4505250 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e237fe640 1 -- 172.21.15.8:0/2278296190 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e34009c30 con 0x5594f4505250 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.224+0000 7f4e237fe640 1 -- 172.21.15.8:0/2278296190 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400fb00 con 0x5594f4505250 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 -- 172.21.15.8:0/2278296190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 msgr2=0x5594f450c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 0x5594f450c010 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4e34004760 tx=0x7f4e34004d90 comp rx=0 tx=0).stop 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 -- 172.21.15.8:0/2278296190 shutdown_connections 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f450e750 0x5594f4501800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505b70 0x5594f45012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- 172.21.15.8:0/2278296190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f4505250 0x5594f450c010 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 -- 172.21.15.8:0/2278296190 >> 172.21.15.8:0/2278296190 conn(0x5594f44ff980 msgr2=0x5594f4500bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 -- 172.21.15.8:0/2278296190 shutdown_connections 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 -- 172.21.15.8:0/2278296190 wait complete. 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 -- start start 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 Processor -- start 2024-02-19T16:28:39.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505250 0x5594f45162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505b70 0x5594f4512640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.225+0000 7f4e429e2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 0x5594f4512b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f4513190 con 0x5594f450e750 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f4513300 con 0x5594f4505250 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f45134c0 con 0x5594f4505b70 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 10 client.0 _ll_register_callbacks cb 0x7f4e28011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 0x5594f4512b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 0x5594f4512b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47586/0 (socket says 172.21.15.8:47586) 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 -- 172.21.15.8:0/1456520811 learned_addr learned my addr 172.21.15.8:0/1456520811 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3d3ad640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505b70 0x5594f4512640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505250 0x5594f45162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 -- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505b70 msgr2=0x5594f4512640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505b70 0x5594f4512640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 -- 172.21.15.8:0/1456520811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505250 msgr2=0x5594f45162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505250 0x5594f45162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 -- 172.21.15.8:0/1456520811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594f442f060 con 0x5594f450e750 2024-02-19T16:28:39.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e3e3af640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 0x5594f4512b80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4e30005ee0 tx=0x7f4e300a0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f9f6cd8f8dcc3a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e217fa640 1 -- 172.21.15.8:0/1456520811 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e300c10a0 con 0x5594f450e750 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e217fa640 1 -- 172.21.15.8:0/1456520811 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4e300c56f0 con 0x5594f450e750 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e34005660 con 0x5594f450e750 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e217fa640 1 -- 172.21.15.8:0/1456520811 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e300a3170 con 0x5594f450e750 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 10 client.16938 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5594f442d8c0 con 0x5594f450e750 2024-02-19T16:28:39.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.226+0000 7f4e217fa640 1 -- 172.21.15.8:0/1456520811 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e34005660 con 0x5594f450e750 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e217fa640 1 -- 172.21.15.8:0/1456520811 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e3003c070 con 0x5594f450e750 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 10 client.16938 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 20 client.16938 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f457b8e0 0x5594f457dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 4 client.16938 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5594f41f8890 con 0x5594f457b8e0 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f457b8e0 0x5594f457dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:39.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e217fa640 10 client.16938 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e3dbae640 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f457b8e0 0x5594f457dd00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ec5e6418a337a server_cookie=b1cc3fa7e9a02221 in_seq=0 out_seq=0 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e217fa640 1 -- 172.21.15.8:0/1456520811 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5594f41f8890 con 0x5594f457b8e0 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e217fa640 10 client.16938 handle_command_reply: tid=0 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 1 client.16938 shutdown 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 msgr2=0x5594f4512b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.227+0000 7f4e429e2740 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 0x5594f4512b80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4e30005ee0 tx=0x7f4e300a0d20 comp rx=0 tx=0).stop 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 shutdown_connections 2024-02-19T16:28:39.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5594f450e750 0x5594f4512b80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5594f457b8e0 0x5594f457dd00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594f4505b70 0x5594f4512640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 --2- 172.21.15.8:0/1456520811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594f4505250 0x5594f45162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e217fa640 0 client.16938 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 >> 172.21.15.8:0/1456520811 conn(0x5594f44ff980 msgr2=0x5594f4501360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 shutdown_connections 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 1 -- 172.21.15.8:0/1456520811 wait complete. 2024-02-19T16:28:39.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e429e2740 20 client.16938 trim_cache size 0 max 16384 2024-02-19T16:28:39.242 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:28:39.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e240a2230 msgr2=0x7f4e240a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e240a2230 0x7f4e240a46f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009fd0 tx=0x7f4e28011570 comp rx=0 tx=0).stop 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381053a0 msgr2=0x7f4e381979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381053a0 0x7f4e381979c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e3002c470 tx=0x7f4e300335f0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 shutdown_connections 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e240a2230 0x7f4e240a46f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e381053a0 0x7f4e381979c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e381014e0 0x7f4e3819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 --2- 172.21.15.8:0/3106358855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e38100b40 0x7f4e3819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 >> 172.21.15.8:0/3106358855 conn(0x7f4e380769b0 msgr2=0x7f4e38103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 shutdown_connections 2024-02-19T16:28:39.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:39.228+0000 7f4e3fe12640 1 -- 172.21.15.8:0/3106358855 wait complete. 2024-02-19T16:28:39.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2277317985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:28:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2277317985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:28:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2277317985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:28:40.246 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:28:40.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:28:40.246 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:28:40.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 -- 172.21.15.8:0/382094164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e74100f80 msgr2=0x7f4e74104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 --2- 172.21.15.8:0/382094164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e74100f80 0x7f4e74104fc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009930 tx=0x7f4e6802bc30 comp rx=0 tx=0).stop 2024-02-19T16:28:40.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 -- 172.21.15.8:0/382094164 shutdown_connections 2024-02-19T16:28:40.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 --2- 172.21.15.8:0/382094164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e741056a0 0x7f4e7410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 --2- 172.21.15.8:0/382094164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e74100f80 0x7f4e74104fc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 --2- 172.21.15.8:0/382094164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e741005d0 0x7f4e741009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 -- 172.21.15.8:0/382094164 >> 172.21.15.8:0/382094164 conn(0x7f4e740774f0 msgr2=0x7f4e74077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 -- 172.21.15.8:0/382094164 shutdown_connections 2024-02-19T16:28:40.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 -- 172.21.15.8:0/382094164 wait complete. 2024-02-19T16:28:40.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.364+0000 7f4e790ae640 1 Processor -- start 2024-02-19T16:28:40.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 -- start start 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e741005d0 0x7f4e7410db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e741056a0 0x7f4e7410e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 0x7f4e741a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741a62e0 con 0x7f4e7410e5d0 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741a6450 con 0x7f4e741056a0 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e790ae640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741a6610 con 0x7f4e741005d0 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 0x7f4e741a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 0x7f4e741a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47606/0 (socket says 172.21.15.8:47606) 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7359e640 1 -- 172.21.15.8:0/209624401 learned_addr learned my addr 172.21.15.8:0/209624401 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7259c640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e741056a0 0x7f4e7410e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e741005d0 0x7f4e7410db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7359e640 1 -- 172.21.15.8:0/209624401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e741005d0 msgr2=0x7f4e7410db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7359e640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e741005d0 0x7f4e7410db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.365+0000 7f4e7359e640 1 -- 172.21.15.8:0/209624401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e741056a0 msgr2=0x7f4e7410e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e7359e640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e741056a0 0x7f4e7410e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e7359e640 1 -- 172.21.15.8:0/209624401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009560 con 0x7f4e7410e5d0 2024-02-19T16:28:40.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e7359e640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 0x7f4e741a5d10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00eb10 tx=0x7f4e6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e5bfff640 1 -- 172.21.15.8:0/209624401 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c004b20 con 0x7f4e7410e5d0 2024-02-19T16:28:40.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e741a68f0 con 0x7f4e7410e5d0 2024-02-19T16:28:40.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e5bfff640 1 -- 172.21.15.8:0/209624401 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c004c80 con 0x7f4e7410e5d0 2024-02-19T16:28:40.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e5bfff640 1 -- 172.21.15.8:0/209624401 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c010650 con 0x7f4e7410e5d0 2024-02-19T16:28:40.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.366+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e741a6dd0 con 0x7f4e7410e5d0 2024-02-19T16:28:40.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.368+0000 7f4e5bfff640 1 -- 172.21.15.8:0/209624401 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4e6c005020 con 0x7f4e7410e5d0 2024-02-19T16:28:40.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 Processor -- start 2024-02-19T16:28:40.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 -- start start 2024-02-19T16:28:40.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a01e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a0272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a029d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560989f8cf90 con 0x56098a0278b0 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560989f8a4c0 con 0x56098a01e380 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560989f85ca0 con 0x56098a01ed30 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a0272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a029d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e5bfff640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e5c0a2260 0x7f4e5c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a029d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47608/0 (socket says 172.21.15.8:47608) 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a0272e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51614/0 (socket says 172.21.15.8:51614) 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7259c640 1 -- 172.21.15.8:0/1777845131 learned_addr learned my addr 172.21.15.8:0/1777845131 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7259c640 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a01e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e7359e640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e5c0a2260 0x7f4e5c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 -- 172.21.15.8:0/1777845131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 msgr2=0x56098a01e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a01e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e5bfff640 1 -- 172.21.15.8:0/209624401 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4e6c014070 con 0x7f4e7410e5d0 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 -- 172.21.15.8:0/1777845131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 msgr2=0x56098a029d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a029d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.369+0000 7f4e72d9d640 1 -- 172.21.15.8:0/1777845131 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560989f480b0 con 0x56098a01ed30 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.370+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a0272e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009c20 tx=0x7f4e64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d161d38b984539 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.370+0000 7f4e7359e640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e5c0a2260 0x7f4e5c0a4720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c09acb0 tx=0x7f4e6c085ae0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.370+0000 7f4e597fa640 1 -- 172.21.15.8:0/1777845131 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004050 con 0x56098a01ed30 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.370+0000 7f4e597fa640 1 -- 172.21.15.8:0/1777845131 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e64005500 con 0x56098a01ed30 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.370+0000 7f4e597fa640 1 -- 172.21.15.8:0/1777845131 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004c60 con 0x56098a01ed30 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.370+0000 7f4e7bd64740 1 -- 172.21.15.8:0/1777845131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 msgr2=0x56098a0272e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a0272e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009c20 tx=0x7f4e64011810 comp rx=0 tx=0).stop 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- 172.21.15.8:0/1777845131 shutdown_connections 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a029d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a0272e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/1777845131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a01e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- 172.21.15.8:0/1777845131 >> 172.21.15.8:0/1777845131 conn(0x56098a018840 msgr2=0x56098a018c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- 172.21.15.8:0/1777845131 shutdown_connections 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- 172.21.15.8:0/1777845131 wait complete. 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- start start 2024-02-19T16:28:40.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 Processor -- start 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a024810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a024d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a025290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560989f2af40 con 0x56098a0278b0 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560989c594b0 con 0x56098a01e380 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560989f0b670 con 0x56098a01ed30 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7bd64740 10 client.0 _ll_register_callbacks cb 0x560989f78f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e72d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a024d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a024810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a025290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e72d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a024d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51620/0 (socket says 172.21.15.8:51620) 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.371+0000 7f4e72d9d640 1 -- 172.21.15.8:0/3100218087 learned_addr learned my addr 172.21.15.8:0/3100218087 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7259c640 1 -- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 msgr2=0x56098a024d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7259c640 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a024d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7259c640 1 -- 172.21.15.8:0/3100218087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 msgr2=0x56098a024810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7259c640 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a024810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7259c640 1 -- 172.21.15.8:0/3100218087 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e64009850 con 0x56098a0278b0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a024d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7259c640 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a025290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802c780 tx=0x7f4e68005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a78abab26e4292 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e56ff5640 1 -- 172.21.15.8:0/3100218087 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68004310 con 0x56098a0278b0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e56ff5640 1 -- 172.21.15.8:0/3100218087 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4e68004470 con 0x56098a0278b0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560989f5ff40 con 0x56098a0278b0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e56ff5640 1 -- 172.21.15.8:0/3100218087 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e68004df0 con 0x56098a0278b0 2024-02-19T16:28:40.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7bd64740 10 client.16950 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560989f84320 con 0x56098a0278b0 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.372+0000 7f4e56ff5640 1 -- 172.21.15.8:0/3100218087 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560989f5ff40 con 0x56098a0278b0 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e56ff5640 1 -- 172.21.15.8:0/3100218087 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e68003950 con 0x56098a0278b0 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e7bd64740 10 client.16950 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e7bd64740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e7bd64740 20 client.16950 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a037d40 0x56098a03a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e7bd64740 4 client.16950 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:40.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560989f7c310 con 0x56098a037d40 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a037d40 0x56098a03a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e56ff5640 10 client.16950 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.373+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a037d40 0x56098a03a130 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce2467ee37d260e server_cookie=9e1da426df96a83 in_seq=0 out_seq=0 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e56ff5640 1 -- 172.21.15.8:0/3100218087 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560989f7c310 con 0x56098a037d40 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e56ff5640 10 client.16950 handle_command_reply: tid=0 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 client.16950 shutdown 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 msgr2=0x56098a025290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a025290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802c780 tx=0x7f4e68005d90 comp rx=0 tx=0).stop 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 shutdown_connections 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a0278b0 0x56098a025290 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a037d40 0x56098a03a130 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01ed30 0x56098a024d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/3100218087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01e380 0x56098a024810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e56ff5640 0 client.16950 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 >> 172.21.15.8:0/3100218087 conn(0x56098a018840 msgr2=0x56098a01a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 shutdown_connections 2024-02-19T16:28:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 1 -- 172.21.15.8:0/3100218087 wait complete. 2024-02-19T16:28:40.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.374+0000 7f4e7bd64740 20 client.16950 trim_cache size 0 max 16384 2024-02-19T16:28:40.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.390+0000 7f4e7bd64740 1 Processor -- start 2024-02-19T16:28:40.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 -- start start 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 0x56098a026f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a032db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01ed30 0x56098a0332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098a025770 con 0x56098a018840 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098a037b40 con 0x56098a01ed30 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098a033830 con 0x56098a01e380 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 0x56098a026f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 0x56098a026f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47614/0 (socket says 172.21.15.8:47614) 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 -- 172.21.15.8:0/506137398 learned_addr learned my addr 172.21.15.8:0/506137398 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a032db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01ed30 0x56098a0332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 -- 172.21.15.8:0/506137398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 msgr2=0x56098a032db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a032db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 -- 172.21.15.8:0/506137398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01ed30 msgr2=0x56098a0332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01ed30 0x56098a0332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 -- 172.21.15.8:0/506137398 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560989f20c00 con 0x56098a018840 2024-02-19T16:28:40.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e7259c640 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 0x56098a026f40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009430 tx=0x7f4e68001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2373e90a8ffc172 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e58ff9640 1 -- 172.21.15.8:0/506137398 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68004310 con 0x56098a018840 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e58ff9640 1 -- 172.21.15.8:0/506137398 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e68048070 con 0x56098a018840 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.391+0000 7f4e58ff9640 1 -- 172.21.15.8:0/506137398 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68005150 con 0x56098a018840 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 -- 172.21.15.8:0/506137398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 msgr2=0x56098a026f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 0x56098a026f40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009430 tx=0x7f4e68001e70 comp rx=0 tx=0).stop 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 -- 172.21.15.8:0/506137398 shutdown_connections 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a01ed30 0x56098a0332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a032db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/506137398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a018840 0x56098a026f40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 -- 172.21.15.8:0/506137398 >> 172.21.15.8:0/506137398 conn(0x56098a01f480 msgr2=0x56098a025060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 -- 172.21.15.8:0/506137398 shutdown_connections 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 -- 172.21.15.8:0/506137398 wait complete. 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 -- start start 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.392+0000 7f4e7bd64740 1 Processor -- start 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a018840 0x56098a02f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a021970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 0x56098a021eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098a0223f0 con 0x56098a01ed30 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098a022560 con 0x56098a018840 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098a022720 con 0x56098a01e380 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 10 client.0 _ll_register_callbacks cb 0x7f4e68005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 0x56098a021eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 0x56098a021eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47618/0 (socket says 172.21.15.8:47618) 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 -- 172.21.15.8:0/2388584334 learned_addr learned my addr 172.21.15.8:0/2388584334 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a021970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a018840 0x56098a02f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 -- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 msgr2=0x56098a021970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a021970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 -- 172.21.15.8:0/2388584334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a018840 msgr2=0x56098a02f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a018840 0x56098a02f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 -- 172.21.15.8:0/2388584334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560989f84320 con 0x56098a01ed30 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7259c640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 0x56098a021eb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802c950 tx=0x7f4e68004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beffd1b19e438490 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:40.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e56ff5640 1 -- 172.21.15.8:0/2388584334 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680043d0 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e56ff5640 1 -- 172.21.15.8:0/2388584334 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f4e680046c0 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.393+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560989fab900 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e56ff5640 1 -- 172.21.15.8:0/2388584334 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e68004880 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 10 client.16956 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560989f46910 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e56ff5640 1 -- 172.21.15.8:0/2388584334 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e6806c020 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e56ff5640 1 -- 172.21.15.8:0/2388584334 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e680052e0 con 0x56098a01ed30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 10 client.16956 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:40.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 20 client.16956 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a0278b0 0x56098a096530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 4 client.16956 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560989c85860 con 0x56098a0278b0 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a0278b0 0x56098a096530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e56ff5640 10 client.16956 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.394+0000 7f4e72d9d640 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a0278b0 0x56098a096530 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cec0570e3032774 server_cookie=8c7398b740922582 in_seq=0 out_seq=0 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e56ff5640 1 -- 172.21.15.8:0/2388584334 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560989c85860 con 0x56098a0278b0 2024-02-19T16:28:40.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e56ff5640 10 client.16956 handle_command_reply: tid=0 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 client.16956 shutdown 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 msgr2=0x56098a021eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 0x56098a021eb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802c950 tx=0x7f4e68004df0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 shutdown_connections 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56098a01ed30 0x56098a021eb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56098a0278b0 0x56098a096530 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56098a01e380 0x56098a021970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 --2- 172.21.15.8:0/2388584334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56098a018840 0x56098a02f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 >> 172.21.15.8:0/2388584334 conn(0x56098a01f480 msgr2=0x56098a0247b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e56ff5640 0 client.16956 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 shutdown_connections 2024-02-19T16:28:40.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 1 -- 172.21.15.8:0/2388584334 wait complete. 2024-02-19T16:28:40.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e7bd64740 20 client.16956 trim_cache size 0 max 16384 2024-02-19T16:28:40.407 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:28:40.408 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:28:40.408 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "3d87a72a-75b0-4fc8-9e09-e3707404ed62", 2024-02-19T16:28:40.408 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:28:40.408 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:28:40.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e5c0a2260 msgr2=0x7f4e5c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e5c0a2260 0x7f4e5c0a4720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c09acb0 tx=0x7f4e6c085ae0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 msgr2=0x7f4e741a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 0x7f4e741a5d10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00eb10 tx=0x7f4e6c00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 shutdown_connections 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e7410e5d0 0x7f4e741a5d10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e741056a0 0x7f4e7410e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4e5c0a2260 0x7f4e5c0a4720 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.396+0000 7f4e790ae640 1 --2- 172.21.15.8:0/209624401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e741005d0 0x7f4e7410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.397+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 >> 172.21.15.8:0/209624401 conn(0x7f4e740774f0 msgr2=0x7f4e740fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.397+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 shutdown_connections 2024-02-19T16:28:40.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.397+0000 7f4e790ae640 1 -- 172.21.15.8:0/209624401 wait complete. 2024-02-19T16:28:40.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:28:40.411 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:28:40.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2314883972 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 msgr2=0x7f33f010e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2314883972 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f010e310 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009ca0 tx=0x7f33e802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2314883972 shutdown_connections 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2314883972 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f010e310 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2314883972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0101490 0x7f33f0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2314883972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33f0107550 0x7f33f0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2314883972 >> 172.21.15.8:0/2314883972 conn(0x7f33f00fcfa0 msgr2=0x7f33f00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2314883972 shutdown_connections 2024-02-19T16:28:40.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.519+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2314883972 wait complete. 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 Processor -- start 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 -- start start 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0101490 0x7f33f019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33f0107550 0x7f33f01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0197e90 con 0x7f33f0101e50 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0198000 con 0x7f33f0107550 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f6fc9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f01981c0 con 0x7f33f0101490 2024-02-19T16:28:40.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47634/0 (socket says 172.21.15.8:47634) 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 -- 172.21.15.8:0/2090674864 learned_addr learned my addr 172.21.15.8:0/2090674864 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f4d65640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0101490 0x7f33f019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33f5566640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33f0107550 0x7f33f01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 -- 172.21.15.8:0/2090674864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0101490 msgr2=0x7f33f019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0101490 0x7f33f019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 -- 172.21.15.8:0/2090674864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33f0107550 msgr2=0x7f33f01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33f0107550 0x7f33f01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.520+0000 7f33effff640 1 -- 172.21.15.8:0/2090674864 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e8009900 con 0x7f33f0101e50 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.521+0000 7f33effff640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f019dbc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e9f0 tx=0x7f33dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.521+0000 7f33edffb640 1 -- 172.21.15.8:0/2090674864 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc009700 con 0x7f33f0101e50 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.521+0000 7f33edffb640 1 -- 172.21.15.8:0/2090674864 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33dc00cd80 con 0x7f33f0101e50 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.521+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f01984a0 con 0x7f33f0101e50 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.521+0000 7f33edffb640 1 -- 172.21.15.8:0/2090674864 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc018920 con 0x7f33f0101e50 2024-02-19T16:28:40.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.521+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f01a46b0 con 0x7f33f0101e50 2024-02-19T16:28:40.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 Processor -- start 2024-02-19T16:28:40.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33edffb640 1 -- 172.21.15.8:0/2090674864 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f33dc018a80 con 0x7f33f0101e50 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 -- start start 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d963664270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d963666ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9635ca280 con 0x55d96365b2b0 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9635c77b0 con 0x55d9636647b0 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9635c2f90 con 0x55d96365bbd0 2024-02-19T16:28:40.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f5566640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f5566640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47636/0 (socket says 172.21.15.8:47636) 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f5566640 1 -- 172.21.15.8:0/2507292266 learned_addr learned my addr 172.21.15.8:0/2507292266 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33f4d65640 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d963664270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33effff640 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d963666ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.524+0000 7f33edffb640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f33e00a2230 0x7f33e00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 -- 172.21.15.8:0/2507292266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 msgr2=0x55d963664270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d963664270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 -- 172.21.15.8:0/2507292266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 msgr2=0x55d963666ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d963666ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 -- 172.21.15.8:0/2507292266 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d963581570 con 0x55d96365b2b0 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33edffb640 1 -- 172.21.15.8:0/2090674864 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f33dc014070 con 0x7f33f0101e50 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f4d65640 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d963664270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f33e00a2230 0x7f33e00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33f5566640 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365b690 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f33e8037040 tx=0x7f33e8032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e312cc644219a09 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33daffd640 1 -- 172.21.15.8:0/2507292266 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80326a0 con 0x55d96365b2b0 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33daffd640 1 -- 172.21.15.8:0/2507292266 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33e80324e0 con 0x55d96365b2b0 2024-02-19T16:28:40.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.525+0000 7f33daffd640 1 -- 172.21.15.8:0/2507292266 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80043d0 con 0x55d96365b2b0 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 -- 172.21.15.8:0/2507292266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 msgr2=0x55d96365b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365b690 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f33e8037040 tx=0x7f33e8032de0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f5566640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f33e00a2230 0x7f33e00a46f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33e804d000 tx=0x7f33e804a000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 -- 172.21.15.8:0/2507292266 shutdown_connections 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d963666ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d963664270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/2507292266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365b690 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 -- 172.21.15.8:0/2507292266 >> 172.21.15.8:0/2507292266 conn(0x55d963655990 msgr2=0x55d963655d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 -- 172.21.15.8:0/2507292266 shutdown_connections 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.526+0000 7f33f9ba2740 1 -- 172.21.15.8:0/2507292266 wait complete. 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 -- start start 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 Processor -- start 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d96365b2b0 0x55d9636641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963661540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9636647b0 0x55d963661b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d963296430 con 0x55d96365bbd0 2024-02-19T16:28:40.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d963252e00 con 0x55d96365b2b0 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d963547fd0 con 0x55d9636647b0 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f9ba2740 10 client.0 _ll_register_callbacks cb 0x55d96359dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963661540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963661540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47646/0 (socket says 172.21.15.8:47646) 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 -- 172.21.15.8:0/426936293 learned_addr learned my addr 172.21.15.8:0/426936293 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f4d65640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d96365b2b0 0x55d9636641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33f5566640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9636647b0 0x55d963661b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 -- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9636647b0 msgr2=0x55d963661b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9636647b0 0x55d963661b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 -- 172.21.15.8:0/426936293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d96365b2b0 msgr2=0x55d9636641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d96365b2b0 0x55d9636641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.527+0000 7f33effff640 1 -- 172.21.15.8:0/426936293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f010ec10 con 0x55d96365bbd0 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33effff640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963661540 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f33dc097330 tx=0x7f33dc09add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab853964c10f9b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33cffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33cffff640 1 -- 172.21.15.8:0/426936293 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0ce3d0 con 0x55d96365bbd0 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33cffff640 1 -- 172.21.15.8:0/426936293 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f33dc0ce530 con 0x55d96365bbd0 2024-02-19T16:28:40.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9632c27e0 con 0x55d96365bbd0 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33cffff640 1 -- 172.21.15.8:0/426936293 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33dc0ce6f0 con 0x55d96365bbd0 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 10 client.16968 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d9635c1610 con 0x55d96365bbd0 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33cffff640 1 -- 172.21.15.8:0/426936293 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d9632c27e0 con 0x55d96365bbd0 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33cffff640 1 -- 172.21.15.8:0/426936293 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33dc0b3c50 con 0x55d96365bbd0 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 10 client.16968 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 20 client.16968 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d963678d20 0x55d96367b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 4 client.16968 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.528+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9632ff720 con 0x55d963678d20 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.529+0000 7f33f4d65640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d963678d20 0x55d96367b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.529+0000 7f33f4d65640 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d963678d20 0x55d96367b140 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57dda75652b5033b server_cookie=400b818ee6a0a6fb in_seq=0 out_seq=0 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.529+0000 7f33cffff640 10 client.16968 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.530+0000 7f33cffff640 1 -- 172.21.15.8:0/426936293 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d9632ff720 con 0x55d963678d20 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.530+0000 7f33cffff640 10 client.16968 handle_command_reply: tid=0 2024-02-19T16:28:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 client.16968 shutdown 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 msgr2=0x55d963661540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963661540 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f33dc097330 tx=0x7f33dc09add0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 shutdown_connections 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d963678d20 0x55d96367b140 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9636647b0 0x55d963661b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963661540 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33cffff640 0 client.16968 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/426936293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d96365b2b0 0x55d9636641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 >> 172.21.15.8:0/426936293 conn(0x55d963655990 msgr2=0x55d963657730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 shutdown_connections 2024-02-19T16:28:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 1 -- 172.21.15.8:0/426936293 wait complete. 2024-02-19T16:28:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.531+0000 7f33f9ba2740 20 client.16968 trim_cache size 0 max 16384 2024-02-19T16:28:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 Processor -- start 2024-02-19T16:28:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 -- start start 2024-02-19T16:28:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365b2b0 0x55d9636614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963663c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d9636641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9636757f0 con 0x55d96365bbd0 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d963656c10 con 0x55d9636647b0 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96364fe20 con 0x55d96365b2b0 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33effff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963663c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f5566640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d9636641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33effff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963663c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47648/0 (socket says 172.21.15.8:47648) 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f5566640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d9636641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:59188/0 (socket says 172.21.15.8:59188) 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33effff640 1 -- 172.21.15.8:0/1970014679 learned_addr learned my addr 172.21.15.8:0/1970014679 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33f4d65640 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365b2b0 0x55d9636614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33effff640 1 -- 172.21.15.8:0/1970014679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365b2b0 msgr2=0x55d9636614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.549+0000 7f33effff640 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365b2b0 0x55d9636614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33effff640 1 -- 172.21.15.8:0/1970014679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 msgr2=0x55d9636641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33effff640 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d9636641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33effff640 1 -- 172.21.15.8:0/1970014679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d96355dac0 con 0x55d96365bbd0 2024-02-19T16:28:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33f5566640 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d9636641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33effff640 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963663c70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0a3090 tx=0x7f33dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb433c6b8962a98 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33da7fc640 1 -- 172.21.15.8:0/1970014679 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0b3b50 con 0x55d96365bbd0 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33da7fc640 1 -- 172.21.15.8:0/1970014679 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33dc0b3cb0 con 0x55d96365bbd0 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.550+0000 7f33da7fc640 1 -- 172.21.15.8:0/1970014679 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0ce600 con 0x55d96365bbd0 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1970014679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 msgr2=0x55d963663c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963663c70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0a3090 tx=0x7f33dc000bc0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1970014679 shutdown_connections 2024-02-19T16:28:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d9636641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365bbd0 0x55d963663c70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1970014679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365b2b0 0x55d9636614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1970014679 >> 172.21.15.8:0/1970014679 conn(0x55d963655990 msgr2=0x55d963660dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1970014679 shutdown_connections 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1970014679 wait complete. 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 -- start start 2024-02-19T16:28:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 Processor -- start 2024-02-19T16:28:40.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.551+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d96366bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f9ba2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d96366c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96366c5c0 con 0x55d96365b2b0 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96366c730 con 0x55d9636647b0 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f9ba2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96366c8f0 con 0x55d96365bbd0 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47664/0 (socket says 172.21.15.8:47664) 2024-02-19T16:28:40.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 -- 172.21.15.8:0/1977102574 learned_addr learned my addr 172.21.15.8:0/1977102574 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f9ba2740 10 client.0 _ll_register_callbacks cb 0x7f33dc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33effff640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d96366bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f5566640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d96366c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 -- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 msgr2=0x55d96366bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d96366bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 -- 172.21.15.8:0/1977102574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 msgr2=0x55d96366c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d96366c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 -- 172.21.15.8:0/1977102574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d963581570 con 0x55d96365b2b0 2024-02-19T16:28:40.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33effff640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d96366bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f5566640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d96366c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f4d65640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365f100 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f33e4004700 tx=0x7f33e4020c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806d67521b8122fb server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33cffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33cffff640 1 -- 172.21.15.8:0/1977102574 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4017040 con 0x55d96365b2b0 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33cffff640 1 -- 172.21.15.8:0/1977102574 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f33e4010880 con 0x55d96365b2b0 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.552+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33e80324e0 con 0x55d96365b2b0 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33cffff640 1 -- 172.21.15.8:0/1977102574 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33e4010a60 con 0x55d96365b2b0 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 10 client.16974 fetch_fsmap learned FSMap version 30 2024-02-19T16:28:40.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d9635c1610 con 0x55d96365b2b0 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33cffff640 1 -- 172.21.15.8:0/1977102574 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33e80324e0 con 0x55d96365b2b0 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33cffff640 1 -- 172.21.15.8:0/1977102574 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33e4004930 con 0x55d96365b2b0 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 10 client.16974 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 20 client.16974 populate_metadata read hostname 'smithi008' 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d9636d5bb0 0x55d9636d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 4 client.16974 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:28:40.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d96334e890 con 0x55d9636d5bb0 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.553+0000 7f33effff640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d9636d5bb0 0x55d9636d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33effff640 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d9636d5bb0 0x55d9636d7fd0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71555f4d72b0767 server_cookie=37df197f508e32c in_seq=0 out_seq=0 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33cffff640 10 client.16974 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33cffff640 1 -- 172.21.15.8:0/1977102574 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55d96334e890 con 0x55d9636d5bb0 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33cffff640 10 client.16974 handle_command_reply: tid=0 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 client.16974 shutdown 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 msgr2=0x55d96365f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365f100 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f33e4004700 tx=0x7f33e4020c00 comp rx=0 tx=0).stop 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 shutdown_connections 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9636647b0 0x55d96366c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55d9636d5bb0 0x55d9636d7fd0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d96365bbd0 0x55d96366bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 --2- 172.21.15.8:0/1977102574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d96365b2b0 0x55d96365f100 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 >> 172.21.15.8:0/1977102574 conn(0x55d963655990 msgr2=0x55d9636541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.554+0000 7f33cffff640 0 client.16974 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 shutdown_connections 2024-02-19T16:28:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f9ba2740 1 -- 172.21.15.8:0/1977102574 wait complete. 2024-02-19T16:28:40.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f9ba2740 20 client.16974 trim_cache size 0 max 16384 2024-02-19T16:28:40.566 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:28:40.566 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2639 inodes in the stack)", 2024-02-19T16:28:40.566 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:28:40.566 INFO:teuthology.orchestra.run.smithi008.stdout: "3d87a72a-75b0-4fc8-9e09-e3707404ed62": { 2024-02-19T16:28:40.567 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:28:40.567 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "3d87a72a-75b0-4fc8-9e09-e3707404ed62", 2024-02-19T16:28:40.567 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:28:40.567 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:28:40.567 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:28:40.567 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f33e00a2230 msgr2=0x7f33e00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f33e00a2230 0x7f33e00a46f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33e804d000 tx=0x7f33e804a000 comp rx=0 tx=0).stop 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 msgr2=0x7f33f019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f019dbc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e9f0 tx=0x7f33dc00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 shutdown_connections 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33f0107550 0x7f33f01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33f0101e50 0x7f33f019dbc0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f33e00a2230 0x7f33e00a46f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 --2- 172.21.15.8:0/2090674864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0101490 0x7f33f019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:28:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 >> 172.21.15.8:0/2090674864 conn(0x7f33f00fcfa0 msgr2=0x7f33f00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:28:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 shutdown_connections 2024-02-19T16:28:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:28:40.555+0000 7f33f6fc9640 1 -- 172.21.15.8:0/2090674864 wait complete. 2024-02-19T16:28:40.569 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:40 smithi008 ceph-mon[32573]: pgmap v176: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:40.570 INFO:tasks.cephfs.filesystem:scrub status for tag:3d87a72a-75b0-4fc8-9e09-e3707404ed62 - {'path': '/', 'tag': '3d87a72a-75b0-4fc8-9e09-e3707404ed62', 'options': 'recursive,force'} 2024-02-19T16:28:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:40 smithi191 ceph-mon[37203]: pgmap v176: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:40 smithi156 ceph-mon[36502]: pgmap v176: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:28:41.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:28:41.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:41 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:28:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:41 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:28:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:41.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:41.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:41.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:41 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:42.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:42 smithi008 ceph-mon[32573]: pgmap v177: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:42.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:42 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:28:42.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:42 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:42 smithi191 ceph-mon[37203]: pgmap v177: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:42 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:28:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:42 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:42 smithi156 ceph-mon[36502]: pgmap v177: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:42 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:28:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:42 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:28:44.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:44 smithi008 ceph-mon[32573]: pgmap v178: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:44 smithi191 ceph-mon[37203]: pgmap v178: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:44 smithi156 ceph-mon[36502]: pgmap v178: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:46.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:46 smithi008 ceph-mon[32573]: pgmap v179: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:46 smithi191 ceph-mon[37203]: pgmap v179: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:46 smithi156 ceph-mon[36502]: pgmap v179: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:48.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:48 smithi008 ceph-mon[32573]: pgmap v180: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:48 smithi191 ceph-mon[37203]: pgmap v180: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:48 smithi156 ceph-mon[36502]: pgmap v180: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:49 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:28:49.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:49 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:28:49.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:49 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:28:49.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:49 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:28:49.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:49 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:28:49.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:49 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:28:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:50 smithi008 ceph-mon[32573]: pgmap v181: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:50 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:28:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:50 smithi191 ceph-mon[37203]: pgmap v181: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:50 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:28:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:50 smithi156 ceph-mon[36502]: pgmap v181: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:50 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:28:52.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:52 smithi008 ceph-mon[32573]: pgmap v182: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:52.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:52 smithi191 ceph-mon[37203]: pgmap v182: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:52.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:52 smithi156 ceph-mon[36502]: pgmap v182: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:54.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:54 smithi008 ceph-mon[32573]: pgmap v183: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:54 smithi191 ceph-mon[37203]: pgmap v183: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:54.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:54 smithi156 ceph-mon[36502]: pgmap v183: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:28:56.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:56 smithi008 ceph-mon[32573]: pgmap v184: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:56 smithi191 ceph-mon[37203]: pgmap v184: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:56.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:56 smithi156 ceph-mon[36502]: pgmap v184: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:28:58.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:28:58 smithi008 ceph-mon[32573]: pgmap v185: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:58.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:28:58 smithi191 ceph-mon[37203]: pgmap v185: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:28:58.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:28:58 smithi156 ceph-mon[36502]: pgmap v185: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:00 smithi008 ceph-mon[32573]: pgmap v186: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:00 smithi191 ceph-mon[37203]: pgmap v186: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:00 smithi156 ceph-mon[36502]: pgmap v186: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:02.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:02 smithi008 ceph-mon[32573]: pgmap v187: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:02.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:02 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:29:02.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:02 smithi191 ceph-mon[37203]: pgmap v187: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:02.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:02 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:29:02.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:02 smithi156 ceph-mon[36502]: pgmap v187: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:02.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:02 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:29:03.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:29:03.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:29:03.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:29:03.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:29:03.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:29:03.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:29:03.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:29:03.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:29:03.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:29:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:04 smithi008 ceph-mon[32573]: pgmap v188: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:04 smithi191 ceph-mon[37203]: pgmap v188: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:04 smithi156 ceph-mon[36502]: pgmap v188: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:06 smithi008 ceph-mon[32573]: pgmap v189: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-02-19T16:29:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:06 smithi191 ceph-mon[37203]: pgmap v189: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-02-19T16:29:06.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:06 smithi156 ceph-mon[36502]: pgmap v189: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-02-19T16:29:08.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:08 smithi008 ceph-mon[32573]: pgmap v190: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-02-19T16:29:08.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:08 smithi191 ceph-mon[37203]: pgmap v190: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-02-19T16:29:08.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:08 smithi156 ceph-mon[36502]: pgmap v190: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-02-19T16:29:10.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:29:10.571 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:29:10.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 -- 172.21.15.8:0/1536326384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500101460 msgr2=0x7f75001018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 --2- 172.21.15.8:0/1536326384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500101460 0x7f75001018e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009930 tx=0x7f74ec02bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:10.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 -- 172.21.15.8:0/1536326384 shutdown_connections 2024-02-19T16:29:10.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 --2- 172.21.15.8:0/1536326384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7500101e20 0x7f750010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 --2- 172.21.15.8:0/1536326384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500101460 0x7f75001018e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 --2- 172.21.15.8:0/1536326384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7500107500 0x7f75001078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 -- 172.21.15.8:0/1536326384 >> 172.21.15.8:0/1536326384 conn(0x7f75000fcf90 msgr2=0x7f75000ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 -- 172.21.15.8:0/1536326384 shutdown_connections 2024-02-19T16:29:10.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.687+0000 7f750725c640 1 -- 172.21.15.8:0/1536326384 wait complete. 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 Processor -- start 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 -- start start 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7500101460 0x7f750019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7500101e20 0x7f750019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 0x7f7500197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500197f80 con 0x7f7500107500 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001980f0 con 0x7f7500101e20 2024-02-19T16:29:10.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f750725c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001982b0 con 0x7f7500101460 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f75057f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 0x7f7500197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f75057f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 0x7f7500197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33224/0 (socket says 172.21.15.8:33224) 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f75057f9640 1 -- 172.21.15.8:0/1688259146 learned_addr learned my addr 172.21.15.8:0/1688259146 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f7504ff8640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7500101460 0x7f750019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.688+0000 7f74fffff640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7500101e20 0x7f750019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f75057f9640 1 -- 172.21.15.8:0/1688259146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7500101460 msgr2=0x7f750019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f75057f9640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7500101460 0x7f750019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f75057f9640 1 -- 172.21.15.8:0/1688259146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7500101e20 msgr2=0x7f750019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f75057f9640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7500101e20 0x7f750019dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f75057f9640 1 -- 172.21.15.8:0/1688259146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec009560 con 0x7f7500107500 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f75057f9640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 0x7f7500197930 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f74f800eb10 tx=0x7f74f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f74fdffb640 1 -- 172.21.15.8:0/1688259146 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8004b20 con 0x7f7500107500 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7500198590 con 0x7f7500107500 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f74fdffb640 1 -- 172.21.15.8:0/1688259146 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f8004c80 con 0x7f7500107500 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f74fdffb640 1 -- 172.21.15.8:0/1688259146 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8010650 con 0x7f7500107500 2024-02-19T16:29:10.695 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.689+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001a4820 con 0x7f7500107500 2024-02-19T16:29:10.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.691+0000 7f74fdffb640 1 -- 172.21.15.8:0/1688259146 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f74f8005020 con 0x7f7500107500 2024-02-19T16:29:10.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 Processor -- start 2024-02-19T16:29:10.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f74fdffb640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f74f00a2230 0x7f74f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 -- start start 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f74fffff640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f74f00a2230 0x7f74f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 0x55a14cdb6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f74fdffb640 1 -- 172.21.15.8:0/1688259146 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f74f8014070 con 0x7f7500107500 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cd25280 con 0x55a14cdb62b0 2024-02-19T16:29:10.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cd227b0 con 0x55a14cdb6bd0 2024-02-19T16:29:10.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7509e39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cd1df90 con 0x55a14cdbf7b0 2024-02-19T16:29:10.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7504ff8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 0x55a14cdb6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7504ff8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 0x55a14cdb6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33240/0 (socket says 172.21.15.8:33240) 2024-02-19T16:29:10.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f7504ff8640 1 -- 172.21.15.8:0/2960956656 learned_addr learned my addr 172.21.15.8:0/2960956656 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:10.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f74fffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.692+0000 7f75057f9640 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f7504ff8640 1 -- 172.21.15.8:0/2960956656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 msgr2=0x55a14cdc1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f7504ff8640 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f7504ff8640 1 -- 172.21.15.8:0/2960956656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 msgr2=0x55a14cdbf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f7504ff8640 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f7504ff8640 1 -- 172.21.15.8:0/2960956656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a14ccdc570 con 0x55a14cdb62b0 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f74fffff640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f74f00a2230 0x7f74f00a46f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74ec005ab0 tx=0x7f74ec004a80 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f7504ff8640 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 0x55a14cdb6690 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009c10 tx=0x7f74f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391e6a8f9352f9de server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f74eaffd640 1 -- 172.21.15.8:0/2960956656 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4004050 con 0x55a14cdb62b0 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f74eaffd640 1 -- 172.21.15.8:0/2960956656 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f4005500 con 0x55a14cdb62b0 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.693+0000 7f74eaffd640 1 -- 172.21.15.8:0/2960956656 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4004be0 con 0x55a14cdb62b0 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 -- 172.21.15.8:0/2960956656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 msgr2=0x55a14cdb6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 0x55a14cdb6690 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009c10 tx=0x7f74f4011810 comp rx=0 tx=0).stop 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 -- 172.21.15.8:0/2960956656 shutdown_connections 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 --2- 172.21.15.8:0/2960956656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb62b0 0x55a14cdb6690 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 -- 172.21.15.8:0/2960956656 >> 172.21.15.8:0/2960956656 conn(0x55a14cdb0990 msgr2=0x55a14cdb0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 -- 172.21.15.8:0/2960956656 shutdown_connections 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 -- 172.21.15.8:0/2960956656 wait complete. 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 -- start start 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.694+0000 7f7509e39740 1 Processor -- start 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb62b0 0x55a14cdbf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c9f1430 con 0x55a14cdb6bd0 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c9ade00 con 0x55a14cdb62b0 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cca2fd0 con 0x55a14cdbf7b0 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 10 client.0 _ll_register_callbacks cb 0x55a14ccf8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33246/0 (socket says 172.21.15.8:33246) 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 -- 172.21.15.8:0/3873156359 learned_addr learned my addr 172.21.15.8:0/3873156359 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:10.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7504ff8640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb62b0 0x55a14cdbf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f75057f9640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 -- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 msgr2=0x55a14cdbca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 -- 172.21.15.8:0/3873156359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb62b0 msgr2=0x55a14cdbf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb62b0 0x55a14cdbf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 -- 172.21.15.8:0/3873156359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f4009870 con 0x55a14cdb6bd0 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74fffff640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbc4c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009930 tx=0x7f74ec02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c273f7ff8e19c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74dffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74dffff640 1 -- 172.21.15.8:0/3873156359 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec005000 con 0x55a14cdb6bd0 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74dffff640 1 -- 172.21.15.8:0/3873156359 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f74ec005160 con 0x55a14cdb6bd0 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a14ca1d7e0 con 0x55a14cdb6bd0 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f74dffff640 1 -- 172.21.15.8:0/3873156359 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74ec005320 con 0x55a14cdb6bd0 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.695+0000 7f7509e39740 10 client.16986 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a14cd1c610 con 0x55a14cdb6bd0 2024-02-19T16:29:10.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f74dffff640 1 -- 172.21.15.8:0/3873156359 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a14ca1d7e0 con 0x55a14cdb6bd0 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f74dffff640 1 -- 172.21.15.8:0/3873156359 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74ec042470 con 0x55a14cdb6bd0 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 10 client.16986 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 20 client.16986 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14cdcf8e0 0x55a14cdd1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 4 client.16986 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:10.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a14ca5a720 con 0x55a14cdcf8e0 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.696+0000 7f7504ff8640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14cdcf8e0 0x55a14cdd1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f74dffff640 10 client.16986 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7504ff8640 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14cdcf8e0 0x55a14cdd1d00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a067043c94f4e0b6 server_cookie=7abb3839c7e5b4e7 in_seq=0 out_seq=0 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f74dffff640 1 -- 172.21.15.8:0/3873156359 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a14ca5a720 con 0x55a14cdcf8e0 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f74dffff640 10 client.16986 handle_command_reply: tid=0 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 client.16986 shutdown 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 msgr2=0x55a14cdbc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbc4c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009930 tx=0x7f74ec02bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 shutdown_connections 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14cdcf8e0 0x55a14cdd1d00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f74dffff640 0 client.16986 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbc4c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 --2- 172.21.15.8:0/3873156359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb62b0 0x55a14cdbf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 >> 172.21.15.8:0/3873156359 conn(0x55a14cdb0990 msgr2=0x55a14cdb2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 shutdown_connections 2024-02-19T16:29:10.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.697+0000 7f7509e39740 1 -- 172.21.15.8:0/3873156359 wait complete. 2024-02-19T16:29:10.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.698+0000 7f7509e39740 20 client.16986 trim_cache size 0 max 16384 2024-02-19T16:29:10.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 Processor -- start 2024-02-19T16:29:10.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 -- start start 2024-02-19T16:29:10.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdbb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cdac2b0 con 0x55a14cdbf7b0 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cdaae20 con 0x55a14cdb6bd0 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7509e39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cdbaa10 con 0x55a14cdb62b0 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33262/0 (socket says 172.21.15.8:33262) 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 -- 172.21.15.8:0/1141420831 learned_addr learned my addr 172.21.15.8:0/1141420831 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f7504ff8640 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdbb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f74fffff640 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 -- 172.21.15.8:0/1141420831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 msgr2=0x55a14cdbb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdbb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 -- 172.21.15.8:0/1141420831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 msgr2=0x55a14cdbebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 -- 172.21.15.8:0/1141420831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a14ccb8ac0 con 0x55a14cdbf7b0 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.714+0000 7f75057f9640 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbf130 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f74f80049d0 tx=0x7f74f800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebb3e0f32e24028 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f74ea7fc640 1 -- 172.21.15.8:0/1141420831 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f80b1070 con 0x55a14cdbf7b0 2024-02-19T16:29:10.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f74ea7fc640 1 -- 172.21.15.8:0/1141420831 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f80a3810 con 0x55a14cdbf7b0 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f74ea7fc640 1 -- 172.21.15.8:0/1141420831 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8095be0 con 0x55a14cdbf7b0 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 -- 172.21.15.8:0/1141420831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 msgr2=0x55a14cdbf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbf130 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f74f80049d0 tx=0x7f74f800cee0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 -- 172.21.15.8:0/1141420831 shutdown_connections 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdbf7b0 0x55a14cdbf130 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdb6bd0 0x55a14cdbebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 --2- 172.21.15.8:0/1141420831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdbb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 -- 172.21.15.8:0/1141420831 >> 172.21.15.8:0/1141420831 conn(0x55a14cdb0990 msgr2=0x55a14cdaf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 -- 172.21.15.8:0/1141420831 shutdown_connections 2024-02-19T16:29:10.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.715+0000 7f7509e39740 1 -- 172.21.15.8:0/1141420831 wait complete. 2024-02-19T16:29:10.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 -- start start 2024-02-19T16:29:10.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 Processor -- start 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdc14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdc2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cdc3180 con 0x55a14cdb6bd0 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cdc32f0 con 0x55a14cdbf7b0 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cdc34b0 con 0x55a14cdb62b0 2024-02-19T16:29:10.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7509e39740 10 client.0 _ll_register_callbacks cb 0x7f74ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f74fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdc2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7504ff8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdc14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f75057f9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f74fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdc2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33266/0 (socket says 172.21.15.8:33266) 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f7504ff8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdc14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:36494/0 (socket says 172.21.15.8:36494) 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f74fffff640 1 -- 172.21.15.8:0/26354466 learned_addr learned my addr 172.21.15.8:0/26354466 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.716+0000 7f75057f9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:36310/0 (socket says 172.21.15.8:36310) 2024-02-19T16:29:10.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74fffff640 1 -- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 msgr2=0x55a14cdc14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74fffff640 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdc14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74fffff640 1 -- 172.21.15.8:0/26354466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 msgr2=0x55a14cdc2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74fffff640 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74fffff640 1 -- 172.21.15.8:0/26354466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001a4820 con 0x55a14cdb6bd0 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f75057f9640 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74fffff640 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdc2630 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec0049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318c40eb97c3fcb2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74dffff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74dffff640 1 -- 172.21.15.8:0/26354466 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec042680 con 0x55a14cdb6bd0 2024-02-19T16:29:10.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74dffff640 1 -- 172.21.15.8:0/26354466 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f74ec042ba0 con 0x55a14cdb6bd0 2024-02-19T16:29:10.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74f4005500 con 0x55a14cdb6bd0 2024-02-19T16:29:10.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74dffff640 1 -- 172.21.15.8:0/26354466 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74ec005000 con 0x55a14cdb6bd0 2024-02-19T16:29:10.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f7509e39740 10 client.16992 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:10.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a14ccde6a0 con 0x55a14cdb6bd0 2024-02-19T16:29:10.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.717+0000 7f74dffff640 1 -- 172.21.15.8:0/26354466 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f4005500 con 0x55a14cdb6bd0 2024-02-19T16:29:10.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f74dffff640 1 -- 172.21.15.8:0/26354466 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74ec003890 con 0x55a14cdb6bd0 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 10 client.16992 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 20 client.16992 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14ce2c500 0x55a14ce2e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 4 client.16992 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a14caa9890 con 0x55a14ce2c500 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7504ff8640 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14ce2c500 0x55a14ce2e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f74dffff640 10 client.16992 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7504ff8640 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14ce2c500 0x55a14ce2e920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ed6a0333e38961 server_cookie=527758d9e8f89834 in_seq=0 out_seq=0 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f74dffff640 1 -- 172.21.15.8:0/26354466 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a14caa9890 con 0x55a14ce2c500 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f74dffff640 10 client.16992 handle_command_reply: tid=0 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 1 client.16992 shutdown 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 msgr2=0x55a14cdc2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.718+0000 7f7509e39740 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdc2630 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec0049a0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 shutdown_connections 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a14cdbf7b0 0x55a14cdc2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a14cdb6bd0 0x55a14cdc2630 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a14ce2c500 0x55a14ce2e920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 --2- 172.21.15.8:0/26354466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a14cdb62b0 0x55a14cdc14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f74dffff640 0 client.16992 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 >> 172.21.15.8:0/26354466 conn(0x55a14cdb0990 msgr2=0x55a14cdafa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 shutdown_connections 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 1 -- 172.21.15.8:0/26354466 wait complete. 2024-02-19T16:29:10.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f7509e39740 20 client.16992 trim_cache size 0 max 16384 2024-02-19T16:29:10.732 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:29:10.732 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:29:10.732 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:29:10.732 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:29:10.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f74f00a2230 msgr2=0x7f74f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f74f00a2230 0x7f74f00a46f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74ec005ab0 tx=0x7f74ec004a80 comp rx=0 tx=0).stop 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 msgr2=0x7f7500197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 0x7f7500197930 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f74f800eb10 tx=0x7f74f800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 shutdown_connections 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7500107500 0x7f7500197930 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7500101e20 0x7f750019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f74f00a2230 0x7f74f00a46f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 --2- 172.21.15.8:0/1688259146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7500101460 0x7f750019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 >> 172.21.15.8:0/1688259146 conn(0x7f75000fcf90 msgr2=0x7f75000fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 shutdown_connections 2024-02-19T16:29:10.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.719+0000 7f750725c640 1 -- 172.21.15.8:0/1688259146 wait complete. 2024-02-19T16:29:10.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:29:10.735 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:29:10.759 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:10 smithi008 ceph-mon[32573]: pgmap v191: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-02-19T16:29:10.759 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:10 smithi008 ceph-mon[32573]: osdmap e97: 12 total, 12 up, 12 in 2024-02-19T16:29:10.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:10 smithi191 ceph-mon[37203]: pgmap v191: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-02-19T16:29:10.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:10 smithi191 ceph-mon[37203]: osdmap e97: 12 total, 12 up, 12 in 2024-02-19T16:29:10.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:10 smithi156 ceph-mon[36502]: pgmap v191: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-02-19T16:29:10.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:10 smithi156 ceph-mon[36502]: osdmap e97: 12 total, 12 up, 12 in 2024-02-19T16:29:10.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 -- 172.21.15.8:0/314192506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41056c0 msgr2=0x7fe3f410d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 --2- 172.21.15.8:0/314192506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41056c0 0x7fe3f410d290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009c10 tx=0x7fe3e402bef0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 -- 172.21.15.8:0/314192506 shutdown_connections 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 --2- 172.21.15.8:0/314192506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41056c0 0x7fe3f410d290 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 --2- 172.21.15.8:0/314192506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f4100fa0 0x7fe3f4104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 --2- 172.21.15.8:0/314192506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3f41005f0 0x7fe3f41009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 -- 172.21.15.8:0/314192506 >> 172.21.15.8:0/314192506 conn(0x7fe3f40774c0 msgr2=0x7fe3f40778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 -- 172.21.15.8:0/314192506 shutdown_connections 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 -- 172.21.15.8:0/314192506 wait complete. 2024-02-19T16:29:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 Processor -- start 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.837+0000 7fe3fb807640 1 -- start start 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3fb807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 0x7fe3f419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3fb807640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3f41056c0 0x7fe3f419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3fb807640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f4197920 0x7fe3f4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3fb807640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4198310 con 0x7fe3f41005f0 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3fb807640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4198480 con 0x7fe3f4197920 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3fb807640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4198640 con 0x7fe3f41056c0 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 0x7fe3f419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 0x7fe3f419d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33274/0 (socket says 172.21.15.8:33274) 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 -- 172.21.15.8:0/2467287773 learned_addr learned my addr 172.21.15.8:0/2467287773 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f8da2640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3f41056c0 0x7fe3f419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f9da4640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f4197920 0x7fe3f4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 -- 172.21.15.8:0/2467287773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3f41056c0 msgr2=0x7fe3f419dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3f41056c0 0x7fe3f419dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 -- 172.21.15.8:0/2467287773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f4197920 msgr2=0x7fe3f4197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f4197920 0x7fe3f4197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:10.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 -- 172.21.15.8:0/2467287773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec009560 con 0x7fe3f41005f0 2024-02-19T16:29:10.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f95a3640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 0x7fe3f419d770 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00e9c0 tx=0x7fe3ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.838+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec00cde0 con 0x7fe3f41005f0 2024-02-19T16:29:10.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.839+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e8009490 con 0x7fe3f41005f0 2024-02-19T16:29:10.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.839+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3ec0035f0 con 0x7fe3f41005f0 2024-02-19T16:29:10.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.839+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec010650 con 0x7fe3f41005f0 2024-02-19T16:29:10.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.839+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4009840 con 0x7fe3f41005f0 2024-02-19T16:29:10.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.839+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4101ac0 con 0x7fe3f41005f0 2024-02-19T16:29:10.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.841+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe3ec003ad0 con 0x7fe3f41005f0 2024-02-19T16:29:10.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.841+0000 7fe3f27fc640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe3dc0a2230 0x7fe3dc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:10.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.841+0000 7fe3f8da2640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe3dc0a2230 0x7fe3dc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:10.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.841+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fe3ec014070 con 0x7fe3f41005f0 2024-02-19T16:29:10.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.842+0000 7fe3f8da2640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe3dc0a2230 0x7fe3dc0a46f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40ffe90 tx=0x7fe3e8005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:10.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:10.846+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe3ec097170 con 0x7fe3f41005f0 2024-02-19T16:29:11.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.066+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3f4068730 con 0x7fe3f41005f0 2024-02-19T16:29:11.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.067+0000 7fe3f27fc640 1 -- 172.21.15.8:0/2467287773 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fe3ec003760 con 0x7fe3f41005f0 2024-02-19T16:29:11.068 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:29:11.069 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:29:11.069 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:29:11.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe3dc0a2230 msgr2=0x7fe3dc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe3dc0a2230 0x7fe3dc0a46f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40ffe90 tx=0x7fe3e8005f90 comp rx=0 tx=0).stop 2024-02-19T16:29:11.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 msgr2=0x7fe3f419d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 0x7fe3f419d770 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00e9c0 tx=0x7fe3ec00ee80 comp rx=0 tx=0).stop 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 shutdown_connections 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3f4197920 0x7fe3f4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe3dc0a2230 0x7fe3dc0a46f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3f41056c0 0x7fe3f419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 --2- 172.21.15.8:0/2467287773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3f41005f0 0x7fe3f419d770 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 >> 172.21.15.8:0/2467287773 conn(0x7fe3f40774c0 msgr2=0x7fe3f40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 shutdown_connections 2024-02-19T16:29:11.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.069+0000 7fe3fb807640 1 -- 172.21.15.8:0/2467287773 wait complete. 2024-02-19T16:29:11.081 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:29:11.081 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:29:11.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 -- 172.21.15.8:0/1519230542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0069090 msgr2=0x7f37e010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 --2- 172.21.15.8:0/1519230542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0069090 0x7f37e010e330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009cd0 tx=0x7f37d802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 -- 172.21.15.8:0/1519230542 shutdown_connections 2024-02-19T16:29:11.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 --2- 172.21.15.8:0/1519230542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0069090 0x7f37e010e330 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 --2- 172.21.15.8:0/1519230542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e00686d0 0x7f37e0068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 --2- 172.21.15.8:0/1519230542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e0107500 0x7f37e01078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 -- 172.21.15.8:0/1519230542 >> 172.21.15.8:0/1519230542 conn(0x7f37e00774f0 msgr2=0x7f37e0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 -- 172.21.15.8:0/1519230542 shutdown_connections 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.225+0000 7f37e559a640 1 -- 172.21.15.8:0/1519230542 wait complete. 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 Processor -- start 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 -- start start 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e00686d0 0x7f37e019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e0069090 0x7f37e019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 0x7f37e01978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0197f40 con 0x7f37e0107500 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e01980b0 con 0x7f37e00686d0 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37e559a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0198270 con 0x7f37e0069090 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 0x7f37e01978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37deffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e00686d0 0x7f37e019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 0x7f37e01978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33298/0 (socket says 172.21.15.8:33298) 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 -- 172.21.15.8:0/3276759731 learned_addr learned my addr 172.21.15.8:0/3276759731 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37d7fff640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e0069090 0x7f37e019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 -- 172.21.15.8:0/3276759731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e0069090 msgr2=0x7f37e019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e0069090 0x7f37e019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 -- 172.21.15.8:0/3276759731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e00686d0 msgr2=0x7f37e019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e00686d0 0x7f37e019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37df7fe640 1 -- 172.21.15.8:0/3276759731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c8009490 con 0x7f37e0107500 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.226+0000 7f37deffd640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e00686d0 0x7f37e019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.227+0000 7f37df7fe640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 0x7f37e01978f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f37d8004af0 tx=0x7f37d8032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.227+0000 7f37dcff9640 1 -- 172.21.15.8:0/3276759731 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d80328b0 con 0x7f37e0107500 2024-02-19T16:29:11.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.227+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d8009900 con 0x7f37e0107500 2024-02-19T16:29:11.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.227+0000 7f37dcff9640 1 -- 172.21.15.8:0/3276759731 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37d8003e50 con 0x7f37e0107500 2024-02-19T16:29:11.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.227+0000 7f37dcff9640 1 -- 172.21.15.8:0/3276759731 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004740 con 0x7f37e0107500 2024-02-19T16:29:11.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.227+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e01a4600 con 0x7f37e0107500 2024-02-19T16:29:11.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.229+0000 7f37dcff9640 1 -- 172.21.15.8:0/3276759731 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f37d800a040 con 0x7f37e0107500 2024-02-19T16:29:11.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.229+0000 7f37e8178740 1 Processor -- start 2024-02-19T16:29:11.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.229+0000 7f37e8178740 1 -- start start 2024-02-19T16:29:11.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37e8178740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d916ffc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37e8178740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917009620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37e8178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d917009b60 0x55d91700bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37e8178740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916fb7960 con 0x55d916ffbf80 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37e8178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916f6b310 con 0x55d917009b60 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37e8178740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916f68840 con 0x55d916ffc8a0 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d916ffc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d916ffc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33314/0 (socket says 172.21.15.8:33314) 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37d7fff640 1 -- 172.21.15.8:0/1922387982 learned_addr learned my addr 172.21.15.8:0/1922387982 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917009620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37dcff9640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37cc0a2230 0x7f37cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37df7fe640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37cc0a2230 0x7f37cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37d7fff640 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d917009b60 0x55d91700bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 -- 172.21.15.8:0/1922387982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d917009b60 msgr2=0x55d91700bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d917009b60 0x55d91700bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 -- 172.21.15.8:0/1922387982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 msgr2=0x55d916ffc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d916ffc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 -- 172.21.15.8:0/1922387982 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d916f26060 con 0x55d916ffc8a0 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37dcff9640 1 -- 172.21.15.8:0/3276759731 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f37d80f0f70 con 0x7f37e0107500 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37deffd640 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917009620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009f20 tx=0x7f37d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f9b94c720371be server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37df7fe640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37cc0a2230 0x7f37cc0a46f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37d80ee860 tx=0x7f37d8086050 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.230+0000 7f37d57fa640 1 -- 172.21.15.8:0/1922387982 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0004030 con 0x55d916ffc8a0 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37d57fa640 1 -- 172.21.15.8:0/1922387982 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37d0005500 con 0x55d916ffc8a0 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37d57fa640 1 -- 172.21.15.8:0/1922387982 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0004c70 con 0x55d916ffc8a0 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 -- 172.21.15.8:0/1922387982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 msgr2=0x55d917009620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917009620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009f20 tx=0x7f37d0011810 comp rx=0 tx=0).stop 2024-02-19T16:29:11.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 -- 172.21.15.8:0/1922387982 shutdown_connections 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d917009b60 0x55d91700bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917009620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 --2- 172.21.15.8:0/1922387982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d916ffc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 -- 172.21.15.8:0/1922387982 >> 172.21.15.8:0/1922387982 conn(0x55d916ff6980 msgr2=0x55d916ff6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 -- 172.21.15.8:0/1922387982 shutdown_connections 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.231+0000 7f37e8178740 1 -- 172.21.15.8:0/1922387982 wait complete. 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 -- start start 2024-02-19T16:29:11.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 Processor -- start 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917006ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d917007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917007550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916f64020 con 0x55d916ffbf80 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916c37430 con 0x55d916ffc8a0 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916bf3e00 con 0x55d917009b60 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37e8178740 10 client.0 _ll_register_callbacks cb 0x55d916f0bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:11.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917006ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917006ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33318/0 (socket says 172.21.15.8:33318) 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 -- 172.21.15.8:0/4272070408 learned_addr learned my addr 172.21.15.8:0/4272070408 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37deffd640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d917007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917007550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 -- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 msgr2=0x55d917007550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917007550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 -- 172.21.15.8:0/4272070408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 msgr2=0x55d917007010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d917007010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 -- 172.21.15.8:0/4272070408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d0009c00 con 0x55d916ffbf80 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37deffd640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d917007010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.232+0000 7f37d7fff640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917006ad0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f37c800f4c0 tx=0x7f37c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a66f5ea119a92d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37c67fc640 1 -- 172.21.15.8:0/4272070408 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c800fe40 con 0x55d916ffbf80 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37c67fc640 1 -- 172.21.15.8:0/4272070408 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f37c8013130 con 0x55d916ffbf80 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d916c637e0 con 0x55d916ffbf80 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37c67fc640 1 -- 172.21.15.8:0/4272070408 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37c80132f0 con 0x55d916ffbf80 2024-02-19T16:29:11.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 10 client.17010 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d916f626a0 con 0x55d916ffbf80 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37c67fc640 1 -- 172.21.15.8:0/4272070408 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d916c637e0 con 0x55d916ffbf80 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37c67fc640 1 -- 172.21.15.8:0/4272070408 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37c8010430 con 0x55d916ffbf80 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 10 client.17010 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 10 client.17010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 20 client.17010 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d917019f30 0x55d91701c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 4 client.17010 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:11.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d916ca0720 con 0x55d917019f30 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.233+0000 7f37deffd640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d917019f30 0x55d91701c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.234+0000 7f37c67fc640 10 client.17010 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.234+0000 7f37deffd640 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d917019f30 0x55d91701c350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f7cb90c2cc0ae21 server_cookie=d7eecb3d1d0a649d in_seq=0 out_seq=0 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.234+0000 7f37c67fc640 1 -- 172.21.15.8:0/4272070408 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d916ca0720 con 0x55d917019f30 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.234+0000 7f37c67fc640 10 client.17010 handle_command_reply: tid=0 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.234+0000 7f37e8178740 1 client.17010 shutdown 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 msgr2=0x55d917006ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917006ad0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f37c800f4c0 tx=0x7f37c8005f90 comp rx=0 tx=0).stop 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 shutdown_connections 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d917019f30 0x55d91701c350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917007550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37c67fc640 0 client.17010 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d917007010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 --2- 172.21.15.8:0/4272070408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917006ad0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 >> 172.21.15.8:0/4272070408 conn(0x55d916ff6980 msgr2=0x55d916ff86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 shutdown_connections 2024-02-19T16:29:11.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 1 -- 172.21.15.8:0/4272070408 wait complete. 2024-02-19T16:29:11.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.235+0000 7f37e8178740 20 client.17010 trim_cache size 0 max 16384 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 Processor -- start 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 -- start start 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917009200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d916ff1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917014f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916ee9130 con 0x55d916ffbf80 2024-02-19T16:29:11.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d916ff7780 con 0x55d916ffc8a0 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.247+0000 7f37e8178740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d91700bf50 con 0x55d917009b60 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917009200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917009200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33330/0 (socket says 172.21.15.8:33330) 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 -- 172.21.15.8:0/82250187 learned_addr learned my addr 172.21.15.8:0/82250187 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37deffd640 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d916ff1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917014f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 -- 172.21.15.8:0/82250187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 msgr2=0x55d917014f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917014f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 -- 172.21.15.8:0/82250187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 msgr2=0x55d916ff1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d916ff1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 -- 172.21.15.8:0/82250187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d916efec20 con 0x55d916ffbf80 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37deffd640 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d916ff1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d7fff640 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917009200 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009c40 tx=0x7f37c8009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5a3bcf68dd564 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d4ff9640 1 -- 172.21.15.8:0/82250187 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c800fe10 con 0x55d916ffbf80 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d4ff9640 1 -- 172.21.15.8:0/82250187 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37c8010430 con 0x55d916ffbf80 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.248+0000 7f37d4ff9640 1 -- 172.21.15.8:0/82250187 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8003850 con 0x55d916ffbf80 2024-02-19T16:29:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- 172.21.15.8:0/82250187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 msgr2=0x55d917009200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917009200 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009c40 tx=0x7f37c8009770 comp rx=0 tx=0).stop 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- 172.21.15.8:0/82250187 shutdown_connections 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d917009b60 0x55d917014f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffc8a0 0x55d916ff1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- 172.21.15.8:0/82250187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d916ffbf80 0x55d917009200 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- 172.21.15.8:0/82250187 >> 172.21.15.8:0/82250187 conn(0x55d916ff6980 msgr2=0x55d916ff5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- 172.21.15.8:0/82250187 shutdown_connections 2024-02-19T16:29:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- 172.21.15.8:0/82250187 wait complete. 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- start start 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 Processor -- start 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffbf80 0x55d917011210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917011750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 0x55d917003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d917003fd0 con 0x55d917009b60 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d917004140 con 0x55d916ffbf80 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d917004300 con 0x55d916ffc8a0 2024-02-19T16:29:11.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.249+0000 7f37e8178740 10 client.0 _ll_register_callbacks cb 0x7f37c8013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 0x55d917003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917011750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 0x55d917003900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33340/0 (socket says 172.21.15.8:33340) 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37d7fff640 1 -- 172.21.15.8:0/3301812144 learned_addr learned my addr 172.21.15.8:0/3301812144 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37d7fff640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffbf80 0x55d917011210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 -- 172.21.15.8:0/3301812144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffbf80 msgr2=0x55d917011210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffbf80 0x55d917011210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 -- 172.21.15.8:0/3301812144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 msgr2=0x55d917003900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 0x55d917003900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 -- 172.21.15.8:0/3301812144 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d916f26060 con 0x55d916ffc8a0 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37d7fff640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 0x55d917003900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37deffd640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917011750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37d0011d10 tx=0x7f37d0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579b3e89c3e2125a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37c67fc640 1 -- 172.21.15.8:0/3301812144 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0029040 con 0x55d916ffc8a0 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d916ff20f0 con 0x55d916ffc8a0 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37c67fc640 1 -- 172.21.15.8:0/3301812144 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f37d0004f50 con 0x55d916ffc8a0 2024-02-19T16:29:11.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37c67fc640 1 -- 172.21.15.8:0/3301812144 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d0005670 con 0x55d916ffc8a0 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37e8178740 10 client.26881 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.250+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d916f626a0 con 0x55d916ffc8a0 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37c67fc640 1 -- 172.21.15.8:0/3301812144 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d916ff20f0 con 0x55d916ffc8a0 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37c67fc640 1 -- 172.21.15.8:0/3301812144 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37d002c070 con 0x55d916ffc8a0 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37e8178740 10 client.26881 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37e8178740 10 client.26881 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37e8178740 20 client.26881 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37e8178740 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d9170769d0 0x55d917078df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37e8178740 4 client.26881 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f37d0005500 con 0x55d9170769d0 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37d7fff640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d9170769d0 0x55d917078df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37c67fc640 10 client.26881 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.251+0000 7f37d7fff640 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d9170769d0 0x55d917078df0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca2a0f21f5ea25b0 server_cookie=295196613189c8b8 in_seq=0 out_seq=0 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37c67fc640 1 -- 172.21.15.8:0/3301812144 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f37d0005500 con 0x55d9170769d0 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37c67fc640 10 client.26881 handle_command_reply: tid=0 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 client.26881 shutdown 2024-02-19T16:29:11.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 msgr2=0x55d917011750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917011750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37d0011d10 tx=0x7f37d0005850 comp rx=0 tx=0).stop 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 shutdown_connections 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d9170769d0 0x55d917078df0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d917009b60 0x55d917003900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37c67fc640 0 client.26881 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d916ffc8a0 0x55d917011750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 --2- 172.21.15.8:0/3301812144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d916ffbf80 0x55d917011210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 >> 172.21.15.8:0/3301812144 conn(0x55d916ff6980 msgr2=0x55d916ff7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 shutdown_connections 2024-02-19T16:29:11.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 1 -- 172.21.15.8:0/3301812144 wait complete. 2024-02-19T16:29:11.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e8178740 20 client.26881 trim_cache size 0 max 16384 2024-02-19T16:29:11.266 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:29:11.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37cc0a2230 msgr2=0x7f37cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37cc0a2230 0x7f37cc0a46f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37d80ee860 tx=0x7f37d8086050 comp rx=0 tx=0).stop 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 msgr2=0x7f37e01978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 0x7f37e01978f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f37d8004af0 tx=0x7f37d8032570 comp rx=0 tx=0).stop 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 shutdown_connections 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37e0107500 0x7f37e01978f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37cc0a2230 0x7f37cc0a46f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e0069090 0x7f37e019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 --2- 172.21.15.8:0/3276759731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37e00686d0 0x7f37e019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 >> 172.21.15.8:0/3276759731 conn(0x7f37e00774f0 msgr2=0x7f37e00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.252+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 shutdown_connections 2024-02-19T16:29:11.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.253+0000 7f37e559a640 1 -- 172.21.15.8:0/3276759731 wait complete. 2024-02-19T16:29:11.269 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:29:11.269 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:29:11.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 -- 172.21.15.8:0/1927572487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a8102760 msgr2=0x7f22a8102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/1927572487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a8102760 0x7f22a8102b40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802bcd0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 -- 172.21.15.8:0/1927572487 shutdown_connections 2024-02-19T16:29:11.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/1927572487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a810bb30 0x7f22a810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/1927572487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22a8103080 0x7f22a810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/1927572487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a8102760 0x7f22a8102b40 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 -- 172.21.15.8:0/1927572487 >> 172.21.15.8:0/1927572487 conn(0x7f22a8077550 msgr2=0x7f22a8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 -- 172.21.15.8:0/1927572487 shutdown_connections 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.373+0000 7f22ad7e4640 1 -- 172.21.15.8:0/1927572487 wait complete. 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 Processor -- start 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 -- start start 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8102760 0x7f22a81003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22a8103080 0x7f22a81008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 0x7f22a8100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8102300 con 0x7f22a810bb30 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81aa500 con 0x7f22a8102760 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22ad7e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81aa640 con 0x7f22a8103080 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 0x7f22a8100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 0x7f22a8100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33354/0 (socket says 172.21.15.8:33354) 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 -- 172.21.15.8:0/582069675 learned_addr learned my addr 172.21.15.8:0/582069675 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8102760 0x7f22a81003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f229ffff640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22a8103080 0x7f22a81008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 -- 172.21.15.8:0/582069675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22a8103080 msgr2=0x7f22a81008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22a8103080 0x7f22a81008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 -- 172.21.15.8:0/582069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8102760 msgr2=0x7f22a81003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8102760 0x7f22a81003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.374+0000 7f22a77fe640 1 -- 172.21.15.8:0/582069675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009560 con 0x7f22a810bb30 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.375+0000 7f22a77fe640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 0x7f22a8100e30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f22a000eb10 tx=0x7f22a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.375+0000 7f22a4ff9640 1 -- 172.21.15.8:0/582069675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004b20 con 0x7f22a810bb30 2024-02-19T16:29:11.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.375+0000 7f22a4ff9640 1 -- 172.21.15.8:0/582069675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22a0004c80 con 0x7f22a810bb30 2024-02-19T16:29:11.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.375+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a81aa920 con 0x7f22a810bb30 2024-02-19T16:29:11.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.375+0000 7f22a4ff9640 1 -- 172.21.15.8:0/582069675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0010650 con 0x7f22a810bb30 2024-02-19T16:29:11.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.375+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81aade0 con 0x7f22a810bb30 2024-02-19T16:29:11.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.377+0000 7f22a4ff9640 1 -- 172.21.15.8:0/582069675 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f22a0005020 con 0x7f22a810bb30 2024-02-19T16:29:11.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.377+0000 7f22b03bd740 1 Processor -- start 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.377+0000 7f22b03bd740 1 -- start start 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f243f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f2448210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f2448750 0x5622f244ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22b03bd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f23fa960 con 0x5622f243fb70 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22b03bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f23ae310 con 0x5622f243f250 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22b03bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f23ab840 con 0x5622f2448750 2024-02-19T16:29:11.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a4ff9640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22940a2230 0x7f22940a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f2448210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f2448210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33362/0 (socket says 172.21.15.8:33362) 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 -- 172.21.15.8:0/2887376269 learned_addr learned my addr 172.21.15.8:0/2887376269 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f229ffff640 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f243f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a77fe640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22940a2230 0x7f22940a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f229ffff640 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f2448750 0x5622f244ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a4ff9640 1 -- 172.21.15.8:0/582069675 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f22a0014070 con 0x7f22a810bb30 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 -- 172.21.15.8:0/2887376269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f2448750 msgr2=0x5622f244ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f2448750 0x5622f244ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 -- 172.21.15.8:0/2887376269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 msgr2=0x5622f243f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f243f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 -- 172.21.15.8:0/2887376269 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622f2369060 con 0x5622f243fb70 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a77fe640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22940a2230 0x7f22940a46f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f22a009acf0 tx=0x7f22a001bb30 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.378+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f2448210 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245d4b7dc099b360 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f229d7fa640 1 -- 172.21.15.8:0/2887376269 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298004740 con 0x5622f243fb70 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f229d7fa640 1 -- 172.21.15.8:0/2887376269 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22980048a0 con 0x5622f243fb70 2024-02-19T16:29:11.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f229d7fa640 1 -- 172.21.15.8:0/2887376269 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298005030 con 0x5622f243fb70 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 -- 172.21.15.8:0/2887376269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 msgr2=0x5622f2448210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f2448210 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802c740 comp rx=0 tx=0).stop 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 -- 172.21.15.8:0/2887376269 shutdown_connections 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f2448750 0x5622f244ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f2448210 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 --2- 172.21.15.8:0/2887376269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f243f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 -- 172.21.15.8:0/2887376269 >> 172.21.15.8:0/2887376269 conn(0x5622f2439980 msgr2=0x5622f2439d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 -- 172.21.15.8:0/2887376269 shutdown_connections 2024-02-19T16:29:11.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.379+0000 7f22b03bd740 1 -- 172.21.15.8:0/2887376269 wait complete. 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 -- start start 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 Processor -- start 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f24456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243fb70 0x5622f2445c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2446150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f23a7020 con 0x5622f2448750 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f207a430 con 0x5622f243fb70 2024-02-19T16:29:11.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f2036e00 con 0x5622f243f250 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22b03bd740 10 client.0 _ll_register_callbacks cb 0x5622f234ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f229ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2446150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243fb70 0x5622f2445c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f229ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2446150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33370/0 (socket says 172.21.15.8:33370) 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f229ffff640 1 -- 172.21.15.8:0/3703076539 learned_addr learned my addr 172.21.15.8:0/3703076539 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.380+0000 7f229ffff640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f24456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22a6ffd640 1 -- 172.21.15.8:0/3703076539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 msgr2=0x5622f24456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f24456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22a6ffd640 1 -- 172.21.15.8:0/3703076539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 msgr2=0x5622f2446150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2446150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22a6ffd640 1 -- 172.21.15.8:0/3703076539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a810ec20 con 0x5622f243fb70 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f229ffff640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2446150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243fb70 0x5622f2445c10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f2298005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1a0edd49246787 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f228e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f228e7fc640 1 -- 172.21.15.8:0/3703076539 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298004740 con 0x5622f243fb70 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622f20a67e0 con 0x5622f243fb70 2024-02-19T16:29:11.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f228e7fc640 1 -- 172.21.15.8:0/3703076539 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f2298004df0 con 0x5622f243fb70 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f228e7fc640 1 -- 172.21.15.8:0/3703076539 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2298004fb0 con 0x5622f243fb70 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22b03bd740 10 client.26786 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5622f23a56a0 con 0x5622f243fb70 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f228e7fc640 1 -- 172.21.15.8:0/3703076539 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5622f20a67e0 con 0x5622f243fb70 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.381+0000 7f228e7fc640 1 -- 172.21.15.8:0/3703076539 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f229803c050 con 0x5622f243fb70 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f22b03bd740 10 client.26786 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f22b03bd740 10 client.26786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f22b03bd740 20 client.26786 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:11.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f22b03bd740 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24585a0 0x5622f245a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f22b03bd740 4 client.26786 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622f20e3720 con 0x5622f24585a0 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f229ffff640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24585a0 0x5622f245a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f229ffff640 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24585a0 0x5622f245a9c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80618bd006c0295b server_cookie=28b39fc1aeacdf7b in_seq=0 out_seq=0 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.382+0000 7f228e7fc640 10 client.26786 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f228e7fc640 1 -- 172.21.15.8:0/3703076539 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5622f20e3720 con 0x5622f24585a0 2024-02-19T16:29:11.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f228e7fc640 10 client.26786 handle_command_reply: tid=0 2024-02-19T16:29:11.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 client.26786 shutdown 2024-02-19T16:29:11.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243fb70 msgr2=0x5622f2445c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243fb70 0x5622f2445c10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f2298005fd0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 shutdown_connections 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24585a0 0x5622f245a9c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2446150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f228e7fc640 0 client.26786 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243fb70 0x5622f2445c10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 --2- 172.21.15.8:0/3703076539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f24456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 >> 172.21.15.8:0/3703076539 conn(0x5622f2439980 msgr2=0x5622f243b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 shutdown_connections 2024-02-19T16:29:11.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 1 -- 172.21.15.8:0/3703076539 wait complete. 2024-02-19T16:29:11.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.383+0000 7f22b03bd740 20 client.26786 trim_cache size 0 max 16384 2024-02-19T16:29:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.395+0000 7f22b03bd740 1 Processor -- start 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.395+0000 7f22b03bd740 1 -- start start 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f2447e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 0x5622f2434a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2453980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22b03bd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f232c130 con 0x5622f2448750 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22b03bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f242ba40 con 0x5622f243f250 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22b03bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f243a780 con 0x5622f243fb70 2024-02-19T16:29:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f229ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2453980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 0x5622f2434a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f229ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2453980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33376/0 (socket says 172.21.15.8:33376) 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 0x5622f2434a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:36570/0 (socket says 172.21.15.8:36570) 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f229ffff640 1 -- 172.21.15.8:0/160473450 learned_addr learned my addr 172.21.15.8:0/160473450 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f229ffff640 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f2447e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 -- 172.21.15.8:0/160473450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 msgr2=0x5622f2447e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f2447e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 -- 172.21.15.8:0/160473450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 msgr2=0x5622f2453980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2453980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 -- 172.21.15.8:0/160473450 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622f2341c20 con 0x5622f243fb70 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 0x5622f2434a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5622f2446ca0 tx=0x7f22980043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be91b4b2a598a569 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.396+0000 7f229cff9640 1 -- 172.21.15.8:0/160473450 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298004130 con 0x5622f243fb70 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f229cff9640 1 -- 172.21.15.8:0/160473450 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f229804e070 con 0x5622f243fb70 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f229cff9640 1 -- 172.21.15.8:0/160473450 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980050e0 con 0x5622f243fb70 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 -- 172.21.15.8:0/160473450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 msgr2=0x5622f2434a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 0x5622f2434a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5622f2446ca0 tx=0x7f22980043d0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 -- 172.21.15.8:0/160473450 shutdown_connections 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f2448750 0x5622f2453980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243fb70 0x5622f2434a10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 --2- 172.21.15.8:0/160473450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f243f250 0x5622f2447e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 -- 172.21.15.8:0/160473450 >> 172.21.15.8:0/160473450 conn(0x5622f2439980 msgr2=0x5622f2438b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 -- 172.21.15.8:0/160473450 shutdown_connections 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 -- 172.21.15.8:0/160473450 wait complete. 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.397+0000 7f22b03bd740 1 -- start start 2024-02-19T16:29:11.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 Processor -- start 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f244bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f244c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f2448750 0x5622f24b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f244cbc0 con 0x5622f243fb70 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f24b3390 con 0x5622f2448750 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f24b3500 con 0x5622f243f250 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 10 client.0 _ll_register_callbacks cb 0x7f2290009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f244c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f244c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33384/0 (socket says 172.21.15.8:33384) 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 -- 172.21.15.8:0/767438986 learned_addr learned my addr 172.21.15.8:0/767438986 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f229ffff640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f244bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f229ffff640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f2448750 0x5622f24b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 -- 172.21.15.8:0/767438986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 msgr2=0x5622f244bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f244bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 -- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f2448750 msgr2=0x5622f24b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f2448750 0x5622f24b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 -- 172.21.15.8:0/767438986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622f23a56a0 con 0x5622f243fb70 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f229ffff640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f2448750 0x5622f24b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22a6ffd640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f244c4a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f229802c210 tx=0x7f2298003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbd190df07bca41 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f228e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f228e7fc640 1 -- 172.21.15.8:0/767438986 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298064060 con 0x5622f243fb70 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f228e7fc640 1 -- 172.21.15.8:0/767438986 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f2298000e40 con 0x5622f243fb70 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.398+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622f24350f0 con 0x5622f243fb70 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f228e7fc640 1 -- 172.21.15.8:0/767438986 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2298003cb0 con 0x5622f243fb70 2024-02-19T16:29:11.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 10 client.17034 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5622f23678c0 con 0x5622f243fb70 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f228e7fc640 1 -- 172.21.15.8:0/767438986 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5622f24350f0 con 0x5622f243fb70 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f228e7fc640 1 -- 172.21.15.8:0/767438986 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f229803c070 con 0x5622f243fb70 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 10 client.17034 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 10 client.17034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 20 client.17034 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24b4900 0x5622f24b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 4 client.17034 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f22980048a0 con 0x5622f24b4900 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f229ffff640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24b4900 0x5622f24b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f228e7fc640 10 client.17034 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.399+0000 7f229ffff640 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24b4900 0x5622f24b6cf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4149e653303a568f server_cookie=e654e63a865e3acf in_seq=0 out_seq=0 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f228e7fc640 1 -- 172.21.15.8:0/767438986 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f22980048a0 con 0x5622f24b4900 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f228e7fc640 10 client.17034 handle_command_reply: tid=0 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 client.17034 shutdown 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 msgr2=0x5622f244c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f244c4a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f229802c210 tx=0x7f2298003ff0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 shutdown_connections 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x5622f24b4900 0x5622f24b6cf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622f2448750 0x5622f24b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f228e7fc640 0 client.17034 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5622f243fb70 0x5622f244c4a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 --2- 172.21.15.8:0/767438986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622f243f250 0x5622f244bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 >> 172.21.15.8:0/767438986 conn(0x5622f2439980 msgr2=0x5622f24454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 shutdown_connections 2024-02-19T16:29:11.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 1 -- 172.21.15.8:0/767438986 wait complete. 2024-02-19T16:29:11.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22b03bd740 20 client.17034 trim_cache size 0 max 16384 2024-02-19T16:29:11.417 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22940a2230 msgr2=0x7f22940a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22940a2230 0x7f22940a46f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f22a009acf0 tx=0x7f22a001bb30 comp rx=0 tx=0).stop 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 msgr2=0x7f22a8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 0x7f22a8100e30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f22a000eb10 tx=0x7f22a000c2d0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 shutdown_connections 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22a810bb30 0x7f22a8100e30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f22940a2230 0x7f22940a46f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22a8103080 0x7f22a81008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 --2- 172.21.15.8:0/582069675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22a8102760 0x7f22a81003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.400+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 >> 172.21.15.8:0/582069675 conn(0x7f22a8077550 msgr2=0x7f22a80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.401+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 shutdown_connections 2024-02-19T16:29:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.401+0000 7f22ad7e4640 1 -- 172.21.15.8:0/582069675 wait complete. 2024-02-19T16:29:11.420 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:29:11.421 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:29:11.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 -- 172.21.15.8:0/3616515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181056c0 msgr2=0x7f111810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 --2- 172.21.15.8:0/3616515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181056c0 0x7f111810d290 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1110009930 tx=0x7f111002bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 -- 172.21.15.8:0/3616515540 shutdown_connections 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 --2- 172.21.15.8:0/3616515540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181056c0 0x7f111810d290 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 --2- 172.21.15.8:0/3616515540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118100fa0 0x7f1118104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 --2- 172.21.15.8:0/3616515540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 0x7f11181009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 -- 172.21.15.8:0/3616515540 >> 172.21.15.8:0/3616515540 conn(0x7f11180774c0 msgr2=0x7f11180778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 -- 172.21.15.8:0/3616515540 shutdown_connections 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 -- 172.21.15.8:0/3616515540 wait complete. 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 Processor -- start 2024-02-19T16:29:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.525+0000 7f111fa88640 1 -- start start 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 0x7f1118196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11181056c0 0x7f1118196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 0x7f11181a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181a6300 con 0x7f11181971c0 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181a6440 con 0x7f11181005f0 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181a6580 con 0x7f11181056c0 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 0x7f11181a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 0x7f11181a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33400/0 (socket says 172.21.15.8:33400) 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 -- 172.21.15.8:0/1198786815 learned_addr learned my addr 172.21.15.8:0/1198786815 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111d824640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 0x7f1118196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111d023640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11181056c0 0x7f1118196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 -- 172.21.15.8:0/1198786815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11181056c0 msgr2=0x7f1118196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11181056c0 0x7f1118196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 -- 172.21.15.8:0/1198786815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 msgr2=0x7f1118196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 0x7f1118196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 -- 172.21.15.8:0/1198786815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009560 con 0x7f11181971c0 2024-02-19T16:29:11.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111d824640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 0x7f1118196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111e025640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 0x7f11181a5dc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1110009900 tx=0x7f111002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f1116ffd640 1 -- 172.21.15.8:0/1198786815 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100042d0 con 0x7f11181971c0 2024-02-19T16:29:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11181a6800 con 0x7f11181971c0 2024-02-19T16:29:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f1116ffd640 1 -- 172.21.15.8:0/1198786815 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1110004430 con 0x7f11181971c0 2024-02-19T16:29:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f1116ffd640 1 -- 172.21.15.8:0/1198786815 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110003bc0 con 0x7f11181971c0 2024-02-19T16:29:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.526+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11181a6d60 con 0x7f11181971c0 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.528+0000 7f1116ffd640 1 -- 172.21.15.8:0/1198786815 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f1110005460 con 0x7f11181971c0 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 Processor -- start 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 -- start start 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f1116ffd640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11000a2230 0x7f11000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 0x5604e88d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8882960 con 0x5604e88c7b70 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8836310 con 0x5604e88c7250 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f112266e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8833840 con 0x5604e88d0750 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.529+0000 7f1116ffd640 1 -- 172.21.15.8:0/1198786815 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f11100f03e0 con 0x7f11181971c0 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d023640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 0x5604e88d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d023640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 0x5604e88d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33404/0 (socket says 172.21.15.8:33404) 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d023640 1 -- 172.21.15.8:0/1514170837 learned_addr learned my addr 172.21.15.8:0/1514170837 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d023640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11000a2230 0x7f11000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111e025640 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 -- 172.21.15.8:0/1514170837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 msgr2=0x5604e88d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 -- 172.21.15.8:0/1514170837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 msgr2=0x5604e88d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 0x5604e88d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 -- 172.21.15.8:0/1514170837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e87f1060 con 0x5604e88c7250 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d023640 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 0x5604e88d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d023640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11000a2230 0x7f11000a46f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5604e88c6d20 tx=0x7f110c005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f111d824640 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88c7630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1108005fd0 tx=0x7f1108011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace1de780e247536 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f10fffff640 1 -- 172.21.15.8:0/1514170837 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108003c30 con 0x5604e88c7250 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f10fffff640 1 -- 172.21.15.8:0/1514170837 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f110800fe80 con 0x5604e88c7250 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.530+0000 7f10fffff640 1 -- 172.21.15.8:0/1514170837 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080193e0 con 0x5604e88c7250 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 -- 172.21.15.8:0/1514170837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 msgr2=0x5604e88c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88c7630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1108005fd0 tx=0x7f1108011c70 comp rx=0 tx=0).stop 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 -- 172.21.15.8:0/1514170837 shutdown_connections 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7b70 0x5604e88d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 --2- 172.21.15.8:0/1514170837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88c7630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 -- 172.21.15.8:0/1514170837 >> 172.21.15.8:0/1514170837 conn(0x5604e88c1980 msgr2=0x5604e88c1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 -- 172.21.15.8:0/1514170837 shutdown_connections 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.531+0000 7f112266e740 1 -- 172.21.15.8:0/1514170837 wait complete. 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 -- start start 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 Processor -- start 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88d0750 0x5604e88ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e882f020 con 0x5604e88c7250 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8502430 con 0x5604e88d0750 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e84bee00 con 0x5604e88c7b70 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f112266e740 10 client.0 _ll_register_callbacks cb 0x5604e87d6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33420/0 (socket says 172.21.15.8:33420) 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 -- 172.21.15.8:0/3899278407 learned_addr learned my addr 172.21.15.8:0/3899278407 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111e025640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88d0750 0x5604e88ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d023640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 -- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 msgr2=0x5604e88cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 -- 172.21.15.8:0/3899278407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88d0750 msgr2=0x5604e88ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88d0750 0x5604e88ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.532+0000 7f111d824640 1 -- 172.21.15.8:0/3899278407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1108011440 con 0x5604e88c7250 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f111d824640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cd770 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f11080045a0 tx=0x7f1108004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3636141b5b90c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f10fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f10fd7fa640 1 -- 172.21.15.8:0/3899278407 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108004a40 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f10fd7fa640 1 -- 172.21.15.8:0/3899278407 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f1108004ba0 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604e852e7e0 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f10fd7fa640 1 -- 172.21.15.8:0/3899278407 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1108004d60 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 10 client.17046 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5604e882d6a0 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f10fd7fa640 1 -- 172.21.15.8:0/3899278407 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5604e852e7e0 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f10fd7fa640 1 -- 172.21.15.8:0/3899278407 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1108005530 con 0x5604e88c7250 2024-02-19T16:29:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 10 client.17046 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 20 client.17046 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e88e0b60 0x5604e88e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 4 client.17046 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.533+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604e856b720 con 0x5604e88e0b60 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.534+0000 7f111d023640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e88e0b60 0x5604e88e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.534+0000 7f10fd7fa640 10 client.17046 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.534+0000 7f111d023640 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e88e0b60 0x5604e88e2f80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1018921b39de5ef7 server_cookie=cf2bd2d0cf8f4869 in_seq=0 out_seq=0 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f10fd7fa640 1 -- 172.21.15.8:0/3899278407 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5604e856b720 con 0x5604e88e0b60 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f10fd7fa640 10 client.17046 handle_command_reply: tid=0 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 client.17046 shutdown 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 msgr2=0x5604e88cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cd770 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f11080045a0 tx=0x7f1108004280 comp rx=0 tx=0).stop 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 shutdown_connections 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88d0750 0x5604e88ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e88e0b60 0x5604e88e2f80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 --2- 172.21.15.8:0/3899278407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cd770 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 >> 172.21.15.8:0/3899278407 conn(0x5604e88c1980 msgr2=0x5604e88c36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f10fd7fa640 0 client.17046 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 shutdown_connections 2024-02-19T16:29:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 1 -- 172.21.15.8:0/3899278407 wait complete. 2024-02-19T16:29:11.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.535+0000 7f112266e740 20 client.17046 trim_cache size 0 max 16384 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.547+0000 7f112266e740 1 Processor -- start 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 -- start start 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7b70 0x5604e88cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e87b4130 con 0x5604e88c7250 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e88cc5e0 con 0x5604e88c7b70 2024-02-19T16:29:11.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f112266e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e88e08e0 con 0x5604e88d0750 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33424/0 (socket says 172.21.15.8:33424) 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 -- 172.21.15.8:0/1732601489 learned_addr learned my addr 172.21.15.8:0/1732601489 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d023640 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7b70 0x5604e88cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111e025640 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 -- 172.21.15.8:0/1732601489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 msgr2=0x5604e88ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 -- 172.21.15.8:0/1732601489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7b70 msgr2=0x5604e88cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7b70 0x5604e88cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 -- 172.21.15.8:0/1732601489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e87c9c20 con 0x5604e88c7250 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f111d824640 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cfea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1108005b40 tx=0x7f11080092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0493f4470c63bc server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f10ff7fe640 1 -- 172.21.15.8:0/1732601489 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080042f0 con 0x5604e88c7250 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f10ff7fe640 1 -- 172.21.15.8:0/1732601489 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1108003e50 con 0x5604e88c7250 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.548+0000 7f10ff7fe640 1 -- 172.21.15.8:0/1732601489 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110801abc0 con 0x5604e88c7250 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 -- 172.21.15.8:0/1732601489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 msgr2=0x5604e88cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cfea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1108005b40 tx=0x7f11080092a0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 -- 172.21.15.8:0/1732601489 shutdown_connections 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88d0750 0x5604e88ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7b70 0x5604e88cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 --2- 172.21.15.8:0/1732601489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88c7250 0x5604e88cfea0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 -- 172.21.15.8:0/1732601489 >> 172.21.15.8:0/1732601489 conn(0x5604e88c1980 msgr2=0x5604e88c0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 -- 172.21.15.8:0/1732601489 shutdown_connections 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 -- 172.21.15.8:0/1732601489 wait complete. 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 -- start start 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.549+0000 7f112266e740 1 Processor -- start 2024-02-19T16:29:11.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 0x5604e88d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e893b5d0 con 0x5604e88d0750 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e893b740 con 0x5604e88c7250 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e893b900 con 0x5604e88c7b70 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f112266e740 10 client.0 _ll_register_callbacks cb 0x7f110c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 0x5604e88d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 0x5604e88d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33428/0 (socket says 172.21.15.8:33428) 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 -- 172.21.15.8:0/639893578 learned_addr learned my addr 172.21.15.8:0/639893578 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111d824640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111d023640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 -- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 msgr2=0x5604e88d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 -- 172.21.15.8:0/639893578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 msgr2=0x5604e88d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 -- 172.21.15.8:0/639893578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604e882d6a0 con 0x5604e88d0750 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f111e025640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 0x5604e88d4760 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f11100036a0 tx=0x7f1110005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec87678da927acb server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f10fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.550+0000 7f10fd7fa640 1 -- 172.21.15.8:0/639893578 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100c0bd0 con 0x5604e88d0750 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f10fd7fa640 1 -- 172.21.15.8:0/639893578 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f11100c5220 con 0x5604e88d0750 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f110800fe80 con 0x5604e88d0750 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f10fd7fa640 1 -- 172.21.15.8:0/639893578 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11100b90f0 con 0x5604e88d0750 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 10 client.17052 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:11.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5604e87ef8c0 con 0x5604e88d0750 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f10fd7fa640 1 -- 172.21.15.8:0/639893578 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f110800fe80 con 0x5604e88d0750 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f10fd7fa640 1 -- 172.21.15.8:0/639893578 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f111003c070 con 0x5604e88d0750 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 10 client.17052 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 20 client.17052 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e893cd50 0x5604e893f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 4 client.17052 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604e85ba890 con 0x5604e893cd50 2024-02-19T16:29:11.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f111d824640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e893cd50 0x5604e893f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f10fd7fa640 10 client.17052 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.551+0000 7f111d824640 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e893cd50 0x5604e893f170 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a679b2fe65c86f5f server_cookie=e151a5ea69846ce5 in_seq=0 out_seq=0 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f10fd7fa640 1 -- 172.21.15.8:0/639893578 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604e85ba890 con 0x5604e893cd50 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f10fd7fa640 10 client.17052 handle_command_reply: tid=0 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 client.17052 shutdown 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 msgr2=0x5604e88d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 0x5604e88d4760 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f11100036a0 tx=0x7f1110005760 comp rx=0 tx=0).stop 2024-02-19T16:29:11.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 shutdown_connections 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5604e88d0750 0x5604e88d4760 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5604e893cd50 0x5604e893f170 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5604e88c7b70 0x5604e88d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 --2- 172.21.15.8:0/639893578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604e88c7250 0x5604e88d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f10fd7fa640 0 client.17052 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 >> 172.21.15.8:0/639893578 conn(0x5604e88c1980 msgr2=0x5604e88c0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 shutdown_connections 2024-02-19T16:29:11.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 1 -- 172.21.15.8:0/639893578 wait complete. 2024-02-19T16:29:11.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.552+0000 7f112266e740 20 client.17052 trim_cache size 0 max 16384 2024-02-19T16:29:11.566 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:29:11.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11000a2230 msgr2=0x7f11000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11000a2230 0x7f11000a46f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5604e88c6d20 tx=0x7f110c005f90 comp rx=0 tx=0).stop 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 msgr2=0x7f11181a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 0x7f11181a5dc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1110009900 tx=0x7f111002c5d0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 shutdown_connections 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f11181971c0 0x7f11181a5dc0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f11000a2230 0x7f11000a46f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11181056c0 0x7f1118196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 --2- 172.21.15.8:0/1198786815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11181005f0 0x7f1118196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:11.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 >> 172.21.15.8:0/1198786815 conn(0x7f11180774c0 msgr2=0x7f11180fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:11.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 shutdown_connections 2024-02-19T16:29:11.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:11.553+0000 7f111fa88640 1 -- 172.21.15.8:0/1198786815 wait complete. 2024-02-19T16:29:11.769 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:11 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2467287773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:29:11.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:11 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2467287773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:29:11.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:11 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2467287773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:29:12.570 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:29:12.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:29:12.571 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:29:12.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 -- 172.21.15.8:0/1536204968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 msgr2=0x7f9b801078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 --2- 172.21.15.8:0/1536204968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b801078e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009930 tx=0x7f9b7002bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:12.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 -- 172.21.15.8:0/1536204968 shutdown_connections 2024-02-19T16:29:12.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 --2- 172.21.15.8:0/1536204968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b80069090 0x7f9b8010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 --2- 172.21.15.8:0/1536204968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b800686d0 0x7f9b80068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 --2- 172.21.15.8:0/1536204968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b801078e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 -- 172.21.15.8:0/1536204968 >> 172.21.15.8:0/1536204968 conn(0x7f9b800774f0 msgr2=0x7f9b80077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 -- 172.21.15.8:0/1536204968 shutdown_connections 2024-02-19T16:29:12.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 -- 172.21.15.8:0/1536204968 wait complete. 2024-02-19T16:29:12.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.689+0000 7f9b8624e640 1 Processor -- start 2024-02-19T16:29:12.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 -- start start 2024-02-19T16:29:12.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b800686d0 0x7f9b80199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 0x7f9b8019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b8019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019ec10 con 0x7f9b80107500 2024-02-19T16:29:12.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019ed80 con 0x7f9b800686d0 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b8624e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019ef40 con 0x7f9b80069090 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b8019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 0x7f9b8019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 0x7f9b8019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:36672/0 (socket says 172.21.15.8:36672) 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b800686d0 0x7f9b80199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 -- 172.21.15.8:0/579001916 learned_addr learned my addr 172.21.15.8:0/579001916 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 -- 172.21.15.8:0/579001916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b800686d0 msgr2=0x7f9b80199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b800686d0 0x7f9b80199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 -- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 msgr2=0x7f9b8019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b8019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7effd640 1 -- 172.21.15.8:0/579001916 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70009560 con 0x7f9b80069090 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.690+0000 7f9b7ffff640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b8019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.691+0000 7f9b7effd640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 0x7f9b8019a290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400e9f0 tx=0x7f9b7400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.691+0000 7f9b7cff9640 1 -- 172.21.15.8:0/579001916 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74009720 con 0x7f9b80069090 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.691+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8019f220 con 0x7f9b80069090 2024-02-19T16:29:12.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.691+0000 7f9b7cff9640 1 -- 172.21.15.8:0/579001916 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b7400cd80 con 0x7f9b80069090 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.691+0000 7f9b7cff9640 1 -- 172.21.15.8:0/579001916 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74010690 con 0x7f9b80069090 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.691+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b801a6aa0 con 0x7f9b80069090 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.693+0000 7f9b7cff9640 1 -- 172.21.15.8:0/579001916 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f9b740046c0 con 0x7f9b80069090 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.693+0000 7f9b7cff9640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9b680a2230 0x7f9b680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.693+0000 7f9b88e22740 1 Processor -- start 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.693+0000 7f9b88e22740 1 -- start start 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.693+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9b680a2230 0x7f9b680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.693+0000 7f9b7cff9640 1 -- 172.21.15.8:0/579001916 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f9b74014070 con 0x7f9b80069090 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559528925380 0x559528925760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 0x559528930d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b88e22740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559528893f90 con 0x55952892e8b0 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b88e22740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595288914c0 con 0x559528925380 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b88e22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952888cca0 con 0x559528925d30 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 0x559528930d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559528925380 0x559528925760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 0x559528930d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33442/0 (socket says 172.21.15.8:33442) 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 -- 172.21.15.8:0/2083733831 learned_addr learned my addr 172.21.15.8:0/2083733831 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9b680a2230 0x7f9b680a46f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005ab0 tx=0x7f9b70005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 -- 172.21.15.8:0/2083733831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 msgr2=0x55952892e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 -- 172.21.15.8:0/2083733831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559528925380 msgr2=0x559528925760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559528925380 0x559528925760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 -- 172.21.15.8:0/2083733831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55952884f0b0 con 0x55952892e8b0 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b7ffff640 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 0x559528930d00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9b78011790 tx=0x7f9b78011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986b689b0595f5b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b65ffb640 1 -- 172.21.15.8:0/2083733831 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011ea0 con 0x55952892e8b0 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b65ffb640 1 -- 172.21.15.8:0/2083733831 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b7800fe80 con 0x55952892e8b0 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.694+0000 7f9b65ffb640 1 -- 172.21.15.8:0/2083733831 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780047d0 con 0x55952892e8b0 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 -- 172.21.15.8:0/2083733831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 msgr2=0x559528930d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 0x559528930d00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9b78011790 tx=0x7f9b78011c50 comp rx=0 tx=0).stop 2024-02-19T16:29:12.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 -- 172.21.15.8:0/2083733831 shutdown_connections 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55952892e8b0 0x559528930d00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2083733831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559528925380 0x559528925760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 -- 172.21.15.8:0/2083733831 >> 172.21.15.8:0/2083733831 conn(0x55952891f840 msgr2=0x55952891fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 -- 172.21.15.8:0/2083733831 shutdown_connections 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 -- 172.21.15.8:0/2083733831 wait complete. 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.695+0000 7f9b88e22740 1 -- start start 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 Processor -- start 2024-02-19T16:29:12.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x55952892b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x55952892bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x55952892c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559528831f40 con 0x559528925d30 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595285604b0 con 0x55952892e8b0 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559528812670 con 0x559528925380 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b88e22740 10 client.0 _ll_register_callbacks cb 0x55952887ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x55952892bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x55952892bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33444/0 (socket says 172.21.15.8:33444) 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 -- 172.21.15.8:0/4120112150 learned_addr learned my addr 172.21.15.8:0/4120112150 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x55952892b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7ffff640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x55952892c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 -- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 msgr2=0x55952892b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x55952892b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 -- 172.21.15.8:0/4120112150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 msgr2=0x55952892c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x55952892c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.696+0000 7f9b7effd640 1 -- 172.21.15.8:0/4120112150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b78011440 con 0x559528925d30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b7effd640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x55952892bd60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c6a0 tx=0x7f9b7409b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f923d0a25821482a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b5effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b5effd640 1 -- 172.21.15.8:0/4120112150 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740a3120 con 0x559528925d30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b5effd640 1 -- 172.21.15.8:0/4120112150 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f9b740a3280 con 0x559528925d30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559528866f40 con 0x559528925d30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b5effd640 1 -- 172.21.15.8:0/4120112150 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b740b3b40 con 0x559528925d30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 10 client.17064 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55952888b320 con 0x559528925d30 2024-02-19T16:29:12.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b5effd640 1 -- 172.21.15.8:0/4120112150 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559528866f40 con 0x559528925d30 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b5effd640 1 -- 172.21.15.8:0/4120112150 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b7401f060 con 0x559528925d30 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 10 client.17064 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 20 client.17064 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952893eea0 0x559528941290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 4 client.17064 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.697+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559528883310 con 0x55952893eea0 2024-02-19T16:29:12.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.698+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952893eea0 0x559528941290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.698+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952893eea0 0x559528941290 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a6cb943e663866 server_cookie=f227a16fc714c4e7 in_seq=0 out_seq=0 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.698+0000 7f9b5effd640 10 client.17064 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.698+0000 7f9b5effd640 1 -- 172.21.15.8:0/4120112150 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559528883310 con 0x55952893eea0 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.698+0000 7f9b5effd640 10 client.17064 handle_command_reply: tid=0 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 client.17064 shutdown 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 msgr2=0x55952892bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x55952892bd60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c6a0 tx=0x7f9b7409b650 comp rx=0 tx=0).stop 2024-02-19T16:29:12.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 shutdown_connections 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x55952892c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x55952892bd60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952893eea0 0x559528941290 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 --2- 172.21.15.8:0/4120112150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x55952892b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b5effd640 0 client.17064 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 >> 172.21.15.8:0/4120112150 conn(0x55952891f840 msgr2=0x5595289217a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 shutdown_connections 2024-02-19T16:29:12.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 1 -- 172.21.15.8:0/4120112150 wait complete. 2024-02-19T16:29:12.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.699+0000 7f9b88e22740 20 client.17064 trim_cache size 0 max 16384 2024-02-19T16:29:12.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 Processor -- start 2024-02-19T16:29:12.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 -- start start 2024-02-19T16:29:12.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 0x55952892df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528939db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952891b180 con 0x559528925380 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559528919dc0 con 0x55952892e8b0 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.715+0000 7f9b88e22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595289251f0 con 0x559528925d30 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 0x55952892df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528939db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 0x55952892df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33456/0 (socket says 172.21.15.8:33456) 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 -- 172.21.15.8:0/673461839 learned_addr learned my addr 172.21.15.8:0/673461839 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7effd640 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 -- 172.21.15.8:0/673461839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 msgr2=0x55952892c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 -- 172.21.15.8:0/673461839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 msgr2=0x559528939db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528939db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 -- 172.21.15.8:0/673461839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559528827c00 con 0x559528925380 2024-02-19T16:29:12.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7ffff640 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528939db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:12.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 0x55952892df50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9b70036040 tx=0x7f9b70044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15ad5bf1fa538e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b657fa640 1 -- 172.21.15.8:0/673461839 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70053070 con 0x559528925380 2024-02-19T16:29:12.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b657fa640 1 -- 172.21.15.8:0/673461839 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b7002c9e0 con 0x559528925380 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.716+0000 7f9b657fa640 1 -- 172.21.15.8:0/673461839 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70005270 con 0x559528925380 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 -- 172.21.15.8:0/673461839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 msgr2=0x55952892df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 0x55952892df50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9b70036040 tx=0x7f9b70044000 comp rx=0 tx=0).stop 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 -- 172.21.15.8:0/673461839 shutdown_connections 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528939db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925d30 0x55952892c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 --2- 172.21.15.8:0/673461839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925380 0x55952892df50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 -- 172.21.15.8:0/673461839 >> 172.21.15.8:0/673461839 conn(0x55952891f840 msgr2=0x55952891e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 -- 172.21.15.8:0/673461839 shutdown_connections 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 -- 172.21.15.8:0/673461839 wait complete. 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 -- start start 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.717+0000 7f9b88e22740 1 Processor -- start 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x559528936440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x5595289327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528932d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559528933240 con 0x559528925d30 2024-02-19T16:29:12.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595289333b0 con 0x55952892e8b0 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559528933570 con 0x559528925380 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 10 client.0 _ll_register_callbacks cb 0x7f9b7400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x5595289327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x5595289327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33462/0 (socket says 172.21.15.8:33462) 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 -- 172.21.15.8:0/2749661434 learned_addr learned my addr 172.21.15.8:0/2749661434 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 -- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 msgr2=0x559528936440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x559528936440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 -- 172.21.15.8:0/2749661434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 msgr2=0x559528932d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528932d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 -- 172.21.15.8:0/2749661434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7801e040 con 0x559528925d30 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b7effd640 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x5595289327c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9b7409ada0 tx=0x7f9b74097700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b348cc6313a482b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b5effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b5effd640 1 -- 172.21.15.8:0/2749661434 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740a3320 con 0x559528925d30 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b5effd640 1 -- 172.21.15.8:0/2749661434 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f9b740b3b40 con 0x559528925d30 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595288b2900 con 0x559528925d30 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b5effd640 1 -- 172.21.15.8:0/2749661434 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b7409a5b0 con 0x559528925d30 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 10 client.17070 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:12.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.718+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f9b8010ec30 con 0x559528925d30 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b5effd640 1 -- 172.21.15.8:0/2749661434 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5595288b2900 con 0x559528925d30 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b5effd640 1 -- 172.21.15.8:0/2749661434 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b7401f040 con 0x559528925d30 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b88e22740 10 client.17070 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b88e22740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b88e22740 20 client.17070 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952899b980 0x55952899dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b88e22740 4 client.17070 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:29:12.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55952858c860 con 0x55952899b980 2024-02-19T16:29:12.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952899b980 0x55952899dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b7f7fe640 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952899b980 0x55952899dda0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5eee057d3ddafd7 server_cookie=960a8356c041e199 in_seq=0 out_seq=0 2024-02-19T16:29:12.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.719+0000 7f9b5effd640 10 client.17070 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b5effd640 1 -- 172.21.15.8:0/2749661434 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55952858c860 con 0x55952899b980 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b5effd640 10 client.17070 handle_command_reply: tid=0 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 client.17070 shutdown 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 msgr2=0x5595289327c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x5595289327c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9b7409ada0 tx=0x7f9b74097700 comp rx=0 tx=0).stop 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 shutdown_connections 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55952892e8b0 0x559528932d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559528925d30 0x5595289327c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55952899b980 0x55952899dda0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 --2- 172.21.15.8:0/2749661434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559528925380 0x559528936440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b5effd640 0 client.17070 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 >> 172.21.15.8:0/2749661434 conn(0x55952891f840 msgr2=0x559528945b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 shutdown_connections 2024-02-19T16:29:12.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 1 -- 172.21.15.8:0/2749661434 wait complete. 2024-02-19T16:29:12.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.721+0000 7f9b88e22740 20 client.17070 trim_cache size 0 max 16384 2024-02-19T16:29:12.738 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:29:12.739 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:29:12.739 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "1595b432-d589-45d7-b402-7fed3879e253", 2024-02-19T16:29:12.739 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:29:12.739 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:29:12.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9b680a2230 msgr2=0x7f9b680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9b680a2230 0x7f9b680a46f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005ab0 tx=0x7f9b70005fd0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 msgr2=0x7f9b8019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 0x7f9b8019a290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400e9f0 tx=0x7f9b7400edf0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 shutdown_connections 2024-02-19T16:29:12.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b80107500 0x7f9b8019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9b680a2230 0x7f9b680a46f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b80069090 0x7f9b8019a290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 --2- 172.21.15.8:0/579001916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b800686d0 0x7f9b80199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 >> 172.21.15.8:0/579001916 conn(0x7f9b800774f0 msgr2=0x7f9b8010cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 shutdown_connections 2024-02-19T16:29:12.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.722+0000 7f9b8624e640 1 -- 172.21.15.8:0/579001916 wait complete. 2024-02-19T16:29:12.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:29:12.744 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:29:12.768 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:12 smithi008 ceph-mon[32573]: pgmap v193: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-02-19T16:29:12.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:12 smithi191 ceph-mon[37203]: pgmap v193: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-02-19T16:29:12.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:12 smithi156 ceph-mon[36502]: pgmap v193: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-02-19T16:29:12.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 -- 172.21.15.8:0/3061592682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8190100f90 msgr2=0x7f8190104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 --2- 172.21.15.8:0/3061592682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8190100f90 0x7f8190104fd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8180009930 tx=0x7f818002bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:12.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 -- 172.21.15.8:0/3061592682 shutdown_connections 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 --2- 172.21.15.8:0/3061592682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901056b0 0x7f819010d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 --2- 172.21.15.8:0/3061592682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8190100f90 0x7f8190104fd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 --2- 172.21.15.8:0/3061592682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81901005e0 0x7f81901009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 -- 172.21.15.8:0/3061592682 >> 172.21.15.8:0/3061592682 conn(0x7f81900775a0 msgr2=0x7f81900779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 -- 172.21.15.8:0/3061592682 shutdown_connections 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.848+0000 7f8197d25640 1 -- 172.21.15.8:0/3061592682 wait complete. 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 Processor -- start 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 -- start start 2024-02-19T16:29:12.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 0x7f819019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81901056b0 0x7f819019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901978f0 0x7f8190197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81901982e0 con 0x7f81901005e0 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8190198450 con 0x7f81901978f0 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8197d25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8190198610 con 0x7f81901056b0 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 0x7f819019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 0x7f819019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33486/0 (socket says 172.21.15.8:33486) 2024-02-19T16:29:12.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f8195ac1640 1 -- 172.21.15.8:0/3962755963 learned_addr learned my addr 172.21.15.8:0/3962755963 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f81962c2640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901978f0 0x7f8190197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.849+0000 7f81952c0640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81901056b0 0x7f819019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8195ac1640 1 -- 172.21.15.8:0/3962755963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81901056b0 msgr2=0x7f819019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8195ac1640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81901056b0 0x7f819019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8195ac1640 1 -- 172.21.15.8:0/3962755963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901978f0 msgr2=0x7f8190197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8195ac1640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901978f0 0x7f8190197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8195ac1640 1 -- 172.21.15.8:0/3962755963 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009560 con 0x7f81901005e0 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f81962c2640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901978f0 0x7f8190197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:12.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8195ac1640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 0x7f819019d740 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f818c00e9f0 tx=0x7f818c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8186ffd640 1 -- 172.21.15.8:0/3962755963 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c009700 con 0x7f81901005e0 2024-02-19T16:29:12.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8186ffd640 1 -- 172.21.15.8:0/3962755963 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f818c00cd80 con 0x7f81901005e0 2024-02-19T16:29:12.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81901a4600 con 0x7f81901005e0 2024-02-19T16:29:12.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8186ffd640 1 -- 172.21.15.8:0/3962755963 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c018920 con 0x7f81901005e0 2024-02-19T16:29:12.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.850+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81901a4b30 con 0x7f81901005e0 2024-02-19T16:29:12.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.852+0000 7f8186ffd640 1 -- 172.21.15.8:0/3962755963 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f818c018a80 con 0x7f81901005e0 2024-02-19T16:29:12.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 Processor -- start 2024-02-19T16:29:12.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 -- start start 2024-02-19T16:29:12.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f8186ffd640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f817c0a22b0 0x7f817c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc87f280 con 0x55c2cc90ffe0 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc87c7b0 con 0x55c2cc910900 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f819a9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc877f90 con 0x55c2cc91dbc0 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f81962c2640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f817c0a22b0 0x7f817c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f81952c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.853+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9103c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33496/0 (socket says 172.21.15.8:33496) 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 -- 172.21.15.8:0/1290297470 learned_addr learned my addr 172.21.15.8:0/1290297470 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f81962c2640 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8186ffd640 1 -- 172.21.15.8:0/3962755963 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f818c014070 con 0x7f81901005e0 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 -- 172.21.15.8:0/1290297470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 msgr2=0x55c2cc91ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 -- 172.21.15.8:0/1290297470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 msgr2=0x55c2cc91d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 -- 172.21.15.8:0/1290297470 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2cc836570 con 0x55c2cc90ffe0 2024-02-19T16:29:12.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f81952c0640 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f81962c2640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f817c0a22b0 0x7f817c0a4770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81880099e0 tx=0x7f8188009ef0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f8195ac1640 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9103c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f818c0a3010 tx=0x7f818c09ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8444b74d65ab74e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f817bfff640 1 -- 172.21.15.8:0/1290297470 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c09bed0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f817bfff640 1 -- 172.21.15.8:0/1290297470 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f818c09b550 con 0x55c2cc90ffe0 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.854+0000 7f817bfff640 1 -- 172.21.15.8:0/1290297470 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c09eef0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 -- 172.21.15.8:0/1290297470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 msgr2=0x55c2cc9103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9103c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f818c0a3010 tx=0x7f818c09ad50 comp rx=0 tx=0).stop 2024-02-19T16:29:12.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 -- 172.21.15.8:0/1290297470 shutdown_connections 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1290297470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9103c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 -- 172.21.15.8:0/1290297470 >> 172.21.15.8:0/1290297470 conn(0x55c2cc90a990 msgr2=0x55c2cc90ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 -- 172.21.15.8:0/1290297470 shutdown_connections 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 -- 172.21.15.8:0/1290297470 wait complete. 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 -- start start 2024-02-19T16:29:12.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.855+0000 7f819a9d8740 1 Processor -- start 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc91d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 0x55c2cc91a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc54b430 con 0x55c2cc91dbc0 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc507e00 con 0x55c2cc90ffe0 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc7fcfd0 con 0x55c2cc910900 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f819a9d8740 10 client.0 _ll_register_callbacks cb 0x55c2cc852fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f81952c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 0x55c2cc91a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc91d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f81962c2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f81952c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 0x55c2cc91a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:36714/0 (socket says 172.21.15.8:36714) 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f81952c0640 1 -- 172.21.15.8:0/156916565 learned_addr learned my addr 172.21.15.8:0/156916565 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f8195ac1640 1 -- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 msgr2=0x55c2cc91a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f8195ac1640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 0x55c2cc91a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f8195ac1640 1 -- 172.21.15.8:0/156916565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc91dbc0 msgr2=0x55c2cc91af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f8195ac1640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f8195ac1640 1 -- 172.21.15.8:0/156916565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81901a4b30 con 0x55c2cc90ffe0 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.856+0000 7f81962c2640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81952c0640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 0x55c2cc91a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f8195ac1640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc91d5b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f818c09b8f0 tx=0x7f818c09af40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7029f73ae5a6635a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81797fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81797fa640 1 -- 172.21.15.8:0/156916565 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0a3540 con 0x55c2cc90ffe0 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2cc5777e0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81797fa640 1 -- 172.21.15.8:0/156916565 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f818c09eef0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81797fa640 1 -- 172.21.15.8:0/156916565 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f818c09ec90 con 0x55c2cc90ffe0 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f819a9d8740 10 client.26837 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c2cc876610 con 0x55c2cc90ffe0 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81797fa640 1 -- 172.21.15.8:0/156916565 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c2cc5777e0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.857+0000 7f81797fa640 1 -- 172.21.15.8:0/156916565 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f818c0974c0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f819a9d8740 10 client.26837 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f819a9d8740 10 client.26837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f819a9d8740 20 client.26837 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:12.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f819a9d8740 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc92d990 0x55c2cc92fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f819a9d8740 4 client.26837 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2cc5b4720 con 0x55c2cc92d990 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f81952c0640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc92d990 0x55c2cc92fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f81952c0640 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc92d990 0x55c2cc92fdb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f640aa56c5df90e1 server_cookie=c63ad5d8e873378d in_seq=0 out_seq=0 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.858+0000 7f81797fa640 10 client.26837 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.859+0000 7f81797fa640 1 -- 172.21.15.8:0/156916565 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2cc5b4720 con 0x55c2cc92d990 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.859+0000 7f81797fa640 10 client.26837 handle_command_reply: tid=0 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 client.26837 shutdown 2024-02-19T16:29:12.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 msgr2=0x55c2cc91d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc91d5b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f818c09b8f0 tx=0x7f818c09af40 comp rx=0 tx=0).stop 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 shutdown_connections 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc92d990 0x55c2cc92fdb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc910900 0x55c2cc91a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 --2- 172.21.15.8:0/156916565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc91d5b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f81797fa640 0 client.26837 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 >> 172.21.15.8:0/156916565 conn(0x55c2cc90a990 msgr2=0x55c2cc90c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 shutdown_connections 2024-02-19T16:29:12.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 1 -- 172.21.15.8:0/156916565 wait complete. 2024-02-19T16:29:12.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.860+0000 7f819a9d8740 20 client.26837 trim_cache size 0 max 16384 2024-02-19T16:29:12.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 Processor -- start 2024-02-19T16:29:12.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 -- start start 2024-02-19T16:29:12.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc919210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc904eb0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc919750 con 0x55c2cc910900 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f819a9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc90f620 con 0x55c2cc91dbc0 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc919210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc919210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33518/0 (socket says 172.21.15.8:33518) 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 -- 172.21.15.8:0/2710412819 learned_addr learned my addr 172.21.15.8:0/2710412819 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f81952c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f81962c2640 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 -- 172.21.15.8:0/2710412819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 msgr2=0x55c2cc91d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 -- 172.21.15.8:0/2710412819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 msgr2=0x55c2cc91d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 -- 172.21.15.8:0/2710412819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2cc812ac0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.879+0000 7f8195ac1640 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc919210 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f818c09ab50 tx=0x7f818c09ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b53b6cc7cbf435c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f817b7fe640 1 -- 172.21.15.8:0/2710412819 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c09a7b0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f817b7fe640 1 -- 172.21.15.8:0/2710412819 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f818c09eef0 con 0x55c2cc90ffe0 2024-02-19T16:29:12.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f817b7fe640 1 -- 172.21.15.8:0/2710412819 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c099150 con 0x55c2cc90ffe0 2024-02-19T16:29:12.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f819a9d8740 1 -- 172.21.15.8:0/2710412819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 msgr2=0x55c2cc919210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f819a9d8740 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc919210 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f818c09ab50 tx=0x7f818c09ad50 comp rx=0 tx=0).stop 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f819a9d8740 1 -- 172.21.15.8:0/2710412819 shutdown_connections 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f819a9d8740 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc91d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f819a9d8740 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc910900 0x55c2cc91d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.880+0000 7f819a9d8740 1 --2- 172.21.15.8:0/2710412819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc90ffe0 0x55c2cc919210 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- 172.21.15.8:0/2710412819 >> 172.21.15.8:0/2710412819 conn(0x55c2cc90a990 msgr2=0x55c2cc91a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- 172.21.15.8:0/2710412819 shutdown_connections 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- 172.21.15.8:0/2710412819 wait complete. 2024-02-19T16:29:12.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- start start 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 Processor -- start 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 0x55c2cc9176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc917c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc924bd0 con 0x55c2cc910900 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc924d40 con 0x55c2cc90ffe0 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cc924f00 con 0x55c2cc91dbc0 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f819a9d8740 10 client.0 _ll_register_callbacks cb 0x7f818c01a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:12.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f81952c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 0x55c2cc9176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f81952c0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 0x55c2cc9176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33528/0 (socket says 172.21.15.8:33528) 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.881+0000 7f81952c0640 1 -- 172.21.15.8:0/1870098844 learned_addr learned my addr 172.21.15.8:0/1870098844 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f8195ac1640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81962c2640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc917c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81952c0640 1 -- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 msgr2=0x55c2cc917c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81952c0640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc917c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81952c0640 1 -- 172.21.15.8:0/1870098844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 msgr2=0x55c2cc9171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:12.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81952c0640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81952c0640 1 -- 172.21.15.8:0/1870098844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2cc876610 con 0x55c2cc910900 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81952c0640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 0x55c2cc9176f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8180036040 tx=0x7f818002c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926f4ccdc87bff4c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81797fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81797fa640 1 -- 172.21.15.8:0/1870098844 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180045040 con 0x55c2cc910900 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81797fa640 1 -- 172.21.15.8:0/1870098844 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f818002cc40 con 0x55c2cc910900 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f818c09b550 con 0x55c2cc910900 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f81797fa640 1 -- 172.21.15.8:0/1870098844 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8180049300 con 0x55c2cc910900 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f819a9d8740 10 client.17088 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:12.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.882+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c2cc8386a0 con 0x55c2cc910900 2024-02-19T16:29:12.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f81797fa640 1 -- 172.21.15.8:0/1870098844 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f818c09b550 con 0x55c2cc910900 2024-02-19T16:29:12.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f81797fa640 1 -- 172.21.15.8:0/1870098844 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f818003c040 con 0x55c2cc910900 2024-02-19T16:29:12.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f819a9d8740 10 client.17088 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f819a9d8740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f819a9d8740 20 client.17088 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc989e60 0x55c2cc98c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f819a9d8740 4 client.17088 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2cc603890 con 0x55c2cc989e60 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.883+0000 7f8195ac1640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc989e60 0x55c2cc98c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:12.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f8195ac1640 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc989e60 0x55c2cc98c280 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3609493c5f67a90 server_cookie=89dc8d0db5f1075b in_seq=0 out_seq=0 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f81797fa640 10 client.17088 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f81797fa640 1 -- 172.21.15.8:0/1870098844 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55c2cc603890 con 0x55c2cc989e60 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f81797fa640 10 client.17088 handle_command_reply: tid=0 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 client.17088 shutdown 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 msgr2=0x55c2cc9176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 0x55c2cc9176f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8180036040 tx=0x7f818002c9c0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 shutdown_connections 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c2cc989e60 0x55c2cc98c280 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2cc91dbc0 0x55c2cc917c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2cc910900 0x55c2cc9176f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 --2- 172.21.15.8:0/1870098844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c2cc90ffe0 0x55c2cc9171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f81797fa640 0 client.17088 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.884+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 >> 172.21.15.8:0/1870098844 conn(0x55c2cc90a990 msgr2=0x55c2cc91a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 shutdown_connections 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f819a9d8740 1 -- 172.21.15.8:0/1870098844 wait complete. 2024-02-19T16:29:12.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f819a9d8740 20 client.17088 trim_cache size 0 max 16384 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2753 inodes in the stack)", 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout: "1595b432-d589-45d7-b402-7fed3879e253": { 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "1595b432-d589-45d7-b402-7fed3879e253", 2024-02-19T16:29:12.896 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:29:12.897 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:29:12.897 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:29:12.897 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:29:12.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f817c0a22b0 msgr2=0x7f817c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f817c0a22b0 0x7f817c0a4770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81880099e0 tx=0x7f8188009ef0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 msgr2=0x7f819019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:12.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 0x7f819019d740 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f818c00e9f0 tx=0x7f818c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 shutdown_connections 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81901978f0 0x7f8190197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f817c0a22b0 0x7f817c0a4770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81901056b0 0x7f819019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 --2- 172.21.15.8:0/3962755963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81901005e0 0x7f819019d740 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 >> 172.21.15.8:0/3962755963 conn(0x7f81900775a0 msgr2=0x7f81900fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 shutdown_connections 2024-02-19T16:29:12.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:12.885+0000 7f8197d25640 1 -- 172.21.15.8:0/3962755963 wait complete. 2024-02-19T16:29:12.900 INFO:tasks.cephfs.filesystem:scrub status for tag:1595b432-d589-45d7-b402-7fed3879e253 - {'path': '/', 'tag': '1595b432-d589-45d7-b402-7fed3879e253', 'options': 'recursive,force'} 2024-02-19T16:29:13.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:13 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:29:13.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:13 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:13.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:13 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:13.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:13 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:13.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:13 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:13.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:13 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:29:13.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:13 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:13.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:13 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:13.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:13 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:13.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:13 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:13.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:13 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:29:13.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:13 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:13.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:13 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:13.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:13 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:13.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:13 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: pgmap v194: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-02-19T16:29:14.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:14.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:14.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:14.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:14 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: pgmap v194: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-02-19T16:29:14.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:14.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:14.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:14.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:14 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: pgmap v194: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-02-19T16:29:14.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:14.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:14.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:14.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:14.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:14 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:16.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:16 smithi008 ceph-mon[32573]: pgmap v195: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:16.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:16 smithi008 ceph-mon[32573]: osdmap e98: 12 total, 12 up, 12 in 2024-02-19T16:29:16.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:16 smithi191 ceph-mon[37203]: pgmap v195: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:16.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:16 smithi191 ceph-mon[37203]: osdmap e98: 12 total, 12 up, 12 in 2024-02-19T16:29:16.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:16 smithi156 ceph-mon[36502]: pgmap v195: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:16.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:16 smithi156 ceph-mon[36502]: osdmap e98: 12 total, 12 up, 12 in 2024-02-19T16:29:18.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:18 smithi008 ceph-mon[32573]: pgmap v197: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-02-19T16:29:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:18 smithi191 ceph-mon[37203]: pgmap v197: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-02-19T16:29:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:18 smithi156 ceph-mon[36502]: pgmap v197: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-02-19T16:29:19.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:19 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:29:19.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:19 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:29:19.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:19 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:29:19.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:19 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:29:19.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:19 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:29:19.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:19 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:29:20.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:20 smithi008 ceph-mon[32573]: pgmap v198: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 223 B/s wr, 7 op/s 2024-02-19T16:29:20.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:20 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:29:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:20 smithi191 ceph-mon[37203]: pgmap v198: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 223 B/s wr, 7 op/s 2024-02-19T16:29:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:20 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:29:20.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:20 smithi156 ceph-mon[36502]: pgmap v198: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 223 B/s wr, 7 op/s 2024-02-19T16:29:20.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:20 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:29:22.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:22 smithi008 ceph-mon[32573]: pgmap v199: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:22 smithi191 ceph-mon[37203]: pgmap v199: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:22 smithi156 ceph-mon[36502]: pgmap v199: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:24.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:24 smithi008 ceph-mon[32573]: pgmap v200: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:24 smithi191 ceph-mon[37203]: pgmap v200: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:24.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:24 smithi156 ceph-mon[36502]: pgmap v200: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:29:26.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:26 smithi008 ceph-mon[32573]: pgmap v201: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:29:26.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:26 smithi191 ceph-mon[37203]: pgmap v201: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:29:26.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:26 smithi156 ceph-mon[36502]: pgmap v201: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:29:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:28 smithi008 ceph-mon[32573]: pgmap v202: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:29:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:28 smithi191 ceph-mon[37203]: pgmap v202: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:29:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:28 smithi156 ceph-mon[36502]: pgmap v202: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:29:30.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:30 smithi008 ceph-mon[32573]: pgmap v203: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:29:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:30 smithi191 ceph-mon[37203]: pgmap v203: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:29:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:30 smithi156 ceph-mon[36502]: pgmap v203: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:29:32.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:32 smithi191 ceph-mon[37203]: pgmap v204: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:32.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:32 smithi156 ceph-mon[36502]: pgmap v204: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:32 smithi008 ceph-mon[32573]: pgmap v204: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:34 smithi191 ceph-mon[37203]: pgmap v205: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:34 smithi156 ceph-mon[36502]: pgmap v205: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:35.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:34 smithi008 ceph-mon[32573]: pgmap v205: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:36.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:36 smithi191 ceph-mon[37203]: pgmap v206: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:36 smithi156 ceph-mon[36502]: pgmap v206: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:37.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:36 smithi008 ceph-mon[32573]: pgmap v206: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:38.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:38 smithi156 ceph-mon[36502]: pgmap v207: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:39.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:38 smithi008 ceph-mon[32573]: pgmap v207: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:39.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:38 smithi191 ceph-mon[37203]: pgmap v207: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:40 smithi156 ceph-mon[36502]: pgmap v208: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:41.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:40 smithi008 ceph-mon[32573]: pgmap v208: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:41.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:40 smithi191 ceph-mon[37203]: pgmap v208: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:42 smithi156 ceph-mon[36502]: pgmap v209: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:42.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:29:42.902 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:29:42.930 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:42 smithi008 ceph-mon[32573]: pgmap v209: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:43.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 -- 172.21.15.8:0/1702630628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300101e20 msgr2=0x7f630010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 --2- 172.21.15.8:0/1702630628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300101e20 0x7f630010e330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009cd0 tx=0x7f62f802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 -- 172.21.15.8:0/1702630628 shutdown_connections 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 --2- 172.21.15.8:0/1702630628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300101e20 0x7f630010e330 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 --2- 172.21.15.8:0/1702630628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101460 0x7f63001018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 --2- 172.21.15.8:0/1702630628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6300107500 0x7f63001078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 -- 172.21.15.8:0/1702630628 >> 172.21.15.8:0/1702630628 conn(0x7f63000fcf90 msgr2=0x7f63000ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 -- 172.21.15.8:0/1702630628 shutdown_connections 2024-02-19T16:29:43.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.017+0000 7f63067f7640 1 -- 172.21.15.8:0/1702630628 wait complete. 2024-02-19T16:29:43.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 Processor -- start 2024-02-19T16:29:43.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 -- start start 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6300101460 0x7f63000737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 0x7f6300073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 0x7f6300074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a5ea0 con 0x7f6300107500 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a6010 con 0x7f6300101460 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.018+0000 7f63067f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a61d0 con 0x7f6300101e20 2024-02-19T16:29:43.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f6304d94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 0x7f6300074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 0x7f6300073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 0x7f6300073d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:46100/0 (socket says 172.21.15.8:46100) 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f6304d94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 0x7f6300074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39488/0 (socket says 172.21.15.8:39488) 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62fffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6300101460 0x7f63000737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1059711113 learned_addr learned my addr 172.21.15.8:0/1059711113 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1059711113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6300101460 msgr2=0x7f63000737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6300101460 0x7f63000737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 msgr2=0x7f6300074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 0x7f6300074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1059711113 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8009900 con 0x7f6300101e20 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f6304d94640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 0x7f6300074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 0x7f6300073d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62f000e9f0 tx=0x7f62f000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62fd7fa640 1 -- 172.21.15.8:0/1059711113 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0009720 con 0x7f6300101e20 2024-02-19T16:29:43.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63001a64b0 con 0x7f6300101e20 2024-02-19T16:29:43.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62fd7fa640 1 -- 172.21.15.8:0/1059711113 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62f000cd80 con 0x7f6300101e20 2024-02-19T16:29:43.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f62fd7fa640 1 -- 172.21.15.8:0/1059711113 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0010690 con 0x7f6300101e20 2024-02-19T16:29:43.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.019+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63001a6970 con 0x7f6300101e20 2024-02-19T16:29:43.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.021+0000 7f62fd7fa640 1 -- 172.21.15.8:0/1059711113 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f62f00046c0 con 0x7f6300101e20 2024-02-19T16:29:43.032 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:42 smithi191 ceph-mon[37203]: pgmap v209: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:43.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 Processor -- start 2024-02-19T16:29:43.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 -- start start 2024-02-19T16:29:43.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f62fd7fa640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f62e80a2230 0x7f62e80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2dee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2deebd0 0x55c4a2df7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f6304d94640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f62e80a2230 0x7f62e80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2d5d280 con 0x55c4a2dee2b0 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2d5a7b0 con 0x55c4a2deebd0 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f63093d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2d55f90 con 0x55c4a2df77b0 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f62fd7fa640 1 -- 172.21.15.8:0/1059711113 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f62f0014070 con 0x7f6300101e20 2024-02-19T16:29:43.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f62fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2dee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f62fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2dee690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39492/0 (socket says 172.21.15.8:39492) 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f62fffff640 1 -- 172.21.15.8:0/4049234350 learned_addr learned my addr 172.21.15.8:0/4049234350 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2deebd0 0x55c4a2df7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.022+0000 7f6304d94640 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62fffff640 1 -- 172.21.15.8:0/4049234350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 msgr2=0x55c4a2df9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62fffff640 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62fffff640 1 -- 172.21.15.8:0/4049234350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2deebd0 msgr2=0x55c4a2df7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62fffff640 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2deebd0 0x55c4a2df7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62fffff640 1 -- 172.21.15.8:0/4049234350 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4a2d14570 con 0x55c4a2dee2b0 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2deebd0 0x55c4a2df7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f6304d94640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f62e80a2230 0x7f62e80a46f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f62f802c630 tx=0x7f62f8032470 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62fffff640 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2dee690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009bf0 tx=0x7f62f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f6186a93a2da98 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62e67fc640 1 -- 172.21.15.8:0/4049234350 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4004230 con 0x55c4a2dee2b0 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62e67fc640 1 -- 172.21.15.8:0/4049234350 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62f4009e60 con 0x55c4a2dee2b0 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.023+0000 7f62e67fc640 1 -- 172.21.15.8:0/4049234350 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4019720 con 0x55c4a2dee2b0 2024-02-19T16:29:43.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 -- 172.21.15.8:0/4049234350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 msgr2=0x55c4a2dee690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2dee690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009bf0 tx=0x7f62f4011810 comp rx=0 tx=0).stop 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 -- 172.21.15.8:0/4049234350 shutdown_connections 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2deebd0 0x55c4a2df7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 --2- 172.21.15.8:0/4049234350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2dee690 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 -- 172.21.15.8:0/4049234350 >> 172.21.15.8:0/4049234350 conn(0x55c4a2de8990 msgr2=0x55c4a2de8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 -- 172.21.15.8:0/4049234350 shutdown_connections 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 -- 172.21.15.8:0/4049234350 wait complete. 2024-02-19T16:29:43.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 -- start start 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.024+0000 7f63093d5740 1 Processor -- start 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2a29430 con 0x55c4a2deebd0 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a29e5e00 con 0x55c4a2dee2b0 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2cdafd0 con 0x55c4a2df77b0 2024-02-19T16:29:43.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 10 client.0 _ll_register_callbacks cb 0x55c4a2d30fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39496/0 (socket says 172.21.15.8:39496) 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1353136674 learned_addr learned my addr 172.21.15.8:0/1353136674 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62fffff640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f6304d94640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 msgr2=0x55c4a2df4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1353136674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2dee2b0 msgr2=0x55c4a2df70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1353136674 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f40098d0 con 0x55c4a2deebd0 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df4450 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f62f009adb0 tx=0x7f62f009ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205e6c0d2c968742 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62df7fe640 1 -- 172.21.15.8:0/1353136674 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f00cc000 con 0x55c4a2deebd0 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4a2a557e0 con 0x55c4a2deebd0 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.025+0000 7f62df7fe640 1 -- 172.21.15.8:0/1353136674 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f62f004c1b0 con 0x55c4a2deebd0 2024-02-19T16:29:43.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f62df7fe640 1 -- 172.21.15.8:0/1353136674 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62f0051240 con 0x55c4a2deebd0 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 10 client.17100 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c4a2d54610 con 0x55c4a2deebd0 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f62df7fe640 1 -- 172.21.15.8:0/1353136674 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c4a2a557e0 con 0x55c4a2deebd0 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f62df7fe640 1 -- 172.21.15.8:0/1353136674 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f62f001f060 con 0x55c4a2deebd0 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 10 client.17100 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 20 client.17100 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e077a0 0x55c4a2e09b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 4 client.17100 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4a2a92720 con 0x55c4a2e077a0 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f62fffff640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e077a0 0x55c4a2e09b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f62df7fe640 10 client.17100 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.026+0000 7f62fffff640 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e077a0 0x55c4a2e09b90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f39802153328ca4 server_cookie=6a0d3caeede2cc9d in_seq=0 out_seq=0 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f62df7fe640 1 -- 172.21.15.8:0/1353136674 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c4a2a92720 con 0x55c4a2e077a0 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f62df7fe640 10 client.17100 handle_command_reply: tid=0 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 client.17100 shutdown 2024-02-19T16:29:43.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 msgr2=0x55c4a2df4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df4450 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f62f009adb0 tx=0x7f62f009ae20 comp rx=0 tx=0).stop 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 shutdown_connections 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e077a0 0x55c4a2e09b90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2df77b0 0x55c4a2df4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df4450 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f62df7fe640 0 client.17100 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 --2- 172.21.15.8:0/1353136674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 >> 172.21.15.8:0/1353136674 conn(0x55c4a2de8990 msgr2=0x55c4a2dea730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 shutdown_connections 2024-02-19T16:29:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.027+0000 7f63093d5740 1 -- 172.21.15.8:0/1353136674 wait complete. 2024-02-19T16:29:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.028+0000 7f63093d5740 20 client.17100 trim_cache size 0 max 16384 2024-02-19T16:29:43.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 Processor -- start 2024-02-19T16:29:43.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 -- start start 2024-02-19T16:29:43.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2e02b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2dedcd0 con 0x55c4a2deebd0 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2e04370 con 0x55c4a2df77b0 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f63093d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2de9a60 con 0x55c4a2dee2b0 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39506/0 (socket says 172.21.15.8:39506) 2024-02-19T16:29:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1666619801 learned_addr learned my addr 172.21.15.8:0/1666619801 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62fffff640 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f6304d94640 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2e02b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1666619801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2dee2b0 msgr2=0x55c4a2df6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1666619801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 msgr2=0x55c4a2e02b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2e02b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 -- 172.21.15.8:0/1666619801 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4a2cf0ac0 con 0x55c4a2deebd0 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f6304d94640 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2e02b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df70c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f62f00977f0 tx=0x7f62f0004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c932ba4d1e02a020 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.044+0000 7f62e5ffb640 1 -- 172.21.15.8:0/1666619801 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f00ce5b0 con 0x55c4a2deebd0 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f62e5ffb640 1 -- 172.21.15.8:0/1666619801 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62f00cbe00 con 0x55c4a2deebd0 2024-02-19T16:29:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f62e5ffb640 1 -- 172.21.15.8:0/1666619801 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f009a270 con 0x55c4a2deebd0 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 -- 172.21.15.8:0/1666619801 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 msgr2=0x55c4a2df70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df70c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f62f00977f0 tx=0x7f62f0004b30 comp rx=0 tx=0).stop 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 -- 172.21.15.8:0/1666619801 shutdown_connections 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2e02b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2deebd0 0x55c4a2df70c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 --2- 172.21.15.8:0/1666619801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 -- 172.21.15.8:0/1666619801 >> 172.21.15.8:0/1666619801 conn(0x55c4a2de8990 msgr2=0x55c4a2de7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 -- 172.21.15.8:0/1666619801 shutdown_connections 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 -- 172.21.15.8:0/1666619801 wait complete. 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.045+0000 7f63093d5740 1 -- start start 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 Processor -- start 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2deebd0 0x55c4a2df1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2df8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2df89b0 con 0x55c4a2dee2b0 2024-02-19T16:29:43.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2df8b20 con 0x55c4a2df77b0 2024-02-19T16:29:43.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a2df8ce0 con 0x55c4a2deebd0 2024-02-19T16:29:43.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f63093d5740 10 client.0 _ll_register_callbacks cb 0x7f62f009ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39522/0 (socket says 172.21.15.8:39522) 2024-02-19T16:29:43.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 -- 172.21.15.8:0/4103367218 learned_addr learned my addr 172.21.15.8:0/4103367218 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2deebd0 0x55c4a2df1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f6304d94640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2df8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 -- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2deebd0 msgr2=0x55c4a2df1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2deebd0 0x55c4a2df1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 -- 172.21.15.8:0/4103367218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 msgr2=0x55c4a2df8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2df8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 -- 172.21.15.8:0/4103367218 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4a2d54610 con 0x55c4a2dee2b0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62fffff640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df1aa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f62f4004320 tx=0x7f62f4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5559a5678c5c8d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.046+0000 7f62df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f62df7fe640 1 -- 172.21.15.8:0/4103367218 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f402a070 con 0x55c4a2dee2b0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f62df7fe640 1 -- 172.21.15.8:0/4103367218 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f62f4005040 con 0x55c4a2dee2b0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62f4009e60 con 0x55c4a2dee2b0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f62df7fe640 1 -- 172.21.15.8:0/4103367218 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62f4004100 con 0x55c4a2dee2b0 2024-02-19T16:29:43.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 10 client.17106 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c4a2d166a0 con 0x55c4a2dee2b0 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f62df7fe640 1 -- 172.21.15.8:0/4103367218 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62f4009e60 con 0x55c4a2dee2b0 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f62df7fe640 1 -- 172.21.15.8:0/4103367218 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f62f4004450 con 0x55c4a2dee2b0 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 10 client.17106 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 20 client.17106 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e63fd0 0x55c4a2e663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 4 client.17106 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4a2ae1890 con 0x55c4a2e63fd0 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.047+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e63fd0 0x55c4a2e663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f62df7fe640 10 client.17106 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f62ff7fe640 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e63fd0 0x55c4a2e663f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24454316ed5a9081 server_cookie=4d956dfae47a91bc in_seq=0 out_seq=0 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f62df7fe640 1 -- 172.21.15.8:0/4103367218 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c4a2ae1890 con 0x55c4a2e63fd0 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f62df7fe640 10 client.17106 handle_command_reply: tid=0 2024-02-19T16:29:43.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 client.17106 shutdown 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 msgr2=0x55c4a2df1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df1aa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f62f4004320 tx=0x7f62f4003f40 comp rx=0 tx=0).stop 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 shutdown_connections 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4a2df77b0 0x55c4a2df8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c4a2e63fd0 0x55c4a2e663f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4a2deebd0 0x55c4a2df1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 --2- 172.21.15.8:0/4103367218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4a2dee2b0 0x55c4a2df1aa0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f62df7fe640 0 client.17106 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 >> 172.21.15.8:0/4103367218 conn(0x55c4a2de8990 msgr2=0x55c4a2df3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 shutdown_connections 2024-02-19T16:29:43.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 1 -- 172.21.15.8:0/4103367218 wait complete. 2024-02-19T16:29:43.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.048+0000 7f63093d5740 20 client.17106 trim_cache size 0 max 16384 2024-02-19T16:29:43.067 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:29:43.067 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:29:43.067 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:29:43.067 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:29:43.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f62e80a2230 msgr2=0x7f62e80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f62e80a2230 0x7f62e80a46f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f62f802c630 tx=0x7f62f8032470 comp rx=0 tx=0).stop 2024-02-19T16:29:43.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 msgr2=0x7f6300073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 0x7f6300073d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62f000e9f0 tx=0x7f62f000edf0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 shutdown_connections 2024-02-19T16:29:43.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6300107500 0x7f6300074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f62e80a2230 0x7f62e80a46f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6300101e20 0x7f6300073d30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 --2- 172.21.15.8:0/1059711113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6300101460 0x7f63000737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 >> 172.21.15.8:0/1059711113 conn(0x7f63000fcf90 msgr2=0x7f63000fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 shutdown_connections 2024-02-19T16:29:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.049+0000 7f63067f7640 1 -- 172.21.15.8:0/1059711113 wait complete. 2024-02-19T16:29:43.071 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:29:43.071 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:29:43.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.210+0000 7f7d7fa23640 1 -- 172.21.15.8:0/3809666635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 msgr2=0x7f7d78106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.210+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/3809666635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d78106870 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009930 tx=0x7f7d6c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- 172.21.15.8:0/3809666635 shutdown_connections 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/3809666635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d78100db0 0x7f7d7810d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/3809666635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d781003f0 0x7f7d78100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/3809666635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d78106870 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- 172.21.15.8:0/3809666635 >> 172.21.15.8:0/3809666635 conn(0x7f7d780774f0 msgr2=0x7f7d78077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- 172.21.15.8:0/3809666635 shutdown_connections 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- 172.21.15.8:0/3809666635 wait complete. 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 Processor -- start 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- start start 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d781003f0 0x7f7d781989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d78100db0 0x7f7d78198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d781a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78199740 con 0x7f7d78106490 2024-02-19T16:29:43.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781998b0 con 0x7f7d781003f0 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.211+0000 7f7d7fa23640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781a4030 con 0x7f7d78100db0 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d781a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d781a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39532/0 (socket says 172.21.15.8:39532) 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 -- 172.21.15.8:0/273974213 learned_addr learned my addr 172.21.15.8:0/273974213 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7cfbe640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d78100db0 0x7f7d78198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7d7bf640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d781003f0 0x7f7d781989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 -- 172.21.15.8:0/273974213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d78100db0 msgr2=0x7f7d78198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d78100db0 0x7f7d78198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 -- 172.21.15.8:0/273974213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d781003f0 msgr2=0x7f7d781989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d781003f0 0x7f7d781989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 -- 172.21.15.8:0/273974213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c009560 con 0x7f7d78106490 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7dfc0640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d781a3a70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000eb10 tx=0x7f7d7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70004b20 con 0x7f7d78106490 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d70004c80 con 0x7f7d78106490 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d781a4310 con 0x7f7d78106490 2024-02-19T16:29:43.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70010650 con 0x7f7d78106490 2024-02-19T16:29:43.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.212+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d781a4820 con 0x7f7d78106490 2024-02-19T16:29:43.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.213+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d780682e0 con 0x7f7d78106490 2024-02-19T16:29:43.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.214+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f7d70005020 con 0x7f7d78106490 2024-02-19T16:29:43.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.215+0000 7f7d6a7fc640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7d640a2230 0x7f7d640a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.215+0000 7f7d7d7bf640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7d640a2230 0x7f7d640a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.215+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7d70014070 con 0x7f7d78106490 2024-02-19T16:29:43.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.215+0000 7f7d7d7bf640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7d640a2230 0x7f7d640a46f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005ab0 tx=0x7f7d6c005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.219+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f7d70097230 con 0x7f7d78106490 2024-02-19T16:29:43.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.437+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d78108960 con 0x7f7d78106490 2024-02-19T16:29:43.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.438+0000 7f7d6a7fc640 1 -- 172.21.15.8:0/273974213 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f7d7009a8f0 con 0x7f7d78106490 2024-02-19T16:29:43.439 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:29:43.440 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:29:43.440 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:29:43.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7d640a2230 msgr2=0x7f7d640a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7d640a2230 0x7f7d640a46f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005ab0 tx=0x7f7d6c005fd0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 msgr2=0x7f7d781a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d781a3a70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000eb10 tx=0x7f7d7000c2d0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 shutdown_connections 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d78106490 0x7f7d781a3a70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f7d640a2230 0x7f7d640a46f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d78100db0 0x7f7d78198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 --2- 172.21.15.8:0/273974213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d781003f0 0x7f7d781989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 >> 172.21.15.8:0/273974213 conn(0x7f7d780774f0 msgr2=0x7f7d78102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 shutdown_connections 2024-02-19T16:29:43.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.440+0000 7f7d7fa23640 1 -- 172.21.15.8:0/273974213 wait complete. 2024-02-19T16:29:43.452 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:29:43.452 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:29:43.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 -- 172.21.15.8:0/1765131085 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c107560 msgr2=0x7fb48c107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 --2- 172.21.15.8:0/1765131085 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c107560 0x7fb48c107940 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009900 tx=0x7fb47c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:43.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 -- 172.21.15.8:0/1765131085 shutdown_connections 2024-02-19T16:29:43.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 --2- 172.21.15.8:0/1765131085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c101e60 0x7fb48c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 --2- 172.21.15.8:0/1765131085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c1014a0 0x7fb48c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 --2- 172.21.15.8:0/1765131085 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c107560 0x7fb48c107940 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.556+0000 7fb491440640 1 -- 172.21.15.8:0/1765131085 >> 172.21.15.8:0/1765131085 conn(0x7fb48c0788f0 msgr2=0x7fb48c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 -- 172.21.15.8:0/1765131085 shutdown_connections 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 -- 172.21.15.8:0/1765131085 wait complete. 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 Processor -- start 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 -- start start 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 0x7fb48c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c101e60 0x7fb48c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c107560 0x7fb48c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c197ed0 con 0x7fb48c1014a0 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c198040 con 0x7fb48c107560 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb491440640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c198200 con 0x7fb48c101e60 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb48affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 0x7fb48c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb48affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 0x7fb48c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39542/0 (socket says 172.21.15.8:39542) 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb48affd640 1 -- 172.21.15.8:0/3573974682 learned_addr learned my addr 172.21.15.8:0/3573974682 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c107560 0x7fb48c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.557+0000 7fb483fff640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c101e60 0x7fb48c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb48affd640 1 -- 172.21.15.8:0/3573974682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c101e60 msgr2=0x7fb48c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb48affd640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c101e60 0x7fb48c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb48affd640 1 -- 172.21.15.8:0/3573974682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c107560 msgr2=0x7fb48c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb48affd640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c107560 0x7fb48c197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb48affd640 1 -- 172.21.15.8:0/3573974682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c009560 con 0x7fb48c1014a0 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb48affd640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 0x7fb48c19d6d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005940 tx=0x7fb47c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb488ff9640 1 -- 172.21.15.8:0/3573974682 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0042d0 con 0x7fb48c1014a0 2024-02-19T16:29:43.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb488ff9640 1 -- 172.21.15.8:0/3573974682 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb47c004430 con 0x7fb48c1014a0 2024-02-19T16:29:43.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c198480 con 0x7fb48c1014a0 2024-02-19T16:29:43.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb488ff9640 1 -- 172.21.15.8:0/3573974682 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c003c50 con 0x7fb48c1014a0 2024-02-19T16:29:43.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.558+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c1a4770 con 0x7fb48c1014a0 2024-02-19T16:29:43.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.560+0000 7fb488ff9640 1 -- 172.21.15.8:0/3573974682 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb47c0054f0 con 0x7fb48c1014a0 2024-02-19T16:29:43.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 Processor -- start 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 -- start start 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x559255536630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x559255541b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592554f1960 con 0x55925553f750 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb488ff9640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4780a2230 0x7fb4780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592554a5310 con 0x559255536250 2024-02-19T16:29:43.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb494018740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592554a2840 con 0x559255536b70 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb483fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x559255541b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4780a2230 0x7fb4780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb48affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb483fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x559255541b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39554/0 (socket says 172.21.15.8:39554) 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb483fff640 1 -- 172.21.15.8:0/618410745 learned_addr learned my addr 172.21.15.8:0/618410745 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x559255536630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.561+0000 7fb488ff9640 1 -- 172.21.15.8:0/3573974682 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb47c0f0440 con 0x7fb48c1014a0 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb483fff640 1 -- 172.21.15.8:0/618410745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 msgr2=0x55925553f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb483fff640 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb483fff640 1 -- 172.21.15.8:0/618410745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 msgr2=0x559255536630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb483fff640 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x559255536630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb483fff640 1 -- 172.21.15.8:0/618410745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559255460060 con 0x55925553f750 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb48affd640 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4780a2230 0x7fb4780a46f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb484005fd0 tx=0x7fb484009f00 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb483fff640 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x559255541b40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb474009c60 tx=0x7fb474011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a60d73e6e23f2bf server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb4817fa640 1 -- 172.21.15.8:0/618410745 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474004250 con 0x55925553f750 2024-02-19T16:29:43.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb4817fa640 1 -- 172.21.15.8:0/618410745 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb474009e10 con 0x55925553f750 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.562+0000 7fb4817fa640 1 -- 172.21.15.8:0/618410745 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474004ca0 con 0x55925553f750 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- 172.21.15.8:0/618410745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 msgr2=0x559255541b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x559255541b40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb474009c60 tx=0x7fb474011810 comp rx=0 tx=0).stop 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- 172.21.15.8:0/618410745 shutdown_connections 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x559255541b40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- 172.21.15.8:0/618410745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x559255536630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- 172.21.15.8:0/618410745 >> 172.21.15.8:0/618410745 conn(0x559255530980 msgr2=0x559255530d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- 172.21.15.8:0/618410745 shutdown_connections 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- 172.21.15.8:0/618410745 wait complete. 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- start start 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 Processor -- start 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x55925553c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925553d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925549e020 con 0x55925553f750 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559255171430 con 0x559255536250 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925512de00 con 0x559255536b70 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb494018740 10 client.0 _ll_register_callbacks cb 0x559255445cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.563+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925553d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb483fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x55925553c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925553d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39566/0 (socket says 172.21.15.8:39566) 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1387534344 learned_addr learned my addr 172.21.15.8:0/1387534344 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 msgr2=0x55925553cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1387534344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 msgr2=0x55925553c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x55925553c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1387534344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb474009890 con 0x55925553f750 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48affd640 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x55925553c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925553d110 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb484003b60 tx=0x7fb48401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f4cfa2130641d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb4727fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb4727fc640 1 -- 172.21.15.8:0/1387534344 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484016040 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb4727fc640 1 -- 172.21.15.8:0/1387534344 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb48401f450 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55925519d7e0 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb4727fc640 1 -- 172.21.15.8:0/1387534344 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48401f610 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb494018740 10 client.17124 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55925549c6a0 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.564+0000 7fb4727fc640 1 -- 172.21.15.8:0/1387534344 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55925519d7e0 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb4727fc640 1 -- 172.21.15.8:0/1387534344 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb484026030 con 0x55925553f750 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb494018740 10 client.17124 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb494018740 10 client.17124 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb494018740 20 client.17124 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb494018740 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55925554fb00 0x559255551f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb494018740 4 client.17124 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592551da720 con 0x55925554fb00 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb48affd640 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55925554fb00 0x559255551f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb4727fc640 10 client.17124 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.565+0000 7fb48affd640 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55925554fb00 0x559255551f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e95fd3feff32863b server_cookie=1c1cdd07e5735acd in_seq=0 out_seq=0 2024-02-19T16:29:43.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb4727fc640 1 -- 172.21.15.8:0/1387534344 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5592551da720 con 0x55925554fb00 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb4727fc640 10 client.17124 handle_command_reply: tid=0 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 client.17124 shutdown 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 msgr2=0x55925553d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925553d110 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb484003b60 tx=0x7fb48401d000 comp rx=0 tx=0).stop 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 shutdown_connections 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55925554fb00 0x559255551f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925553d110 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536b70 0x55925553cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb4727fc640 0 client.17124 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:43.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 --2- 172.21.15.8:0/1387534344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536250 0x55925553c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 >> 172.21.15.8:0/1387534344 conn(0x559255530980 msgr2=0x5592555326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 shutdown_connections 2024-02-19T16:29:43.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 1 -- 172.21.15.8:0/1387534344 wait complete. 2024-02-19T16:29:43.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.566+0000 7fb494018740 20 client.17124 trim_cache size 0 max 16384 2024-02-19T16:29:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 Processor -- start 2024-02-19T16:29:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 -- start start 2024-02-19T16:29:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x55925553edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536b70 0x55925553cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925554aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559255423130 con 0x55925553f750 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559255535c20 con 0x559255536b70 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb494018740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559255535ee0 con 0x559255536250 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925554aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x55925553edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925554aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39580/0 (socket says 172.21.15.8:39580) 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb483fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536b70 0x55925553cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x55925553edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:46198/0 (socket says 172.21.15.8:46198) 2024-02-19T16:29:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1022888477 learned_addr learned my addr 172.21.15.8:0/1022888477 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1022888477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 msgr2=0x55925553edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.579+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x55925553edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1022888477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536b70 msgr2=0x55925553cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536b70 0x55925553cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb48b7fe640 1 -- 172.21.15.8:0/1022888477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559255438c20 con 0x55925553f750 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb483fff640 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536b70 0x55925553cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925554aad0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb484004a00 tx=0x7fb484004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bafd9357cfcd05 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb480ff9640 1 -- 172.21.15.8:0/1022888477 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484016070 con 0x55925553f750 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb480ff9640 1 -- 172.21.15.8:0/1022888477 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4840044c0 con 0x55925553f750 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb480ff9640 1 -- 172.21.15.8:0/1022888477 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484037410 con 0x55925553f750 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.580+0000 7fb494018740 1 -- 172.21.15.8:0/1022888477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 msgr2=0x55925554aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925554aad0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb484004a00 tx=0x7fb484004a30 comp rx=0 tx=0).stop 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- 172.21.15.8:0/1022888477 shutdown_connections 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55925553f750 0x55925554aad0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559255536b70 0x55925553cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- 172.21.15.8:0/1022888477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x55925553edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- 172.21.15.8:0/1022888477 >> 172.21.15.8:0/1022888477 conn(0x559255530980 msgr2=0x55925552fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- 172.21.15.8:0/1022888477 shutdown_connections 2024-02-19T16:29:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- 172.21.15.8:0/1022888477 wait complete. 2024-02-19T16:29:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- start start 2024-02-19T16:29:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 Processor -- start 2024-02-19T16:29:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x559255538f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 0x559255539480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925553f750 0x5592555399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592555ae8d0 con 0x559255536b70 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592555aea40 con 0x55925553f750 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592555aec00 con 0x559255536250 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb494018740 10 client.0 _ll_register_callbacks cb 0x7fb484009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb483fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 0x559255539480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb483fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 0x559255539480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39590/0 (socket says 172.21.15.8:39590) 2024-02-19T16:29:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb483fff640 1 -- 172.21.15.8:0/4078269026 learned_addr learned my addr 172.21.15.8:0/4078269026 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb48b7fe640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925553f750 0x5592555399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.581+0000 7fb48affd640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x559255538f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb483fff640 1 -- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 msgr2=0x559255538f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb483fff640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x559255538f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb483fff640 1 -- 172.21.15.8:0/4078269026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925553f750 msgr2=0x5592555399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb483fff640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925553f750 0x5592555399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb483fff640 1 -- 172.21.15.8:0/4078269026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559255460060 con 0x559255536b70 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb483fff640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 0x559255539480 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb474009b80 tx=0x7fb474009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aef260809cdc0b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb4727fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb4727fc640 1 -- 172.21.15.8:0/4078269026 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474003800 con 0x559255536b70 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb4727fc640 1 -- 172.21.15.8:0/4078269026 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb474003960 con 0x559255536b70 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb474009e10 con 0x559255536b70 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb4727fc640 1 -- 172.21.15.8:0/4078269026 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb474003b20 con 0x559255536b70 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb494018740 10 client.17130 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55925549c6a0 con 0x559255536b70 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb4727fc640 1 -- 172.21.15.8:0/4078269026 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb474009e10 con 0x559255536b70 2024-02-19T16:29:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.582+0000 7fb4727fc640 1 -- 172.21.15.8:0/4078269026 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb474021040 con 0x559255536b70 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 10 client.17130 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 10 client.17130 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 20 client.17130 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5592555b0430 0x5592555b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 4 client.17130 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559255229890 con 0x5592555b0430 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb48affd640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5592555b0430 0x5592555b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb48affd640 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5592555b0430 0x5592555b2850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1dfd07b9261b9c6 server_cookie=7f4a8f31f2d5c544 in_seq=0 out_seq=0 2024-02-19T16:29:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb4727fc640 10 client.17130 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb4727fc640 1 -- 172.21.15.8:0/4078269026 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559255229890 con 0x5592555b0430 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb4727fc640 10 client.17130 handle_command_reply: tid=0 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 client.17130 shutdown 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 msgr2=0x559255539480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 0x559255539480 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb474009b80 tx=0x7fb474009d50 comp rx=0 tx=0).stop 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 shutdown_connections 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x5592555b0430 0x5592555b2850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55925553f750 0x5592555399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559255536b70 0x559255539480 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 --2- 172.21.15.8:0/4078269026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559255536250 0x559255538f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb4727fc640 0 client.17130 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 >> 172.21.15.8:0/4078269026 conn(0x559255530980 msgr2=0x559255531b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 shutdown_connections 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.583+0000 7fb494018740 1 -- 172.21.15.8:0/4078269026 wait complete. 2024-02-19T16:29:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb494018740 20 client.17130 trim_cache size 0 max 16384 2024-02-19T16:29:43.598 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:29:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4780a2230 msgr2=0x7fb4780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4780a2230 0x7fb4780a46f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb484005fd0 tx=0x7fb484009f00 comp rx=0 tx=0).stop 2024-02-19T16:29:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 msgr2=0x7fb48c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 0x7fb48c19d6d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005940 tx=0x7fb47c02c670 comp rx=0 tx=0).stop 2024-02-19T16:29:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 shutdown_connections 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c107560 0x7fb48c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb4780a2230 0x7fb4780a46f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c101e60 0x7fb48c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 --2- 172.21.15.8:0/3573974682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb48c1014a0 0x7fb48c19d6d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 >> 172.21.15.8:0/3573974682 conn(0x7fb48c0788f0 msgr2=0x7fb48c100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 shutdown_connections 2024-02-19T16:29:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.584+0000 7fb491440640 1 -- 172.21.15.8:0/3573974682 wait complete. 2024-02-19T16:29:43.601 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:29:43.601 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:29:43.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 -- 172.21.15.8:0/3102579901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea4100410 msgr2=0x7faea4100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 --2- 172.21.15.8:0/3102579901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea4100410 0x7faea4100890 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fae94009930 tx=0x7fae9402bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:43.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 -- 172.21.15.8:0/3102579901 shutdown_connections 2024-02-19T16:29:43.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 --2- 172.21.15.8:0/3102579901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea4100dd0 0x7faea410d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 --2- 172.21.15.8:0/3102579901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea4100410 0x7faea4100890 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 --2- 172.21.15.8:0/3102579901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea41064b0 0x7faea4106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 -- 172.21.15.8:0/3102579901 >> 172.21.15.8:0/3102579901 conn(0x7faea4077550 msgr2=0x7faea4077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 -- 172.21.15.8:0/3102579901 shutdown_connections 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 -- 172.21.15.8:0/3102579901 wait complete. 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.704+0000 7faeabb48640 1 Processor -- start 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 -- start start 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea4100410 0x7faea41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea4100dd0 0x7faea4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 0x7faea41a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4199790 con 0x7faea41064b0 2024-02-19T16:29:43.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea41a3f90 con 0x7faea4100410 2024-02-19T16:29:43.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeabb48640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea41a40d0 con 0x7faea4100dd0 2024-02-19T16:29:43.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 0x7faea41a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 0x7faea41a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39604/0 (socket says 172.21.15.8:39604) 2024-02-19T16:29:43.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 -- 172.21.15.8:0/852249213 learned_addr learned my addr 172.21.15.8:0/852249213 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faea90e3640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea4100dd0 0x7faea4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faea98e4640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea4100410 0x7faea41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 -- 172.21.15.8:0/852249213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea4100dd0 msgr2=0x7faea4198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea4100dd0 0x7faea4198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 -- 172.21.15.8:0/852249213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea4100410 msgr2=0x7faea41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea4100410 0x7faea41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 -- 172.21.15.8:0/852249213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94009560 con 0x7faea41064b0 2024-02-19T16:29:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7faeaa0e5640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 0x7faea41a3a50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00eb40 tx=0x7fae9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.705+0000 7fae9affd640 1 -- 172.21.15.8:0/852249213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00ceb0 con 0x7faea41064b0 2024-02-19T16:29:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.706+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea41a4330 con 0x7faea41064b0 2024-02-19T16:29:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.706+0000 7fae9affd640 1 -- 172.21.15.8:0/852249213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9c005020 con 0x7faea41064b0 2024-02-19T16:29:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.706+0000 7fae9affd640 1 -- 172.21.15.8:0/852249213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c010690 con 0x7faea41064b0 2024-02-19T16:29:43.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.706+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea41a4810 con 0x7faea41064b0 2024-02-19T16:29:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.708+0000 7fae9affd640 1 -- 172.21.15.8:0/852249213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fae9c005190 con 0x7faea41064b0 2024-02-19T16:29:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 Processor -- start 2024-02-19T16:29:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 -- start start 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca26630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca2f750 0x55ed2ca31b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c9e1960 con 0x55ed2ca26250 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c995310 con 0x55ed2ca2f750 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeae802740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c992840 con 0x55ed2ca26b70 2024-02-19T16:29:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faea90e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca26630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faea90e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca26630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39616/0 (socket says 172.21.15.8:39616) 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faea90e3640 1 -- 172.21.15.8:0/376361999 learned_addr learned my addr 172.21.15.8:0/376361999 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faea98e4640 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7fae9affd640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fae900a2230 0x7fae900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faea90e3640 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca2f750 0x55ed2ca31b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7faeaa0e5640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fae900a2230 0x7fae900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea90e3640 1 -- 172.21.15.8:0/376361999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 msgr2=0x55ed2ca2f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.709+0000 7fae9affd640 1 -- 172.21.15.8:0/852249213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fae9c014070 con 0x7faea41064b0 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea90e3640 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea90e3640 1 -- 172.21.15.8:0/376361999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca2f750 msgr2=0x55ed2ca31b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea90e3640 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca2f750 0x55ed2ca31b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea90e3640 1 -- 172.21.15.8:0/376361999 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2c950060 con 0x55ed2ca26250 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea98e4640 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faea90e3640 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca26630 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fae94009900 tx=0x7fae94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1033815a8a1505ee server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7faeaa0e5640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fae900a2230 0x7fae900a46f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0cbec0 tx=0x7fae9c090780 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7fae8ffff640 1 -- 172.21.15.8:0/376361999 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9402c5d0 con 0x55ed2ca26250 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7fae8ffff640 1 -- 172.21.15.8:0/376361999 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae94004030 con 0x55ed2ca26250 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.710+0000 7fae8ffff640 1 -- 172.21.15.8:0/376361999 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940052b0 con 0x55ed2ca26250 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 -- 172.21.15.8:0/376361999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 msgr2=0x55ed2ca26630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca26630 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fae94009900 tx=0x7fae94005fd0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 -- 172.21.15.8:0/376361999 shutdown_connections 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca2f750 0x55ed2ca31b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 --2- 172.21.15.8:0/376361999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca26630 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 -- 172.21.15.8:0/376361999 >> 172.21.15.8:0/376361999 conn(0x55ed2ca20980 msgr2=0x55ed2ca20d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 -- 172.21.15.8:0/376361999 shutdown_connections 2024-02-19T16:29:43.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 -- 172.21.15.8:0/376361999 wait complete. 2024-02-19T16:29:43.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 -- start start 2024-02-19T16:29:43.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.711+0000 7faeae802740 1 Processor -- start 2024-02-19T16:29:43.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 0x55ed2ca2d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c98e020 con 0x55ed2ca2f750 2024-02-19T16:29:43.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c661430 con 0x55ed2ca26250 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c61de00 con 0x55ed2ca26b70 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 10 client.0 _ll_register_callbacks cb 0x55ed2c935cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 0x55ed2ca2d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 0x55ed2ca2d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39626/0 (socket says 172.21.15.8:39626) 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 -- 172.21.15.8:0/3733046345 learned_addr learned my addr 172.21.15.8:0/3733046345 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea98e4640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 -- 172.21.15.8:0/3733046345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 msgr2=0x55ed2ca2ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 -- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 msgr2=0x55ed2ca2c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 -- 172.21.15.8:0/3733046345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea410dbe0 con 0x55ed2ca2f750 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faea90e3640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 0x55ed2ca2d1f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fae94009430 tx=0x7fae94004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26b69b12a257814 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7fae8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7fae8d7fa640 1 -- 172.21.15.8:0/3733046345 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94005260 con 0x55ed2ca2f750 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7fae8d7fa640 1 -- 172.21.15.8:0/3733046345 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fae940043b0 con 0x55ed2ca2f750 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed2c68d7e0 con 0x55ed2ca2f750 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7fae8d7fa640 1 -- 172.21.15.8:0/3733046345 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9402c890 con 0x55ed2ca2f750 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 10 client.17142 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.712+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed2c98c6a0 con 0x55ed2ca2f750 2024-02-19T16:29:43.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7fae8d7fa640 1 -- 172.21.15.8:0/3733046345 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed2c68d7e0 con 0x55ed2ca2f750 2024-02-19T16:29:43.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7fae8d7fa640 1 -- 172.21.15.8:0/3733046345 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fae9404b020 con 0x55ed2ca2f750 2024-02-19T16:29:43.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faeae802740 10 client.17142 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faeae802740 10 client.17142 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:29:43.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faeae802740 20 client.17142 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faeae802740 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca3f720 0x55ed2ca41b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faeae802740 4 client.17142 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed2c6ca720 con 0x55ed2ca3f720 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faea98e4640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca3f720 0x55ed2ca41b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7fae8d7fa640 10 client.17142 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.713+0000 7faea98e4640 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca3f720 0x55ed2ca41b40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef774572c107725d server_cookie=ce745d54b9689c2f in_seq=0 out_seq=0 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7fae8d7fa640 1 -- 172.21.15.8:0/3733046345 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed2c6ca720 con 0x55ed2ca3f720 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7fae8d7fa640 10 client.17142 handle_command_reply: tid=0 2024-02-19T16:29:43.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 client.17142 shutdown 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 msgr2=0x55ed2ca2d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 0x55ed2ca2d1f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fae94009430 tx=0x7fae94004df0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 shutdown_connections 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca3f720 0x55ed2ca41b40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca2f750 0x55ed2ca2d1f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca26b70 0x55ed2ca2ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7fae8d7fa640 0 client.17142 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:43.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 --2- 172.21.15.8:0/3733046345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 >> 172.21.15.8:0/3733046345 conn(0x55ed2ca20980 msgr2=0x55ed2ca226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 shutdown_connections 2024-02-19T16:29:43.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.714+0000 7faeae802740 1 -- 172.21.15.8:0/3733046345 wait complete. 2024-02-19T16:29:43.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.715+0000 7faeae802740 20 client.17142 trim_cache size 0 max 16384 2024-02-19T16:29:43.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 Processor -- start 2024-02-19T16:29:43.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 -- start start 2024-02-19T16:29:43.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 0x55ed2ca26f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca3aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2c913130 con 0x55ed2ca26b70 2024-02-19T16:29:43.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ca27490 con 0x55ed2ca26250 2024-02-19T16:29:43.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faeae802740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ca421a0 con 0x55ed2ca2f750 2024-02-19T16:29:43.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 0x55ed2ca26f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 0x55ed2ca26f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39628/0 (socket says 172.21.15.8:39628) 2024-02-19T16:29:43.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 -- 172.21.15.8:0/4232814025 learned_addr learned my addr 172.21.15.8:0/4232814025 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea90e3640 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca3aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea90e3640 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 -- 172.21.15.8:0/4232814025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 msgr2=0x55ed2ca3aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca3aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 -- 172.21.15.8:0/4232814025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 msgr2=0x55ed2ca2eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea98e4640 1 -- 172.21.15.8:0/4232814025 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2c928c20 con 0x55ed2ca26b70 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.727+0000 7faea90e3640 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faea98e4640 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 0x55ed2ca26f50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x55ed2ca2bfd0 tx=0x7faea0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b955fd66938d573 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7fae8f7fe640 1 -- 172.21.15.8:0/4232814025 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0017030 con 0x55ed2ca26b70 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7fae8f7fe640 1 -- 172.21.15.8:0/4232814025 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faea0018040 con 0x55ed2ca26b70 2024-02-19T16:29:43.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7fae8f7fe640 1 -- 172.21.15.8:0/4232814025 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea001f960 con 0x55ed2ca26b70 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 -- 172.21.15.8:0/4232814025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 msgr2=0x55ed2ca26f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 0x55ed2ca26f50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x55ed2ca2bfd0 tx=0x7faea0005f90 comp rx=0 tx=0).stop 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 -- 172.21.15.8:0/4232814025 shutdown_connections 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca3aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26b70 0x55ed2ca26f50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 --2- 172.21.15.8:0/4232814025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26250 0x55ed2ca2eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 -- 172.21.15.8:0/4232814025 >> 172.21.15.8:0/4232814025 conn(0x55ed2ca20980 msgr2=0x55ed2ca1fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 -- 172.21.15.8:0/4232814025 shutdown_connections 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.728+0000 7faeae802740 1 -- 172.21.15.8:0/4232814025 wait complete. 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 -- start start 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 Processor -- start 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26b70 0x55ed2ca34af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ca2a5d0 con 0x55ed2ca26250 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ca2a740 con 0x55ed2ca26b70 2024-02-19T16:29:43.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ca9afe0 con 0x55ed2ca2f750 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faeae802740 10 client.0 _ll_register_callbacks cb 0x7fae94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca345b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39634/0 (socket says 172.21.15.8:39634) 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 -- 172.21.15.8:0/1755998670 learned_addr learned my addr 172.21.15.8:0/1755998670 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea98e4640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26b70 0x55ed2ca34af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 -- 172.21.15.8:0/1755998670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 msgr2=0x55ed2ca35030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca35030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 -- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26b70 msgr2=0x55ed2ca34af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.729+0000 7faea90e3640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26b70 0x55ed2ca34af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faea90e3640 1 -- 172.21.15.8:0/1755998670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0009b00 con 0x55ed2ca26250 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faea90e3640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca345b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fae94009850 tx=0x7fae94050040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9c12ed80d07702 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7fae8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7fae8d7fa640 1 -- 172.21.15.8:0/1755998670 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94046070 con 0x55ed2ca26250 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae94004030 con 0x55ed2ca26250 2024-02-19T16:29:43.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7fae8d7fa640 1 -- 172.21.15.8:0/1755998670 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fae94000e40 con 0x55ed2ca26250 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7fae8d7fa640 1 -- 172.21.15.8:0/1755998670 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae94004e60 con 0x55ed2ca26250 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 10 client.17148 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed2c94e8c0 con 0x55ed2ca26250 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7fae8d7fa640 1 -- 172.21.15.8:0/1755998670 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae94004030 con 0x55ed2ca26250 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7fae8d7fa640 1 -- 172.21.15.8:0/1755998670 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fae9404b040 con 0x55ed2ca26250 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 10 client.17148 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 10 client.17148 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:29:43.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 20 client.17148 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca9c1f0 0x55ed2ca9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 4 client.17148 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.730+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed2c719890 con 0x55ed2ca9c1f0 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faea98e4640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca9c1f0 0x55ed2ca9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faea98e4640 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca9c1f0 0x55ed2ca9e5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9641d1471d230b6 server_cookie=336c53d3cc366f54 in_seq=0 out_seq=0 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7fae8d7fa640 10 client.17148 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7fae8d7fa640 1 -- 172.21.15.8:0/1755998670 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed2c719890 con 0x55ed2ca9c1f0 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7fae8d7fa640 10 client.17148 handle_command_reply: tid=0 2024-02-19T16:29:43.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 client.17148 shutdown 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 msgr2=0x55ed2ca345b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca345b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fae94009850 tx=0x7fae94050040 comp rx=0 tx=0).stop 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 shutdown_connections 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55ed2ca9c1f0 0x55ed2ca9e5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed2ca2f750 0x55ed2ca35030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ca26b70 0x55ed2ca34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7fae8d7fa640 0 client.17148 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:29:43.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 --2- 172.21.15.8:0/1755998670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ed2ca26250 0x55ed2ca345b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 >> 172.21.15.8:0/1755998670 conn(0x55ed2ca20980 msgr2=0x55ed2ca2c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 shutdown_connections 2024-02-19T16:29:43.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 1 -- 172.21.15.8:0/1755998670 wait complete. 2024-02-19T16:29:43.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.731+0000 7faeae802740 20 client.17148 trim_cache size 0 max 16384 2024-02-19T16:29:43.751 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:29:43.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fae900a2230 msgr2=0x7fae900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fae900a2230 0x7fae900a46f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0cbec0 tx=0x7fae9c090780 comp rx=0 tx=0).stop 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 msgr2=0x7faea41a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 0x7faea41a3a50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00eb40 tx=0x7fae9c00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 shutdown_connections 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea41064b0 0x7faea41a3a50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fae900a2230 0x7fae900a46f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea4100dd0 0x7faea4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 --2- 172.21.15.8:0/852249213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea4100410 0x7faea41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 >> 172.21.15.8:0/852249213 conn(0x7faea4077550 msgr2=0x7faea4103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 shutdown_connections 2024-02-19T16:29:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.732+0000 7faeabb48640 1 -- 172.21.15.8:0/852249213 wait complete. 2024-02-19T16:29:43.754 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/273974213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:29:43.755 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:29:43.756 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:29:43.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/273974213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:29:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 -- 172.21.15.8:0/3951963373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe641074f0 msgr2=0x7efe641078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3951963373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe641074f0 0x7efe641078d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efe58009930 tx=0x7efe5802bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 -- 172.21.15.8:0/3951963373 shutdown_connections 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3951963373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe640690b0 0x7efe6410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3951963373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe640686f0 0x7efe64068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3951963373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe641074f0 0x7efe641078d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 -- 172.21.15.8:0/3951963373 >> 172.21.15.8:0/3951963373 conn(0x7efe640774f0 msgr2=0x7efe64077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 -- 172.21.15.8:0/3951963373 shutdown_connections 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.858+0000 7efe69bf7640 1 -- 172.21.15.8:0/3951963373 wait complete. 2024-02-19T16:29:43.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 Processor -- start 2024-02-19T16:29:43.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 -- start start 2024-02-19T16:29:43.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe640686f0 0x7efe6419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 0x7efe6419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe641074f0 0x7efe641978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64197f10 con 0x7efe640690b0 2024-02-19T16:29:43.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64198080 con 0x7efe640686f0 2024-02-19T16:29:43.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.859+0000 7efe69bf7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64198240 con 0x7efe641074f0 2024-02-19T16:29:43.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 0x7efe6419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 0x7efe6419dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39650/0 (socket says 172.21.15.8:39650) 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 -- 172.21.15.8:0/3728491162 learned_addr learned my addr 172.21.15.8:0/3728491162 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe637fe640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe640686f0 0x7efe6419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe63fff640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe641074f0 0x7efe641978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 -- 172.21.15.8:0/3728491162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe641074f0 msgr2=0x7efe641978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe641074f0 0x7efe641978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 -- 172.21.15.8:0/3728491162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe640686f0 msgr2=0x7efe6419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe640686f0 0x7efe6419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 -- 172.21.15.8:0/3728491162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58009560 con 0x7efe640690b0 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe62ffd640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 0x7efe6419dc50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efe5400e9f0 tx=0x7efe5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe60ff9640 1 -- 172.21.15.8:0/3728491162 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54009700 con 0x7efe640690b0 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe64198520 con 0x7efe640690b0 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe60ff9640 1 -- 172.21.15.8:0/3728491162 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe5400cd80 con 0x7efe640690b0 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe60ff9640 1 -- 172.21.15.8:0/3728491162 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54018920 con 0x7efe640690b0 2024-02-19T16:29:43.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.860+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe641a47b0 con 0x7efe640690b0 2024-02-19T16:29:43.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe60ff9640 1 -- 172.21.15.8:0/3728491162 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7efe54018a80 con 0x7efe640690b0 2024-02-19T16:29:43.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 Processor -- start 2024-02-19T16:29:43.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 -- start start 2024-02-19T16:29:43.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 0x556159397a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x5561593a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 0x5561593a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615934e960 con 0x5561593a0b60 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556159302310 con 0x556159397660 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561592ff840 con 0x556159397f80 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe60ff9640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efe4c0a2230 0x7efe4c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 0x5561593a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe63fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 0x556159397a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe637fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x5561593a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe63fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 0x556159397a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48072/0 (socket says 172.21.15.8:48072) 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 0x5561593a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39662/0 (socket says 172.21.15.8:39662) 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe637fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x5561593a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:46298/0 (socket says 172.21.15.8:46298) 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe63fff640 1 -- 172.21.15.8:0/2515864131 learned_addr learned my addr 172.21.15.8:0/2515864131 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe60ff9640 1 -- 172.21.15.8:0/3728491162 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7efe54014070 con 0x7efe640690b0 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.863+0000 7efe63fff640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efe4c0a2230 0x7efe4c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 -- 172.21.15.8:0/2515864131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 msgr2=0x5561593a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x5561593a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 -- 172.21.15.8:0/2515864131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 msgr2=0x5561593a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 0x5561593a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 -- 172.21.15.8:0/2515864131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561592bd060 con 0x556159397660 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe62ffd640 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 0x5561593a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 0x556159397a40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe5c011e20 tx=0x7efe5c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ba3ac9ebebb6d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe63fff640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efe4c0a2230 0x7efe4c0a46f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00fc90 tx=0x7efe5c01e000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe49ffb640 1 -- 172.21.15.8:0/2515864131 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c01dcb0 con 0x556159397660 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe49ffb640 1 -- 172.21.15.8:0/2515864131 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe5c009e50 con 0x556159397660 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.864+0000 7efe49ffb640 1 -- 172.21.15.8:0/2515864131 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c004290 con 0x556159397660 2024-02-19T16:29:43.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 -- 172.21.15.8:0/2515864131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 msgr2=0x556159397a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 0x556159397a40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe5c011e20 tx=0x7efe5c00f790 comp rx=0 tx=0).stop 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 -- 172.21.15.8:0/2515864131 shutdown_connections 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5561593a0b60 0x5561593a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x5561593a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/2515864131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397660 0x556159397a40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 -- 172.21.15.8:0/2515864131 >> 172.21.15.8:0/2515864131 conn(0x556159391d90 msgr2=0x556159392170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 -- 172.21.15.8:0/2515864131 shutdown_connections 2024-02-19T16:29:43.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 -- 172.21.15.8:0/2515864131 wait complete. 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 -- start start 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.865+0000 7efe6c7d5740 1 Processor -- start 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x55615939dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x55615939e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x55615939e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561592fb020 con 0x556159397660 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556158fce430 con 0x5561593a0b60 2024-02-19T16:29:43.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556158f8ae00 con 0x556159397f80 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 10 client.0 _ll_register_callbacks cb 0x5561592a2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x55615939dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x55615939dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39678/0 (socket says 172.21.15.8:39678) 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 -- 172.21.15.8:0/78096074 learned_addr learned my addr 172.21.15.8:0/78096074 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe62ffd640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x55615939e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe63fff640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x55615939e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 -- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 msgr2=0x55615939e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x55615939e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 -- 172.21.15.8:0/78096074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 msgr2=0x55615939e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x55615939e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 -- 172.21.15.8:0/78096074 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c019040 con 0x556159397660 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe637fe640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x55615939dc00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efe58006010 tx=0x7efe5802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fb4703fbfc488d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe42ffd640 1 -- 172.21.15.8:0/78096074 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58004040 con 0x556159397660 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe42ffd640 1 -- 172.21.15.8:0/78096074 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7efe580041a0 con 0x556159397660 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556159037720 con 0x556159397660 2024-02-19T16:29:43.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe42ffd640 1 -- 172.21.15.8:0/78096074 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe58004360 con 0x556159397660 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 10 client.17160 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.866+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5561592f96a0 con 0x556159397660 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe42ffd640 1 -- 172.21.15.8:0/78096074 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556159037720 con 0x556159397660 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe42ffd640 1 -- 172.21.15.8:0/78096074 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efe5803c040 con 0x556159397660 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe6c7d5740 10 client.17160 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe6c7d5740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe6c7d5740 20 client.17160 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5561593b0f60 0x5561593b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe6c7d5740 4 client.17160 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556159086890 con 0x5561593b0f60 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe62ffd640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5561593b0f60 0x5561593b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe62ffd640 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5561593b0f60 0x5561593b3350 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b5986dbb8070f3 server_cookie=e40a9e53da1a3aaf in_seq=0 out_seq=0 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.867+0000 7efe42ffd640 10 client.17160 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe42ffd640 1 -- 172.21.15.8:0/78096074 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556159086890 con 0x5561593b0f60 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe42ffd640 10 client.17160 handle_command_reply: tid=0 2024-02-19T16:29:43.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 client.17160 shutdown 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 msgr2=0x55615939dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x55615939dc00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efe58006010 tx=0x7efe5802ca10 comp rx=0 tx=0).stop 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 shutdown_connections 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x55615939e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5561593b0f60 0x5561593b3350 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397f80 0x55615939e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/78096074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x55615939dc00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe42ffd640 0 client.17160 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 >> 172.21.15.8:0/78096074 conn(0x556159391d90 msgr2=0x556159393250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 shutdown_connections 2024-02-19T16:29:43.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 1 -- 172.21.15.8:0/78096074 wait complete. 2024-02-19T16:29:43.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.868+0000 7efe6c7d5740 20 client.17160 trim_cache size 0 max 16384 2024-02-19T16:29:43.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 Processor -- start 2024-02-19T16:29:43.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 -- start start 2024-02-19T16:29:43.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397660 0x55615938cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 0x55615939e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x5561593938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556159280130 con 0x556159397f80 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556159389360 con 0x5561593a0b60 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615939c1d0 con 0x556159397660 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 0x55615939e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 0x55615939e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39682/0 (socket says 172.21.15.8:39682) 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 -- 172.21.15.8:0/4152976002 learned_addr learned my addr 172.21.15.8:0/4152976002 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe637fe640 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397660 0x55615938cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe63fff640 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x5561593938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 -- 172.21.15.8:0/4152976002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397660 msgr2=0x55615938cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397660 0x55615938cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 -- 172.21.15.8:0/4152976002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 msgr2=0x5561593938d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x5561593938d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 -- 172.21.15.8:0/4152976002 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556159295c20 con 0x556159397f80 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.881+0000 7efe62ffd640 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 0x55615939e4a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efe540978c0 tx=0x7efe5406e020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751aa529fc39fc81 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe497fa640 1 -- 172.21.15.8:0/4152976002 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe540a35e0 con 0x556159397f80 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe497fa640 1 -- 172.21.15.8:0/4152976002 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe540991f0 con 0x556159397f80 2024-02-19T16:29:43.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe497fa640 1 -- 172.21.15.8:0/4152976002 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe540ac540 con 0x556159397f80 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 -- 172.21.15.8:0/4152976002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 msgr2=0x55615939e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 0x55615939e4a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efe540978c0 tx=0x7efe5406e020 comp rx=0 tx=0).stop 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 -- 172.21.15.8:0/4152976002 shutdown_connections 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561593a0b60 0x5561593938d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397f80 0x55615939e4a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/4152976002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556159397660 0x55615938cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 -- 172.21.15.8:0/4152976002 >> 172.21.15.8:0/4152976002 conn(0x556159391d90 msgr2=0x5561593931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 -- 172.21.15.8:0/4152976002 shutdown_connections 2024-02-19T16:29:43.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 -- 172.21.15.8:0/4152976002 wait complete. 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.882+0000 7efe6c7d5740 1 -- start start 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 Processor -- start 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x5561593a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397f80 0x5561593a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561593a0b60 0x5561593a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561593a5670 con 0x556159397660 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561593a57e0 con 0x556159397f80 2024-02-19T16:29:43.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55615940c7b0 con 0x5561593a0b60 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe6c7d5740 10 client.0 _ll_register_callbacks cb 0x7efe58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x5561593a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x5561593a8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39694/0 (socket says 172.21.15.8:39694) 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 -- 172.21.15.8:0/1654021330 learned_addr learned my addr 172.21.15.8:0/1654021330 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe62ffd640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397f80 0x5561593a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe63fff640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561593a0b60 0x5561593a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 -- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561593a0b60 msgr2=0x5561593a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561593a0b60 0x5561593a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 -- 172.21.15.8:0/1654021330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397f80 msgr2=0x5561593a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397f80 0x5561593a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 -- 172.21.15.8:0/1654021330 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe64198520 con 0x556159397660 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe63fff640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561593a0b60 0x5561593a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe637fe640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x5561593a8700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efe580038b0 tx=0x7efe58004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f2109b847a1b8c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.883+0000 7efe42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe42ffd640 1 -- 172.21.15.8:0/1654021330 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58005850 con 0x556159397660 2024-02-19T16:29:43.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe42ffd640 1 -- 172.21.15.8:0/1654021330 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7efe58000e40 con 0x556159397660 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556158ffa7e0 con 0x556159397660 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe42ffd640 1 -- 172.21.15.8:0/1654021330 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe580035f0 con 0x556159397660 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 10 client.17166 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5561592bb8c0 con 0x556159397660 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe42ffd640 1 -- 172.21.15.8:0/1654021330 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556158ffa7e0 con 0x556159397660 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe42ffd640 1 -- 172.21.15.8:0/1654021330 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efe5803c040 con 0x556159397660 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 10 client.17166 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:43.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 20 client.17166 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55615940dc60 0x556159410080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 4 client.17166 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556158fa6700 con 0x55615940dc60 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.884+0000 7efe62ffd640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55615940dc60 0x556159410080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe62ffd640 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55615940dc60 0x556159410080 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4280947b0559edd server_cookie=ad52374f2c1428cf in_seq=0 out_seq=0 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe42ffd640 10 client.17166 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe42ffd640 1 -- 172.21.15.8:0/1654021330 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556158fa6700 con 0x55615940dc60 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe42ffd640 10 client.17166 handle_command_reply: tid=0 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 client.17166 shutdown 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 msgr2=0x5561593a8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x5561593a8700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efe580038b0 tx=0x7efe58004540 comp rx=0 tx=0).stop 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 shutdown_connections 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55615940dc60 0x556159410080 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561593a0b60 0x5561593a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556159397f80 0x5561593a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe42ffd640 0 client.17166 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 --2- 172.21.15.8:0/1654021330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556159397660 0x5561593a8700 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 >> 172.21.15.8:0/1654021330 conn(0x556159391d90 msgr2=0x556159393a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 shutdown_connections 2024-02-19T16:29:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 1 -- 172.21.15.8:0/1654021330 wait complete. 2024-02-19T16:29:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.885+0000 7efe6c7d5740 20 client.17166 trim_cache size 0 max 16384 2024-02-19T16:29:43.903 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:29:43.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efe4c0a2230 msgr2=0x7efe4c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efe4c0a2230 0x7efe4c0a46f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00fc90 tx=0x7efe5c01e000 comp rx=0 tx=0).stop 2024-02-19T16:29:43.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 msgr2=0x7efe6419dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:43.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 0x7efe6419dc50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efe5400e9f0 tx=0x7efe5400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 shutdown_connections 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7efe4c0a2230 0x7efe4c0a46f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe641074f0 0x7efe641978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efe640690b0 0x7efe6419dc50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 --2- 172.21.15.8:0/3728491162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe640686f0 0x7efe6419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 >> 172.21.15.8:0/3728491162 conn(0x7efe640774f0 msgr2=0x7efe640fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 shutdown_connections 2024-02-19T16:29:43.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:43.886+0000 7efe69bf7640 1 -- 172.21.15.8:0/3728491162 wait complete. 2024-02-19T16:29:44.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/273974213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:29:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:44 smithi156 ceph-mon[36502]: pgmap v210: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:44.906 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:29:44.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:29:44.907 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:29:44.935 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:44 smithi008 ceph-mon[32573]: pgmap v210: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:45.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 -- 172.21.15.8:0/2942339186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc101ae0 msgr2=0x7fe7dc105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2942339186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc101ae0 0x7fe7dc105080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009930 tx=0x7fe7c802bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:45.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 -- 172.21.15.8:0/2942339186 shutdown_connections 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2942339186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1057b0 0x7fe7dc10d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2942339186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc101ae0 0x7fe7dc105080 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2942339186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc101130 0x7fe7dc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 -- 172.21.15.8:0/2942339186 >> 172.21.15.8:0/2942339186 conn(0x7fe7dc075730 msgr2=0x7fe7dc075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 -- 172.21.15.8:0/2942339186 shutdown_connections 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.025+0000 7fe7e315c640 1 -- 172.21.15.8:0/2942339186 wait complete. 2024-02-19T16:29:45.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 Processor -- start 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 -- start start 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc101130 0x7fe7dc19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1057b0 0x7fe7dc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 0x7fe7dc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1983a0 con 0x7fe7dc1979b0 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc198510 con 0x7fe7dc1057b0 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e315c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1986d0 con 0x7fe7dc101130 2024-02-19T16:29:45.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.026+0000 7fe7e16f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 0x7fe7dc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 0x7fe7dc197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39714/0 (socket says 172.21.15.8:39714) 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1057b0 0x7fe7dc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 -- 172.21.15.8:0/2385789635 learned_addr learned my addr 172.21.15.8:0/2385789635 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc101130 0x7fe7dc19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 -- 172.21.15.8:0/2385789635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc101130 msgr2=0x7fe7dc19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc101130 0x7fe7dc19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 -- 172.21.15.8:0/2385789635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1057b0 msgr2=0x7fe7dc19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1057b0 0x7fe7dc19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 -- 172.21.15.8:0/2385789635 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8009560 con 0x7fe7dc1979b0 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 0x7fe7dc197e60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400eb10 tx=0x7fe7d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7d9ffb640 1 -- 172.21.15.8:0/2385789635 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4004b20 con 0x7fe7dc1979b0 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7d9ffb640 1 -- 172.21.15.8:0/2385789635 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d4004c80 con 0x7fe7dc1979b0 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc102050 con 0x7fe7dc1979b0 2024-02-19T16:29:45.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7d9ffb640 1 -- 172.21.15.8:0/2385789635 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4010650 con 0x7fe7dc1979b0 2024-02-19T16:29:45.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.027+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc1024e0 con 0x7fe7dc1979b0 2024-02-19T16:29:45.034 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:44 smithi191 ceph-mon[37203]: pgmap v210: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:45.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.029+0000 7fe7d9ffb640 1 -- 172.21.15.8:0/2385789635 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe7d4005020 con 0x7fe7dc1979b0 2024-02-19T16:29:45.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 Processor -- start 2024-02-19T16:29:45.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 -- start start 2024-02-19T16:29:45.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7d9ffb640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe7cc0a2230 0x7fe7cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd2a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd2a8b0 0x55785fd2cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fc8ff90 con 0x55785fd21d30 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fc8d4c0 con 0x55785fd21380 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.030+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fc88ca0 con 0x55785fd2a8b0 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7dbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd21760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:48132/0 (socket says 172.21.15.8:48132) 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7dbfff640 1 -- 172.21.15.8:0/1646229091 learned_addr learned my addr 172.21.15.8:0/1646229091 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd2a8b0 0x55785fd2cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe7cc0a2230 0x7fe7cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7d9ffb640 1 -- 172.21.15.8:0/2385789635 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe7d4014070 con 0x7fe7dc1979b0 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd2a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 -- 172.21.15.8:0/1646229091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 msgr2=0x55785fd21760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd21760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 -- 172.21.15.8:0/1646229091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 msgr2=0x55785fd2a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd2a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 -- 172.21.15.8:0/1646229091 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55785fc4b0b0 con 0x55785fd2a8b0 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd2a8b0 0x55785fd2cd00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40cbdd0 tx=0x7fe7d4009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f1c94069e395fc server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe7cc0a2230 0x7fe7cc0a46f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc198af0 tx=0x7fe7d0005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7c6ffd640 1 -- 172.21.15.8:0/1646229091 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4066ce0 con 0x55785fd2a8b0 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7c6ffd640 1 -- 172.21.15.8:0/1646229091 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d409b070 con 0x55785fd2a8b0 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.031+0000 7fe7c6ffd640 1 -- 172.21.15.8:0/1646229091 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40b4160 con 0x55785fd2a8b0 2024-02-19T16:29:45.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/1646229091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd2a8b0 msgr2=0x55785fd2cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd2a8b0 0x55785fd2cd00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40cbdd0 tx=0x7fe7d4009a70 comp rx=0 tx=0).stop 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/1646229091 shutdown_connections 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd2a8b0 0x55785fd2cd00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd2a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/1646229091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd21760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/1646229091 >> 172.21.15.8:0/1646229091 conn(0x55785fd1b840 msgr2=0x55785fd1bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/1646229091 shutdown_connections 2024-02-19T16:29:45.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/1646229091 wait complete. 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 -- start start 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.032+0000 7fe7e5e0c740 1 Processor -- start 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd27800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd27d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd2a8b0 0x55785fd28280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fc2df40 con 0x55785fd21d30 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785f95c4b0 con 0x55785fd2a8b0 2024-02-19T16:29:45.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fc0e670 con 0x55785fd21380 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e5e0c740 10 client.0 _ll_register_callbacks cb 0x55785fc7bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd27d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd27d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39744/0 (socket says 172.21.15.8:39744) 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 -- 172.21.15.8:0/2005459966 learned_addr learned my addr 172.21.15.8:0/2005459966 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd27800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd2a8b0 0x55785fd28280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 -- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 msgr2=0x55785fd27800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd27800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 -- 172.21.15.8:0/2005459966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd2a8b0 msgr2=0x55785fd28280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd2a8b0 0x55785fd28280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 -- 172.21.15.8:0/2005459966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc1024e0 con 0x55785fd21d30 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd27d40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009230 tx=0x7fe7c802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcb8320a0efe5e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.033+0000 7fe7bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7bbfff640 1 -- 172.21.15.8:0/2005459966 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8004250 con 0x55785fd21d30 2024-02-19T16:29:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7bbfff640 1 -- 172.21.15.8:0/2005459966 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe7c80043b0 con 0x55785fd21d30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55785fc62f40 con 0x55785fd21d30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7bbfff640 1 -- 172.21.15.8:0/2005459966 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c8004df0 con 0x55785fd21d30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 10 client.17178 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55785fc87320 con 0x55785fd21d30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7bbfff640 1 -- 172.21.15.8:0/2005459966 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55785fc62f40 con 0x55785fd21d30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7bbfff640 1 -- 172.21.15.8:0/2005459966 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7c80039a0 con 0x55785fd21d30 2024-02-19T16:29:45.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 10 client.17178 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 20 client.17178 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd3ae60 0x55785fd3d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 4 client.17178 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55785fc7f310 con 0x55785fd3ae60 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.034+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd3ae60 0x55785fd3d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.035+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd3ae60 0x55785fd3d280 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5ab06b4e7fbb04 server_cookie=5728086eade49d16 in_seq=0 out_seq=0 2024-02-19T16:29:45.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.035+0000 7fe7bbfff640 10 client.17178 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.035+0000 7fe7bbfff640 1 -- 172.21.15.8:0/2005459966 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55785fc7f310 con 0x55785fd3ae60 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.035+0000 7fe7bbfff640 10 client.17178 handle_command_reply: tid=0 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.035+0000 7fe7e5e0c740 1 client.17178 shutdown 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 msgr2=0x55785fd27d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd27d40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009230 tx=0x7fe7c802c720 comp rx=0 tx=0).stop 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 shutdown_connections 2024-02-19T16:29:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd2a8b0 0x55785fd28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd21d30 0x55785fd27d40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd3ae60 0x55785fd3d280 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/2005459966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7bbfff640 0 client.17178 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 >> 172.21.15.8:0/2005459966 conn(0x55785fd1b840 msgr2=0x55785fd1d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 shutdown_connections 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/2005459966 wait complete. 2024-02-19T16:29:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.036+0000 7fe7e5e0c740 20 client.17178 trim_cache size 0 max 16384 2024-02-19T16:29:45.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 Processor -- start 2024-02-19T16:29:45.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 -- start start 2024-02-19T16:29:45.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd29f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21d30 0x55785fd280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fd230a0 con 0x55785fd2a8b0 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fd378f0 con 0x55785fd21380 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fd22670 con 0x55785fd21d30 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e16f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e16f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd35db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39746/0 (socket says 172.21.15.8:39746) 2024-02-19T16:29:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e16f9640 1 -- 172.21.15.8:0/789804737 learned_addr learned my addr 172.21.15.8:0/789804737 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd29f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.052+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21d30 0x55785fd280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e16f9640 1 -- 172.21.15.8:0/789804737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21d30 msgr2=0x55785fd280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21d30 0x55785fd280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e16f9640 1 -- 172.21.15.8:0/789804737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 msgr2=0x55785fd29f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd29f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e16f9640 1 -- 172.21.15.8:0/789804737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55785fc23c00 con 0x55785fd2a8b0 2024-02-19T16:29:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd35db0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40a32e0 tx=0x7fe7d4009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad14174b4fbd5ec server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7c67fc640 1 -- 172.21.15.8:0/789804737 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40a3810 con 0x55785fd2a8b0 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7c67fc640 1 -- 172.21.15.8:0/789804737 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d409f1c0 con 0x55785fd2a8b0 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7c67fc640 1 -- 172.21.15.8:0/789804737 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40cc050 con 0x55785fd2a8b0 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/789804737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 msgr2=0x55785fd35db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.053+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd35db0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40a32e0 tx=0x7fe7d4009200 comp rx=0 tx=0).stop 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/789804737 shutdown_connections 2024-02-19T16:29:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd35db0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21d30 0x55785fd280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/789804737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21380 0x55785fd29f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/789804737 >> 172.21.15.8:0/789804737 conn(0x55785fd1b840 msgr2=0x55785fd1a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/789804737 shutdown_connections 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/789804737 wait complete. 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- start start 2024-02-19T16:29:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 Processor -- start 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd24550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21d30 0x55785fd24a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fd251b0 con 0x55785fd2a8b0 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fd95c10 con 0x55785fd21d30 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785fd95d80 con 0x55785fd21380 2024-02-19T16:29:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e5e0c740 10 client.0 _ll_register_callbacks cb 0x7fe7c80049b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e16f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e16f9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39750/0 (socket says 172.21.15.8:39750) 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7e16f9640 1 -- 172.21.15.8:0/3232462221 learned_addr learned my addr 172.21.15.8:0/3232462221 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.054+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21d30 0x55785fd24a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd24550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e16f9640 1 -- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 msgr2=0x55785fd24550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd24550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e16f9640 1 -- 172.21.15.8:0/3232462221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21d30 msgr2=0x55785fd24a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21d30 0x55785fd24a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e16f9640 1 -- 172.21.15.8:0/3232462221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55785fc4b0b0 con 0x55785fd2a8b0 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7dbfff640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21d30 0x55785fd24a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e16f9640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd956d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe7d409b000 tx=0x7fe7d40b3da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81769d759edef4ce server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7bbfff640 1 -- 172.21.15.8:0/3232462221 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d409b4d0 con 0x55785fd2a8b0 2024-02-19T16:29:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7bbfff640 1 -- 172.21.15.8:0/3232462221 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe7d409b070 con 0x55785fd2a8b0 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55785fcae900 con 0x55785fd2a8b0 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7bbfff640 1 -- 172.21.15.8:0/3232462221 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d4097310 con 0x55785fd2a8b0 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e5e0c740 10 client.17184 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55785fc49910 con 0x55785fd2a8b0 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7bbfff640 1 -- 172.21.15.8:0/3232462221 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d4079e90 con 0x55785fd2a8b0 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.055+0000 7fe7bbfff640 1 -- 172.21.15.8:0/3232462221 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7d401f060 con 0x55785fd2a8b0 2024-02-19T16:29:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e5e0c740 10 client.17184 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e5e0c740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e5e0c740 20 client.17184 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd971c0 0x55785fd995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e5e0c740 4 client.17184 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55785f988860 con 0x55785fd971c0 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd971c0 0x55785fd995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7e0ef8640 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd971c0 0x55785fd995b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06043f95e9f66cb server_cookie=313d2b0a88b58657 in_seq=0 out_seq=0 2024-02-19T16:29:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.056+0000 7fe7bbfff640 10 client.17184 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7bbfff640 1 -- 172.21.15.8:0/3232462221 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55785f988860 con 0x55785fd971c0 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7bbfff640 10 client.17184 handle_command_reply: tid=0 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 client.17184 shutdown 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 msgr2=0x55785fd956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd956d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe7d409b000 tx=0x7fe7d40b3da0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 shutdown_connections 2024-02-19T16:29:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55785fd2a8b0 0x55785fd956d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55785fd21d30 0x55785fd24a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55785fd971c0 0x55785fd995b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7bbfff640 0 client.17184 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 --2- 172.21.15.8:0/3232462221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55785fd21380 0x55785fd24550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.057+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 >> 172.21.15.8:0/3232462221 conn(0x55785fd1b840 msgr2=0x55785fd1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 shutdown_connections 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e5e0c740 1 -- 172.21.15.8:0/3232462221 wait complete. 2024-02-19T16:29:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e5e0c740 20 client.17184 trim_cache size 0 max 16384 2024-02-19T16:29:45.071 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:29:45.071 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:29:45.071 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "bf42fe0c-1b5f-478c-91b5-e4ca560997cc", 2024-02-19T16:29:45.071 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:29:45.071 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:29:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe7cc0a2230 msgr2=0x7fe7cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe7cc0a2230 0x7fe7cc0a46f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc198af0 tx=0x7fe7d0005f90 comp rx=0 tx=0).stop 2024-02-19T16:29:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 msgr2=0x7fe7dc197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 0x7fe7dc197e60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400eb10 tx=0x7fe7d400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 shutdown_connections 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7dc1979b0 0x7fe7dc197e60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7dc1057b0 0x7fe7dc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe7cc0a2230 0x7fe7cc0a46f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 --2- 172.21.15.8:0/2385789635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7dc101130 0x7fe7dc19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 >> 172.21.15.8:0/2385789635 conn(0x7fe7dc075730 msgr2=0x7fe7dc104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 shutdown_connections 2024-02-19T16:29:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.058+0000 7fe7e315c640 1 -- 172.21.15.8:0/2385789635 wait complete. 2024-02-19T16:29:45.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:29:45.075 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:29:45.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 -- 172.21.15.8:0/2399525507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190107500 msgr2=0x7f51901078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 --2- 172.21.15.8:0/2399525507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190107500 0x7f51901078e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5180009930 tx=0x7f518002bc30 comp rx=0 tx=0).stop 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 -- 172.21.15.8:0/2399525507 shutdown_connections 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 --2- 172.21.15.8:0/2399525507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5190101e20 0x7f519010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 --2- 172.21.15.8:0/2399525507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190101460 0x7f51901018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 --2- 172.21.15.8:0/2399525507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190107500 0x7f51901078e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 -- 172.21.15.8:0/2399525507 >> 172.21.15.8:0/2399525507 conn(0x7f51900fcf90 msgr2=0x7f51900ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 -- 172.21.15.8:0/2399525507 shutdown_connections 2024-02-19T16:29:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.222+0000 7f5196888640 1 -- 172.21.15.8:0/2399525507 wait complete. 2024-02-19T16:29:45.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 Processor -- start 2024-02-19T16:29:45.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 -- start start 2024-02-19T16:29:45.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 0x7f519019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5190101e20 0x7f519019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190107500 0x7f5190197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190197fd0 con 0x7f5190101460 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190198140 con 0x7f5190107500 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f5196888640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190198300 con 0x7f5190101e20 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f518ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 0x7f519019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.223+0000 7f518ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 0x7f519019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39760/0 (socket says 172.21.15.8:39760) 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 -- 172.21.15.8:0/1165435528 learned_addr learned my addr 172.21.15.8:0/1165435528 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518f7fe640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5190101e20 0x7f519019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f5194e25640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190107500 0x7f5190197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 -- 172.21.15.8:0/1165435528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5190101e20 msgr2=0x7f519019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5190101e20 0x7f519019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 -- 172.21.15.8:0/1165435528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190107500 msgr2=0x7f5190197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190107500 0x7f5190197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 -- 172.21.15.8:0/1165435528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180009560 con 0x7f5190101460 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f5194e25640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190107500 0x7f5190197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518ffff640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 0x7f519019d7d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5180009230 tx=0x7f518002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518d7fa640 1 -- 172.21.15.8:0/1165435528 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180004350 con 0x7f5190101460 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518d7fa640 1 -- 172.21.15.8:0/1165435528 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51800044b0 con 0x7f5190101460 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f518d7fa640 1 -- 172.21.15.8:0/1165435528 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180003d70 con 0x7f5190101460 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5190198580 con 0x7f5190101460 2024-02-19T16:29:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.224+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51901a4870 con 0x7f5190101460 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.227+0000 7f518d7fa640 1 -- 172.21.15.8:0/1165435528 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f5180004df0 con 0x7f5190101460 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 Processor -- start 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 -- start start 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518d7fa640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f51780a22b0 0x7f51780a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a65690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a6e7b0 0x562682a70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626829d4280 con 0x562682a652b0 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626829d17b0 con 0x562682a65bd0 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f519946f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626829ccf90 con 0x562682a6e7b0 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f5194e25640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f51780a22b0 0x7f51780a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a65690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518d7fa640 1 -- 172.21.15.8:0/1165435528 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f51800f03f0 con 0x7f5190101460 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a65690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39776/0 (socket says 172.21.15.8:39776) 2024-02-19T16:29:45.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 -- 172.21.15.8:0/3388760044 learned_addr learned my addr 172.21.15.8:0/3388760044 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f5194e25640 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a6e7b0 0x562682a70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518f7fe640 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 -- 172.21.15.8:0/3388760044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a6e7b0 msgr2=0x562682a70ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a6e7b0 0x562682a70ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 -- 172.21.15.8:0/3388760044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 msgr2=0x562682a6e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.228+0000 7f518ffff640 1 -- 172.21.15.8:0/3388760044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56268298b570 con 0x562682a652b0 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.229+0000 7f518ffff640 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a65690 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f51800ed850 tx=0x7f51800bc4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413372e4acab6d07 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.229+0000 7f5194e25640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f51780a22b0 0x7f51780a4770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5188009fd0 tx=0x7f5188009ee0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.229+0000 7f51767fc640 1 -- 172.21.15.8:0/3388760044 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800c0be0 con 0x562682a652b0 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.229+0000 7f51767fc640 1 -- 172.21.15.8:0/3388760044 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51800c5230 con 0x562682a652b0 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.229+0000 7f51767fc640 1 -- 172.21.15.8:0/3388760044 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800b9100 con 0x562682a652b0 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.229+0000 7f519946f740 1 -- 172.21.15.8:0/3388760044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 msgr2=0x562682a65690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a65690 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f51800ed850 tx=0x7f51800bc4b0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- 172.21.15.8:0/3388760044 shutdown_connections 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a6e7b0 0x562682a70ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- 172.21.15.8:0/3388760044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a65690 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- 172.21.15.8:0/3388760044 >> 172.21.15.8:0/3388760044 conn(0x562682a5f990 msgr2=0x562682a5fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- 172.21.15.8:0/3388760044 shutdown_connections 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- 172.21.15.8:0/3388760044 wait complete. 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- start start 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 Processor -- start 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a65bd0 0x562682a6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a6e7b0 0x562682a6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626826a0430 con 0x562682a652b0 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268265ce00 con 0x562682a6e7b0 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.230+0000 7f519946f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682951fd0 con 0x562682a65bd0 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f519946f740 10 client.0 _ll_register_callbacks cb 0x5626829a7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a6e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39792/0 (socket says 172.21.15.8:39792) 2024-02-19T16:29:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a65bd0 0x562682a6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 -- 172.21.15.8:0/1334573066 learned_addr learned my addr 172.21.15.8:0/1334573066 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f5194e25640 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a6e7b0 0x562682a6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 -- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a65bd0 msgr2=0x562682a6b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a65bd0 0x562682a6b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 -- 172.21.15.8:0/1334573066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a6e7b0 msgr2=0x562682a6bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a6e7b0 0x562682a6bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 -- 172.21.15.8:0/1334573066 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51901a4870 con 0x562682a652b0 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f518ffff640 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a6e200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5180000c00 tx=0x7f51800bcbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8608a3c12a869d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f516f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f516f7fe640 1 -- 172.21.15.8:0/1334573066 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800c0be0 con 0x562682a652b0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f516f7fe640 1 -- 172.21.15.8:0/1334573066 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f51800ab1d0 con 0x562682a652b0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626826cc7e0 con 0x562682a652b0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f516f7fe640 1 -- 172.21.15.8:0/1334573066 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51800ab390 con 0x562682a652b0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f519946f740 10 client.17196 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.231+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626829cb610 con 0x562682a652b0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f516f7fe640 1 -- 172.21.15.8:0/1334573066 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626826cc7e0 con 0x562682a652b0 2024-02-19T16:29:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f516f7fe640 1 -- 172.21.15.8:0/1334573066 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f518003c070 con 0x562682a652b0 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f519946f740 10 client.17196 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f519946f740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f519946f740 20 client.17196 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f519946f740 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682a7e390 0x562682a80780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f519946f740 4 client.17196 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562682709720 con 0x562682a7e390 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.232+0000 7f518f7fe640 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682a7e390 0x562682a80780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.233+0000 7f518f7fe640 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682a7e390 0x562682a80780 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed051ab9202a29bc server_cookie=d0323ef2e4bff14a in_seq=0 out_seq=0 2024-02-19T16:29:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.233+0000 7f516f7fe640 10 client.17196 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f516f7fe640 1 -- 172.21.15.8:0/1334573066 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562682709720 con 0x562682a7e390 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f516f7fe640 10 client.17196 handle_command_reply: tid=0 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 client.17196 shutdown 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 msgr2=0x562682a6e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a6e200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5180000c00 tx=0x7f51800bcbd0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 shutdown_connections 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a6e7b0 0x562682a6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682a7e390 0x562682a80780 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a65bd0 0x562682a6b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 --2- 172.21.15.8:0/1334573066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a652b0 0x562682a6e200 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f516f7fe640 0 client.17196 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 >> 172.21.15.8:0/1334573066 conn(0x562682a5f990 msgr2=0x562682a61730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 shutdown_connections 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 1 -- 172.21.15.8:0/1334573066 wait complete. 2024-02-19T16:29:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.234+0000 7f519946f740 20 client.17196 trim_cache size 0 max 16384 2024-02-19T16:29:45.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 Processor -- start 2024-02-19T16:29:45.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 -- start start 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a6dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682a6caf0 con 0x562682a6e7b0 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682a6c700 con 0x562682a65bd0 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f519946f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682709720 con 0x562682a652b0 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f5194e25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f5194e25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a798f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39794/0 (socket says 172.21.15.8:39794) 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f5194e25640 1 -- 172.21.15.8:0/512187939 learned_addr learned my addr 172.21.15.8:0/512187939 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f518f7fe640 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.252+0000 7f518ffff640 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a6dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5194e25640 1 -- 172.21.15.8:0/512187939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 msgr2=0x562682a6dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5194e25640 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a6dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5194e25640 1 -- 172.21.15.8:0/512187939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 msgr2=0x562682a6e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5194e25640 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5194e25640 1 -- 172.21.15.8:0/512187939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562682967ac0 con 0x562682a6e7b0 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5194e25640 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a798f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5188014f00 tx=0x7f518801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1abbccedd459656 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5175ffb640 1 -- 172.21.15.8:0/512187939 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518802c680 con 0x562682a6e7b0 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5175ffb640 1 -- 172.21.15.8:0/512187939 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5188025070 con 0x562682a6e7b0 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.253+0000 7f5175ffb640 1 -- 172.21.15.8:0/512187939 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880105d0 con 0x562682a6e7b0 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 -- 172.21.15.8:0/512187939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 msgr2=0x562682a798f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a798f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5188014f00 tx=0x7f518801d000 comp rx=0 tx=0).stop 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 -- 172.21.15.8:0/512187939 shutdown_connections 2024-02-19T16:29:45.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a798f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a6e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 --2- 172.21.15.8:0/512187939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a6dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 -- 172.21.15.8:0/512187939 >> 172.21.15.8:0/512187939 conn(0x562682a5f990 msgr2=0x562682a5fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 -- 172.21.15.8:0/512187939 shutdown_connections 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 -- 172.21.15.8:0/512187939 wait complete. 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.254+0000 7f519946f740 1 -- start start 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 Processor -- start 2024-02-19T16:29:45.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a71930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a71e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682ad9bd0 con 0x562682a6e7b0 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682ad9d10 con 0x562682a65bd0 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682ad9e50 con 0x562682a652b0 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f519946f740 10 client.0 _ll_register_callbacks cb 0x7f51840097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a71e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a71e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39804/0 (socket says 172.21.15.8:39804) 2024-02-19T16:29:45.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f518f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a71930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 -- 172.21.15.8:0/2349412644 learned_addr learned my addr 172.21.15.8:0/2349412644 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f518ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 -- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 msgr2=0x562682a713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 -- 172.21.15.8:0/2349412644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 msgr2=0x562682a71930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a71930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 -- 172.21.15.8:0/2349412644 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5188014c90 con 0x562682a6e7b0 2024-02-19T16:29:45.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f5194e25640 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a71e70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5188004ee0 tx=0x7f5188021d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4654909c1a72efd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.255+0000 7f516f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f516f7fe640 1 -- 172.21.15.8:0/2349412644 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188021800 con 0x562682a6e7b0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f516f7fe640 1 -- 172.21.15.8:0/2349412644 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f5188027070 con 0x562682a6e7b0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51800c5230 con 0x562682a6e7b0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f516f7fe640 1 -- 172.21.15.8:0/2349412644 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5188021be0 con 0x562682a6e7b0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f519946f740 10 client.17202 fetch_fsmap learned FSMap version 30 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56268298d6a0 con 0x562682a6e7b0 2024-02-19T16:29:45.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.256+0000 7f516f7fe640 1 -- 172.21.15.8:0/2349412644 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51800c5230 con 0x562682a6e7b0 2024-02-19T16:29:45.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f516f7fe640 1 -- 172.21.15.8:0/2349412644 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5188033030 con 0x562682a6e7b0 2024-02-19T16:29:45.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f519946f740 10 client.17202 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:29:45.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f519946f740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:29:45.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f519946f740 20 client.17202 populate_metadata read hostname 'smithi008' 2024-02-19T16:29:45.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f519946f740 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682adb5b0 0x562682add9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f519946f740 4 client.17202 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562682758890 con 0x562682adb5b0 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f518ffff640 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682adb5b0 0x562682add9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f518ffff640 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682adb5b0 0x562682add9d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7f56f79255271 server_cookie=9d7e7068ba14a672 in_seq=0 out_seq=0 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.257+0000 7f516f7fe640 10 client.17202 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f516f7fe640 1 -- 172.21.15.8:0/2349412644 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x562682758890 con 0x562682adb5b0 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f516f7fe640 10 client.17202 handle_command_reply: tid=0 2024-02-19T16:29:45.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 client.17202 shutdown 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 msgr2=0x562682a71e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a71e70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5188004ee0 tx=0x7f5188021d80 comp rx=0 tx=0).stop 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 shutdown_connections 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562682a6e7b0 0x562682a71e70 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562682a65bd0 0x562682a71930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x562682adb5b0 0x562682add9d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 --2- 172.21.15.8:0/2349412644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562682a652b0 0x562682a713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f516f7fe640 0 client.17202 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:29:45.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 >> 172.21.15.8:0/2349412644 conn(0x562682a5f990 msgr2=0x562682a5e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 shutdown_connections 2024-02-19T16:29:45.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.258+0000 7f519946f740 1 -- 172.21.15.8:0/2349412644 wait complete. 2024-02-19T16:29:45.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f519946f740 20 client.17202 trim_cache size 0 max 16384 2024-02-19T16:29:45.269 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2255 inodes in the stack)", 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: "bf42fe0c-1b5f-478c-91b5-e4ca560997cc": { 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "bf42fe0c-1b5f-478c-91b5-e4ca560997cc", 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:29:45.270 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:29:45.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f51780a22b0 msgr2=0x7f51780a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f51780a22b0 0x7f51780a4770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5188009fd0 tx=0x7f5188009ee0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 msgr2=0x7f519019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 0x7f519019d7d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5180009230 tx=0x7f518002c6c0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 shutdown_connections 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5190107500 0x7f5190197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f51780a22b0 0x7f51780a4770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5190101e20 0x7f519019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 --2- 172.21.15.8:0/1165435528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5190101460 0x7f519019d7d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:29:45.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 >> 172.21.15.8:0/1165435528 conn(0x7f51900fcf90 msgr2=0x7f51900fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:29:45.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 shutdown_connections 2024-02-19T16:29:45.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:29:45.259+0000 7f5196888640 1 -- 172.21.15.8:0/1165435528 wait complete. 2024-02-19T16:29:45.273 INFO:tasks.cephfs.filesystem:scrub status for tag:bf42fe0c-1b5f-478c-91b5-e4ca560997cc - {'path': '/', 'tag': 'bf42fe0c-1b5f-478c-91b5-e4ca560997cc', 'options': 'recursive,force'} 2024-02-19T16:29:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:45 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:45.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:45 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:45 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:46.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:45 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:45 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:45 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: pgmap v211: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:29:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:46.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:46 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: pgmap v211: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:47.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:29:47.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:46 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: pgmap v211: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:29:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:29:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:47.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:46 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:29:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:48 smithi156 ceph-mon[36502]: pgmap v212: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:48 smithi008 ceph-mon[32573]: pgmap v212: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:49.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:48 smithi191 ceph-mon[37203]: pgmap v212: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:29:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:50 smithi156 ceph-mon[36502]: pgmap v213: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:29:51.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:50 smithi008 ceph-mon[32573]: pgmap v213: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:29:51.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:50 smithi191 ceph-mon[37203]: pgmap v213: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:29:52.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:52 smithi156 ceph-mon[36502]: pgmap v214: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:53.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:52 smithi008 ceph-mon[32573]: pgmap v214: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:53.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:52 smithi191 ceph-mon[37203]: pgmap v214: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:54.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:54 smithi156 ceph-mon[36502]: pgmap v215: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:54.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:54 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:29:54.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:54 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:29:55.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:54 smithi008 ceph-mon[32573]: pgmap v215: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:55.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:54 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:29:55.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:54 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:29:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:54 smithi191 ceph-mon[37203]: pgmap v215: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:54 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:29:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:54 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:29:55.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:55 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:29:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:55 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:29:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:55 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:29:56.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:56 smithi156 ceph-mon[36502]: pgmap v216: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:29:57.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:56 smithi008 ceph-mon[32573]: pgmap v216: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:29:57.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:56 smithi191 ceph-mon[37203]: pgmap v216: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:29:57.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:57 smithi156 ceph-mon[36502]: pgmap v217: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:58.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:57 smithi008 ceph-mon[32573]: pgmap v217: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:29:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:57 smithi191 ceph-mon[37203]: pgmap v217: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:30:00.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:29:59 smithi008 ceph-mon[32573]: pgmap v218: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:30:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:29:59 smithi191 ceph-mon[37203]: pgmap v218: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:30:00.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:29:59 smithi156 ceph-mon[36502]: pgmap v218: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:30:01.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:00 smithi008 ceph-mon[32573]: overall HEALTH_OK 2024-02-19T16:30:01.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:00 smithi191 ceph-mon[37203]: overall HEALTH_OK 2024-02-19T16:30:01.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:00 smithi156 ceph-mon[36502]: overall HEALTH_OK 2024-02-19T16:30:02.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:01 smithi008 ceph-mon[32573]: pgmap v219: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:01 smithi191 ceph-mon[37203]: pgmap v219: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:02.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:01 smithi156 ceph-mon[36502]: pgmap v219: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:03 smithi008 ceph-mon[32573]: pgmap v220: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:30:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:30:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:30:04.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:03 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:30:04.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:03 smithi191 ceph-mon[37203]: pgmap v220: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:04.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:30:04.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:30:04.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:30:04.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:03 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:30:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:03 smithi156 ceph-mon[36502]: pgmap v220: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:30:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:30:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:30:04.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:03 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:30:06.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:05 smithi008 ceph-mon[32573]: pgmap v221: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:30:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:05 smithi191 ceph-mon[37203]: pgmap v221: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:30:06.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:05 smithi156 ceph-mon[36502]: pgmap v221: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:30:08.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:07 smithi008 ceph-mon[32573]: pgmap v222: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:30:08.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:07 smithi191 ceph-mon[37203]: pgmap v222: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:30:08.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:07 smithi156 ceph-mon[36502]: pgmap v222: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:30:10.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:09 smithi008 ceph-mon[32573]: pgmap v223: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:30:10.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:09 smithi191 ceph-mon[37203]: pgmap v223: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:30:10.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:09 smithi156 ceph-mon[36502]: pgmap v223: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:30:11.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:10 smithi008 ceph-mon[32573]: osdmap e99: 12 total, 12 up, 12 in 2024-02-19T16:30:11.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:10 smithi191 ceph-mon[37203]: osdmap e99: 12 total, 12 up, 12 in 2024-02-19T16:30:11.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:10 smithi156 ceph-mon[36502]: osdmap e99: 12 total, 12 up, 12 in 2024-02-19T16:30:12.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:11 smithi008 ceph-mon[32573]: pgmap v225: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:30:12.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:11 smithi191 ceph-mon[37203]: pgmap v225: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:30:12.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:11 smithi156 ceph-mon[36502]: pgmap v225: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:30:14.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:13 smithi008 ceph-mon[32573]: pgmap v226: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:30:14.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:13 smithi191 ceph-mon[37203]: pgmap v226: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:30:14.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:13 smithi156 ceph-mon[36502]: pgmap v226: 129 pgs: 129 active+clean; 79 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:30:15.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:30:15.276 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:30:15.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.393+0000 7fdc42547640 1 -- 172.21.15.8:0/2477495050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 msgr2=0x7fdc3c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.393+0000 7fdc42547640 1 --2- 172.21.15.8:0/2477495050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c101150 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009900 tx=0x7fdc2402bc30 comp rx=0 tx=0).stop 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 -- 172.21.15.8:0/2477495050 shutdown_connections 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 --2- 172.21.15.8:0/2477495050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc3c105750 0x7fdc3c10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 --2- 172.21.15.8:0/2477495050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3c101720 0x7fdc3c105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 --2- 172.21.15.8:0/2477495050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c101150 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 -- 172.21.15.8:0/2477495050 >> 172.21.15.8:0/2477495050 conn(0x7fdc3c076920 msgr2=0x7fdc3c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 -- 172.21.15.8:0/2477495050 shutdown_connections 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 -- 172.21.15.8:0/2477495050 wait complete. 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 Processor -- start 2024-02-19T16:30:15.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.394+0000 7fdc42547640 1 -- start start 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc42547640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc42547640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3c105750 0x7fdc3c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc42547640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc3c197960 0x7fdc3c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc42547640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c198350 con 0x7fdc3c100d70 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc42547640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1984c0 con 0x7fdc3c105750 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc42547640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c198680 con 0x7fdc3c197960 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55840/0 (socket says 172.21.15.8:55840) 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 -- 172.21.15.8:0/338808766 learned_addr learned my addr 172.21.15.8:0/338808766 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3c105750 0x7fdc3c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc3c197960 0x7fdc3c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 -- 172.21.15.8:0/338808766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc3c197960 msgr2=0x7fdc3c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc3c197960 0x7fdc3c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 -- 172.21.15.8:0/338808766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3c105750 msgr2=0x7fdc3c19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3c105750 0x7fdc3c19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 -- 172.21.15.8:0/338808766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009560 con 0x7fdc3c100d70 2024-02-19T16:30:15.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c19d7b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fdc24005940 tx=0x7fdc2402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.395+0000 7fdc397fa640 1 -- 172.21.15.8:0/338808766 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24004350 con 0x7fdc3c100d70 2024-02-19T16:30:15.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.396+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c1a4670 con 0x7fdc3c100d70 2024-02-19T16:30:15.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.396+0000 7fdc397fa640 1 -- 172.21.15.8:0/338808766 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc240044b0 con 0x7fdc3c100d70 2024-02-19T16:30:15.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.396+0000 7fdc397fa640 1 -- 172.21.15.8:0/338808766 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24003d30 con 0x7fdc3c100d70 2024-02-19T16:30:15.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.396+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c1a4b20 con 0x7fdc3c100d70 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc397fa640 1 -- 172.21.15.8:0/338808766 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fdc24004df0 con 0x7fdc3c100d70 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 Processor -- start 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 -- start start 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b600270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 0x55ad9b602ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b566280 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5637b0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.398+0000 7fdc451f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b55ef90 con 0x55ad9b6007b0 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc397fa640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdc280a2230 0x7fdc280a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b600270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 0x55ad9b602ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:49734/0 (socket says 172.21.15.8:49734) 2024-02-19T16:30:15.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b600270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55842/0 (socket says 172.21.15.8:55842) 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 0x55ad9b602ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:52664/0 (socket says 172.21.15.8:52664) 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 -- 172.21.15.8:0/973348849 learned_addr learned my addr 172.21.15.8:0/973348849 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdc280a2230 0x7fdc280a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc397fa640 1 -- 172.21.15.8:0/338808766 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fdc240f0490 con 0x7fdc3c100d70 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 -- 172.21.15.8:0/973348849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 msgr2=0x55ad9b602ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 0x55ad9b602ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 -- 172.21.15.8:0/973348849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 msgr2=0x55ad9b600270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b600270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 -- 172.21.15.8:0/973348849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9b51d570 con 0x55ad9b5f72b0 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 0x55ad9b602ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdc280a2230 0x7fdc280a46f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400fc90 tx=0x7fdc3401e000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5f7690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc34011da0 tx=0x7fdc3400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f097b67b2e600ae server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.399+0000 7fdc327fc640 1 -- 172.21.15.8:0/973348849 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3401dbf0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc327fc640 1 -- 172.21.15.8:0/973348849 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc34009dc0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc327fc640 1 -- 172.21.15.8:0/973348849 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340042b0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 -- 172.21.15.8:0/973348849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 msgr2=0x55ad9b5f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5f7690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc34011da0 tx=0x7fdc3400f790 comp rx=0 tx=0).stop 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 -- 172.21.15.8:0/973348849 shutdown_connections 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b6007b0 0x55ad9b602ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b600270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 --2- 172.21.15.8:0/973348849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5f7690 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 -- 172.21.15.8:0/973348849 >> 172.21.15.8:0/973348849 conn(0x55ad9b5f1990 msgr2=0x55ad9b5f1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 -- 172.21.15.8:0/973348849 shutdown_connections 2024-02-19T16:30:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 -- 172.21.15.8:0/973348849 wait complete. 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.400+0000 7fdc451f4740 1 -- start start 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 Processor -- start 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b600260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b232430 con 0x55ad9b5f72b0 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b1eee00 con 0x55ad9b6007b0 2024-02-19T16:30:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b4e3fd0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc451f4740 10 client.0 _ll_register_callbacks cb 0x55ad9b539fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b600260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b600260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55850/0 (socket says 172.21.15.8:55850) 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 -- 172.21.15.8:0/2315707294 learned_addr learned my addr 172.21.15.8:0/2315707294 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc40ae4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 -- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 msgr2=0x55ad9b5fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 -- 172.21.15.8:0/2315707294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 msgr2=0x55ad9b5fdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5fdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc3bfff640 1 -- 172.21.15.8:0/2315707294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc34019040 con 0x55ad9b5f72b0 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.401+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b600260 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdc240bc7d0 tx=0x7fdc240b8770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7f391082fbb62a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2315707294 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240bd460 con 0x55ad9b5f72b0 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2315707294 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fdc240035f0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad9b25e7e0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2315707294 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc240bd1d0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 10 client.17214 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ad9b55d610 con 0x55ad9b5f72b0 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2315707294 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ad9b25e7e0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2315707294 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc2403c070 con 0x55ad9b5f72b0 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 10 client.17214 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:15.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 20 client.17214 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b610530 0x55ad9b612920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 4 client.17214 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.402+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad9b29b720 con 0x55ad9b610530 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.403+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b610530 0x55ad9b612920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.403+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b610530 0x55ad9b612920 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb05af5cd8650b18 server_cookie=384685badad21614 in_seq=0 out_seq=0 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.403+0000 7fdc1b7fe640 10 client.17214 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:15.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2315707294 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ad9b29b720 con 0x55ad9b610530 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc1b7fe640 10 client.17214 handle_command_reply: tid=0 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 client.17214 shutdown 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 msgr2=0x55ad9b600260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b600260 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdc240bc7d0 tx=0x7fdc240b8770 comp rx=0 tx=0).stop 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 shutdown_connections 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b610530 0x55ad9b612920 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2315707294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b600260 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc1b7fe640 0 client.17214 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:15.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 >> 172.21.15.8:0/2315707294 conn(0x55ad9b5f1990 msgr2=0x55ad9b5f3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 shutdown_connections 2024-02-19T16:30:15.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 1 -- 172.21.15.8:0/2315707294 wait complete. 2024-02-19T16:30:15.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.404+0000 7fdc451f4740 20 client.17214 trim_cache size 0 max 16384 2024-02-19T16:30:15.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 Processor -- start 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 -- start start 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b600260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5f17f0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5ebe80 con 0x55ad9b6007b0 2024-02-19T16:30:15.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc451f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5f6f10 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.420+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55864/0 (socket says 172.21.15.8:55864) 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 -- 172.21.15.8:0/4133894586 learned_addr learned my addr 172.21.15.8:0/4133894586 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc40ae4640 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b600260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 -- 172.21.15.8:0/4133894586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 msgr2=0x55ad9b5ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 -- 172.21.15.8:0/4133894586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 msgr2=0x55ad9b600260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b600260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 -- 172.21.15.8:0/4133894586 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9b4f9ac0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fd590 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fdc240b9450 tx=0x7fdc240057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4328a647395220 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc31ffb640 1 -- 172.21.15.8:0/4133894586 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240035f0 con 0x55ad9b5f72b0 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc31ffb640 1 -- 172.21.15.8:0/4133894586 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc240d5070 con 0x55ad9b5f72b0 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.421+0000 7fdc31ffb640 1 -- 172.21.15.8:0/4133894586 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240c4e30 con 0x55ad9b5f72b0 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 -- 172.21.15.8:0/4133894586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 msgr2=0x55ad9b5fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fd590 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fdc240b9450 tx=0x7fdc240057e0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 -- 172.21.15.8:0/4133894586 shutdown_connections 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b600260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 --2- 172.21.15.8:0/4133894586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fd590 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 -- 172.21.15.8:0/4133894586 >> 172.21.15.8:0/4133894586 conn(0x55ad9b5f1990 msgr2=0x55ad9b5fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 -- 172.21.15.8:0/4133894586 shutdown_connections 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 -- 172.21.15.8:0/4133894586 wait complete. 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 -- start start 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 Processor -- start 2024-02-19T16:30:15.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.422+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5fa3d0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5fa540 con 0x55ad9b6007b0 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9b5fa700 con 0x55ad9b5f72b0 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 10 client.0 _ll_register_callbacks cb 0x7fdc24005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc40ae4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:52698/0 (socket says 172.21.15.8:52698) 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3bfff640 1 -- 172.21.15.8:0/2664498200 learned_addr learned my addr 172.21.15.8:0/2664498200 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 -- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f72b0 msgr2=0x55ad9b5fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 -- 172.21.15.8:0/2664498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 msgr2=0x55ad9b5f9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5f9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 -- 172.21.15.8:0/2664498200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc34019040 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc3b7fe640 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5f9880 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x55ad9b5fce50 tx=0x7fdc2c004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329b48c7172195bb server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2664498200 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c017040 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2664498200 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fdc2c020b00 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc34009dc0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2664498200 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc2c020cc0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 10 client.17220 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.423+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ad9b51f6a0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2664498200 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc34009dc0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2664498200 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc2c0107c0 con 0x55ad9b5f7bd0 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc451f4740 10 client.17220 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc451f4740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc451f4740 20 client.17220 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b6714e0 0x55ad9b673900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc451f4740 4 client.17220 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad9b2ea890 con 0x55ad9b6714e0 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b6714e0 0x55ad9b673900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc1b7fe640 10 client.17220 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.424+0000 7fdc3bfff640 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b6714e0 0x55ad9b673900 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de63cb4dd9e52327 server_cookie=198b5179acf7571f in_seq=0 out_seq=0 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc1b7fe640 1 -- 172.21.15.8:0/2664498200 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ad9b2ea890 con 0x55ad9b6714e0 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc1b7fe640 10 client.17220 handle_command_reply: tid=0 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 client.17220 shutdown 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 msgr2=0x55ad9b5f9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5f9880 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x55ad9b5fce50 tx=0x7fdc2c004120 comp rx=0 tx=0).stop 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 shutdown_connections 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad9b6007b0 0x55ad9b5f9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ad9b5f7bd0 0x55ad9b5f9880 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ad9b6714e0 0x55ad9b673900 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 --2- 172.21.15.8:0/2664498200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ad9b5f72b0 0x55ad9b5fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc1b7fe640 0 client.17220 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:15.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 >> 172.21.15.8:0/2664498200 conn(0x55ad9b5f1990 msgr2=0x55ad9b6079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 shutdown_connections 2024-02-19T16:30:15.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 1 -- 172.21.15.8:0/2664498200 wait complete. 2024-02-19T16:30:15.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc451f4740 20 client.17220 trim_cache size 0 max 16384 2024-02-19T16:30:15.437 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:30:15.437 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:30:15.437 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:30:15.437 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:30:15.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdc280a2230 msgr2=0x7fdc280a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc42547640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdc280a2230 0x7fdc280a46f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400fc90 tx=0x7fdc3401e000 comp rx=0 tx=0).stop 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 msgr2=0x7fdc3c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.425+0000 7fdc42547640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c19d7b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fdc24005940 tx=0x7fdc2402c670 comp rx=0 tx=0).stop 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 shutdown_connections 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fdc280a2230 0x7fdc280a46f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc3c197960 0x7fdc3c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3c105750 0x7fdc3c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 --2- 172.21.15.8:0/338808766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc3c100d70 0x7fdc3c19d7b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 >> 172.21.15.8:0/338808766 conn(0x7fdc3c076920 msgr2=0x7fdc3c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 shutdown_connections 2024-02-19T16:30:15.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.426+0000 7fdc42547640 1 -- 172.21.15.8:0/338808766 wait complete. 2024-02-19T16:30:15.440 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:30:15.440 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:30:15.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2771072699 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881056d0 msgr2=0x7ffa8810d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2771072699 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881056d0 0x7ffa8810d2a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009cd0 tx=0x7ffa7802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2771072699 shutdown_connections 2024-02-19T16:30:15.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2771072699 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881056d0 0x7ffa8810d2a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2771072699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa88100fb0 0x7ffa88104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2771072699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa88100600 0x7ffa881009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2771072699 >> 172.21.15.8:0/2771072699 conn(0x7ffa88076770 msgr2=0x7ffa88076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2771072699 shutdown_connections 2024-02-19T16:30:15.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.541+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2771072699 wait complete. 2024-02-19T16:30:15.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 Processor -- start 2024-02-19T16:30:15.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 -- start start 2024-02-19T16:30:15.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa88100600 0x7ffa88198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa881056d0 0x7ffa88198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 0x7ffa881a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa881a4040 con 0x7ffa881993f0 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa881a41b0 con 0x7ffa88100600 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa8d93e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa881a4370 con 0x7ffa881056d0 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa877fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 0x7ffa881a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa877fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 0x7ffa881a39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56428/0 (socket says 172.21.15.8:56428) 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa877fe640 1 -- 172.21.15.8:0/2619233189 learned_addr learned my addr 172.21.15.8:0/2619233189 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa86ffd640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa88100600 0x7ffa88198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.542+0000 7ffa867fc640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa881056d0 0x7ffa88198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa877fe640 1 -- 172.21.15.8:0/2619233189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa881056d0 msgr2=0x7ffa88198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa877fe640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa881056d0 0x7ffa88198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa877fe640 1 -- 172.21.15.8:0/2619233189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa88100600 msgr2=0x7ffa88198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa877fe640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa88100600 0x7ffa88198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa877fe640 1 -- 172.21.15.8:0/2619233189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa70009490 con 0x7ffa881993f0 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa877fe640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 0x7ffa881a39a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ffa78004af0 tx=0x7ffa78032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780328b0 con 0x7ffa881993f0 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa78003e50 con 0x7ffa881993f0 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78004740 con 0x7ffa881993f0 2024-02-19T16:30:15.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa78009900 con 0x7ffa881993f0 2024-02-19T16:30:15.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.543+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa881a4910 con 0x7ffa881993f0 2024-02-19T16:30:15.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.544+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa88101ad0 con 0x7ffa881993f0 2024-02-19T16:30:15.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.545+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ffa7800a040 con 0x7ffa881993f0 2024-02-19T16:30:15.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.545+0000 7ffa8c93c640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ffa740a2230 0x7ffa740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.546+0000 7ffa86ffd640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ffa740a2230 0x7ffa740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.546+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7ffa780f0fb0 con 0x7ffa881993f0 2024-02-19T16:30:15.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.546+0000 7ffa86ffd640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ffa740a2230 0x7ffa740a46f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009bd0 tx=0x7ffa80005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.550+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ffa780b98f0 con 0x7ffa881993f0 2024-02-19T16:30:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.774+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa88108900 con 0x7ffa881993f0 2024-02-19T16:30:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.774+0000 7ffa8c93c640 1 -- 172.21.15.8:0/2619233189 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ffa780bcfb0 con 0x7ffa881993f0 2024-02-19T16:30:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:30:15.777 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:30:15.777 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:30:15.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ffa740a2230 msgr2=0x7ffa740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ffa740a2230 0x7ffa740a46f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009bd0 tx=0x7ffa80005f90 comp rx=0 tx=0).stop 2024-02-19T16:30:15.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 msgr2=0x7ffa881a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 0x7ffa881a39a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ffa78004af0 tx=0x7ffa78032570 comp rx=0 tx=0).stop 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 shutdown_connections 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa881993f0 0x7ffa881a39a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ffa740a2230 0x7ffa740a46f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa881056d0 0x7ffa88198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 --2- 172.21.15.8:0/2619233189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa88100600 0x7ffa88198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.776+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 >> 172.21.15.8:0/2619233189 conn(0x7ffa88076770 msgr2=0x7ffa880fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.777+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 shutdown_connections 2024-02-19T16:30:15.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.777+0000 7ffa8d93e640 1 -- 172.21.15.8:0/2619233189 wait complete. 2024-02-19T16:30:15.789 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:30:15.789 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 -- 172.21.15.8:0/755033747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc224101e60 msgr2=0x7fc22410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 --2- 172.21.15.8:0/755033747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc224101e60 0x7fc22410e320 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009ca0 tx=0x7fc21c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 -- 172.21.15.8:0/755033747 shutdown_connections 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 --2- 172.21.15.8:0/755033747 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc224101e60 0x7fc22410e320 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 --2- 172.21.15.8:0/755033747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2241014a0 0x7fc224101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 --2- 172.21.15.8:0/755033747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc224107560 0x7fc224107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 -- 172.21.15.8:0/755033747 >> 172.21.15.8:0/755033747 conn(0x7fc2240788f0 msgr2=0x7fc224078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 -- 172.21.15.8:0/755033747 shutdown_connections 2024-02-19T16:30:15.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 -- 172.21.15.8:0/755033747 wait complete. 2024-02-19T16:30:15.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 Processor -- start 2024-02-19T16:30:15.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.892+0000 7fc22a56c640 1 -- start start 2024-02-19T16:30:15.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 0x7fc2240737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc224101e60 0x7fc224073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc224107560 0x7fc224074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2241a5e10 con 0x7fc2241014a0 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2241a5f80 con 0x7fc224101e60 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2241a6140 con 0x7fc224107560 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 0x7fc2240737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 0x7fc2240737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56446/0 (socket says 172.21.15.8:56446) 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 -- 172.21.15.8:0/4063106913 learned_addr learned my addr 172.21.15.8:0/4063106913 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc228b09640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc224107560 0x7fc224074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc2237fe640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc224101e60 0x7fc224073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 -- 172.21.15.8:0/4063106913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc224107560 msgr2=0x7fc224074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc224107560 0x7fc224074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 -- 172.21.15.8:0/4063106913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc224101e60 msgr2=0x7fc224073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc224101e60 0x7fc224073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 -- 172.21.15.8:0/4063106913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c009900 con 0x7fc2241014a0 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc223fff640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 0x7fc2240737f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc21800e9f0 tx=0x7fc21800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc2217fa640 1 -- 172.21.15.8:0/4063106913 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218009720 con 0x7fc2241014a0 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2241a6420 con 0x7fc2241014a0 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc2217fa640 1 -- 172.21.15.8:0/4063106913 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc21800cd80 con 0x7fc2241014a0 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.893+0000 7fc2217fa640 1 -- 172.21.15.8:0/4063106913 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218010690 con 0x7fc2241014a0 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.894+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2241a68e0 con 0x7fc2241014a0 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc2217fa640 1 -- 172.21.15.8:0/4063106913 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fc2180046c0 con 0x7fc2241014a0 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 Processor -- start 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 -- start start 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc2217fa640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc20c0a2230 0x7fc20c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e1d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc228b09640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc20c0a2230 0x7fc20c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e26210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc2217fa640 1 -- 172.21.15.8:0/4063106913 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fc218014070 con 0x7fc2241014a0 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e28b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287dd8960 con 0x55e287e1d250 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287d8c310 con 0x55e287e1db70 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc22d223740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287d89840 con 0x55e287e26750 2024-02-19T16:30:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e1d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e1d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56462/0 (socket says 172.21.15.8:56462) 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc223fff640 1 -- 172.21.15.8:0/3602514268 learned_addr learned my addr 172.21.15.8:0/3602514268 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.896+0000 7fc2237fe640 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e26210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc228b09640 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e28b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc223fff640 1 -- 172.21.15.8:0/3602514268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 msgr2=0x55e287e28b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc223fff640 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e28b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc223fff640 1 -- 172.21.15.8:0/3602514268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 msgr2=0x55e287e26210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc223fff640 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e26210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc223fff640 1 -- 172.21.15.8:0/3602514268 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e287d47060 con 0x55e287e1d250 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc228b09640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc20c0a2230 0x7fc20c0a46f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc21c004af0 tx=0x7fc21c032340 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc223fff640 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e1d630 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc2180a3030 tx=0x7fc21809b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d697695783c97b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc20a7fc640 1 -- 172.21.15.8:0/3602514268 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21809ef10 con 0x55e287e1d250 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc20a7fc640 1 -- 172.21.15.8:0/3602514268 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2180a3560 con 0x55e287e1d250 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.897+0000 7fc20a7fc640 1 -- 172.21.15.8:0/3602514268 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180ce380 con 0x55e287e1d250 2024-02-19T16:30:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- 172.21.15.8:0/3602514268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 msgr2=0x55e287e1d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e1d630 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc2180a3030 tx=0x7fc21809b4c0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- 172.21.15.8:0/3602514268 shutdown_connections 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e28b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e26210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- 172.21.15.8:0/3602514268 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e1d630 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- 172.21.15.8:0/3602514268 >> 172.21.15.8:0/3602514268 conn(0x55e287e17980 msgr2=0x55e287e17d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- 172.21.15.8:0/3602514268 shutdown_connections 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- 172.21.15.8:0/3602514268 wait complete. 2024-02-19T16:30:15.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- start start 2024-02-19T16:30:15.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 Processor -- start 2024-02-19T16:30:15.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e23820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e23d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e26750 0x55e287e242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287d85020 con 0x55e287e1d250 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287a58430 con 0x55e287e26750 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.898+0000 7fc22d223740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287a14e00 con 0x55e287e1db70 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc22d223740 10 client.0 _ll_register_callbacks cb 0x55e287d2ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e23820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e23820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56470/0 (socket says 172.21.15.8:56470) 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 -- 172.21.15.8:0/1874078355 learned_addr learned my addr 172.21.15.8:0/1874078355 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc228b09640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e26750 0x55e287e242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2237fe640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e23d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 -- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 msgr2=0x55e287e23d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e23d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 -- 172.21.15.8:0/1874078355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e26750 msgr2=0x55e287e242a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e26750 0x55e287e242a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 -- 172.21.15.8:0/1874078355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2241a68e0 con 0x55e287e1d250 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2237fe640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc223fff640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e23820 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc2180cc120 tx=0x7fc218004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb83e149306a8948 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2037fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2037fe640 1 -- 172.21.15.8:0/1874078355 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180990c0 con 0x55e287e1d250 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2037fe640 1 -- 172.21.15.8:0/1874078355 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fc2180a31b0 con 0x55e287e1d250 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e287a847e0 con 0x55e287e1d250 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2037fe640 1 -- 172.21.15.8:0/1874078355 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21809a210 con 0x55e287e1d250 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc22d223740 10 client.17238 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e287d836a0 con 0x55e287e1d250 2024-02-19T16:30:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.899+0000 7fc2037fe640 1 -- 172.21.15.8:0/1874078355 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e287a847e0 con 0x55e287e1d250 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc2037fe640 1 -- 172.21.15.8:0/1874078355 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc21801f060 con 0x55e287e1d250 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc22d223740 10 client.17238 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc22d223740 10 client.17238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc22d223740 20 client.17238 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc22d223740 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e36940 0x55e287e38d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc22d223740 4 client.17238 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e287ac1720 con 0x55e287e36940 2024-02-19T16:30:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc2237fe640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e36940 0x55e287e38d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc2037fe640 10 client.17238 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.900+0000 7fc2237fe640 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e36940 0x55e287e38d60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b0894d794a7a6f3 server_cookie=2d20bd63cc619c7f in_seq=0 out_seq=0 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc2037fe640 1 -- 172.21.15.8:0/1874078355 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e287ac1720 con 0x55e287e36940 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc2037fe640 10 client.17238 handle_command_reply: tid=0 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 client.17238 shutdown 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 msgr2=0x55e287e23820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e23820 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc2180cc120 tx=0x7fc218004b30 comp rx=0 tx=0).stop 2024-02-19T16:30:15.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 shutdown_connections 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e36940 0x55e287e38d60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e26750 0x55e287e242a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc2037fe640 0 client.17238 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 --2- 172.21.15.8:0/1874078355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e23820 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 >> 172.21.15.8:0/1874078355 conn(0x55e287e17980 msgr2=0x55e287e196d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 shutdown_connections 2024-02-19T16:30:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 1 -- 172.21.15.8:0/1874078355 wait complete. 2024-02-19T16:30:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.901+0000 7fc22d223740 20 client.17238 trim_cache size 0 max 16384 2024-02-19T16:30:15.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 Processor -- start 2024-02-19T16:30:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 -- start start 2024-02-19T16:30:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e31ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e32010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287d0a130 con 0x55e287e1d250 2024-02-19T16:30:15.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287e171f0 con 0x55e287e1db70 2024-02-19T16:30:15.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc22d223740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287e393e0 con 0x55e287e26750 2024-02-19T16:30:15.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56472/0 (socket says 172.21.15.8:56472) 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 -- 172.21.15.8:0/3109695825 learned_addr learned my addr 172.21.15.8:0/3109695825 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc228b09640 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e32010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc2237fe640 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e31ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 -- 172.21.15.8:0/3109695825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 msgr2=0x55e287e32010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e32010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 -- 172.21.15.8:0/3109695825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 msgr2=0x55e287e31ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e31ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 -- 172.21.15.8:0/3109695825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e287d1fc20 con 0x55e287e1d250 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc228b09640 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e32010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc223fff640 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e240c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc21809aac0 tx=0x7fc2180cbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c3e781de1a34ac server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc209ffb640 1 -- 172.21.15.8:0/3109695825 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180b3af0 con 0x55e287e1d250 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc209ffb640 1 -- 172.21.15.8:0/3109695825 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc21808c070 con 0x55e287e1d250 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.914+0000 7fc209ffb640 1 -- 172.21.15.8:0/3109695825 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180a3140 con 0x55e287e1d250 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 -- 172.21.15.8:0/3109695825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 msgr2=0x55e287e240c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e240c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc21809aac0 tx=0x7fc2180cbbd0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 -- 172.21.15.8:0/3109695825 shutdown_connections 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e26750 0x55e287e32010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1db70 0x55e287e31ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 --2- 172.21.15.8:0/3109695825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e1d250 0x55e287e240c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 -- 172.21.15.8:0/3109695825 >> 172.21.15.8:0/3109695825 conn(0x55e287e17980 msgr2=0x55e287e3d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 -- 172.21.15.8:0/3109695825 shutdown_connections 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 -- 172.21.15.8:0/3109695825 wait complete. 2024-02-19T16:30:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.915+0000 7fc22d223740 1 -- start start 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 Processor -- start 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1d250 0x55e287e2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 0x55e287e91230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287e2acc0 con 0x55e287e26750 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287e91770 con 0x55e287e1d250 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e287e918e0 con 0x55e287e1db70 2024-02-19T16:30:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc22d223740 10 client.0 _ll_register_callbacks cb 0x7fc2140097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 0x55e287e91230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 0x55e287e91230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56488/0 (socket says 172.21.15.8:56488) 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 -- 172.21.15.8:0/3750231187 learned_addr learned my addr 172.21.15.8:0/3750231187 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc223fff640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1d250 0x55e287e2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc2237fe640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 -- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 msgr2=0x55e287e2a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e2a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 -- 172.21.15.8:0/3750231187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1d250 msgr2=0x55e287e2a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1d250 0x55e287e2a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc228b09640 1 -- 172.21.15.8:0/3750231187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22410ec20 con 0x55e287e26750 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.916+0000 7fc223fff640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1d250 0x55e287e2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc228b09640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 0x55e287e91230 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009ca0 tx=0x7fc21c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d15552f72cde85 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc2037fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc2037fe640 1 -- 172.21.15.8:0/3750231187 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c032ab0 con 0x55e287e26750 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc2037fe640 1 -- 172.21.15.8:0/3750231187 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fc21c047040 con 0x55e287e26750 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2180a3560 con 0x55e287e26750 2024-02-19T16:30:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc2037fe640 1 -- 172.21.15.8:0/3750231187 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21c003970 con 0x55e287e26750 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 10 client.17244 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e287d836a0 con 0x55e287e26750 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc2037fe640 1 -- 172.21.15.8:0/3750231187 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2180a3560 con 0x55e287e26750 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc2037fe640 1 -- 172.21.15.8:0/3750231187 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc21c04e420 con 0x55e287e26750 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 10 client.17244 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 20 client.17244 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e930b0 0x55e287e954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 4 client.17244 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e287b10890 con 0x55e287e930b0 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.917+0000 7fc223fff640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e930b0 0x55e287e954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc2037fe640 10 client.17244 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc223fff640 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e930b0 0x55e287e954d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a987f90795bf8dc1 server_cookie=654cb66509841859 in_seq=0 out_seq=0 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc2037fe640 1 -- 172.21.15.8:0/3750231187 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e287b10890 con 0x55e287e930b0 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc2037fe640 10 client.17244 handle_command_reply: tid=0 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 client.17244 shutdown 2024-02-19T16:30:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 msgr2=0x55e287e91230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 0x55e287e91230 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009ca0 tx=0x7fc21c02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 shutdown_connections 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e287e930b0 0x55e287e954d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e287e26750 0x55e287e91230 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e287e1db70 0x55e287e2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc2037fe640 0 client.17244 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 --2- 172.21.15.8:0/3750231187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e287e1d250 0x55e287e2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 >> 172.21.15.8:0/3750231187 conn(0x55e287e17980 msgr2=0x55e287e3d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 shutdown_connections 2024-02-19T16:30:15.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 1 -- 172.21.15.8:0/3750231187 wait complete. 2024-02-19T16:30:15.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.918+0000 7fc22d223740 20 client.17244 trim_cache size 0 max 16384 2024-02-19T16:30:15.939 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:30:15.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc20c0a2230 msgr2=0x7fc20c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc20c0a2230 0x7fc20c0a46f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc21c004af0 tx=0x7fc21c032340 comp rx=0 tx=0).stop 2024-02-19T16:30:15.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 msgr2=0x7fc2240737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:15.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 0x7fc2240737f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc21800e9f0 tx=0x7fc21800edf0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 shutdown_connections 2024-02-19T16:30:15.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc20c0a2230 0x7fc20c0a46f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc224107560 0x7fc224074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc224101e60 0x7fc224073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 --2- 172.21.15.8:0/4063106913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc2241014a0 0x7fc2240737f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:15.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 >> 172.21.15.8:0/4063106913 conn(0x7fc2240788f0 msgr2=0x7fc224100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:15.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 shutdown_connections 2024-02-19T16:30:15.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:15.919+0000 7fc22a56c640 1 -- 172.21.15.8:0/4063106913 wait complete. 2024-02-19T16:30:15.945 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:30:15.945 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:30:16.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:15 smithi008 ceph-mon[32573]: pgmap v227: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:16.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:15 smithi191 ceph-mon[37203]: pgmap v227: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:16.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.049+0000 7ff570483640 1 -- 172.21.15.8:0/967765027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 msgr2=0x7ff56810d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.049+0000 7ff570483640 1 --2- 172.21.15.8:0/967765027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56810d1d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff560009be0 tx=0x7ff56002bef0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 -- 172.21.15.8:0/967765027 shutdown_connections 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 --2- 172.21.15.8:0/967765027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56810d1d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 --2- 172.21.15.8:0/967765027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5681006d0 0x7ff568100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 --2- 172.21.15.8:0/967765027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5681064b0 0x7ff568106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 -- 172.21.15.8:0/967765027 >> 172.21.15.8:0/967765027 conn(0x7ff5680773d0 msgr2=0x7ff5680777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 -- 172.21.15.8:0/967765027 shutdown_connections 2024-02-19T16:30:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 -- 172.21.15.8:0/967765027 wait complete. 2024-02-19T16:30:16.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.050+0000 7ff570483640 1 Processor -- start 2024-02-19T16:30:16.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 -- start start 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5681006d0 0x7ff56819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5681064b0 0x7ff568196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff568196f00 con 0x7ff568101090 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff568197070 con 0x7ff5681064b0 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff570483640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff568197230 con 0x7ff5681006d0 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56502/0 (socket says 172.21.15.8:56502) 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 -- 172.21.15.8:0/1125731090 learned_addr learned my addr 172.21.15.8:0/1125731090 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56e21f640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5681006d0 0x7ff56819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56ea20640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5681064b0 0x7ff568196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 -- 172.21.15.8:0/1125731090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5681006d0 msgr2=0x7ff56819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5681006d0 0x7ff56819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 -- 172.21.15.8:0/1125731090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5681064b0 msgr2=0x7ff568196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5681064b0 0x7ff568196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 -- 172.21.15.8:0/1125731090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff564009490 con 0x7ff568101090 2024-02-19T16:30:16.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.051+0000 7ff56da1e640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56819cbf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e9f0 tx=0x7ff55800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.052+0000 7ff55f7fe640 1 -- 172.21.15.8:0/1125731090 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55800ccc0 con 0x7ff568101090 2024-02-19T16:30:16.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.052+0000 7ff55f7fe640 1 -- 172.21.15.8:0/1125731090 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff55800ce20 con 0x7ff568101090 2024-02-19T16:30:16.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.052+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff560009840 con 0x7ff568101090 2024-02-19T16:30:16.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.052+0000 7ff55f7fe640 1 -- 172.21.15.8:0/1125731090 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558010690 con 0x7ff568101090 2024-02-19T16:30:16.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.052+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5681a3630 con 0x7ff568101090 2024-02-19T16:30:16.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.054+0000 7ff55f7fe640 1 -- 172.21.15.8:0/1125731090 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff5580046c0 con 0x7ff568101090 2024-02-19T16:30:16.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.054+0000 7ff573057740 1 Processor -- start 2024-02-19T16:30:16.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.054+0000 7ff573057740 1 -- start start 2024-02-19T16:30:16.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff573057740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aed7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff573057740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 0x558b2aee0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff573057740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2aee2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff573057740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ae92960 con 0x558b2aed7b70 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff573057740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ae46310 con 0x558b2aee0750 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff573057740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ae43840 con 0x558b2aed7250 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff55f7fe640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5540a2230 0x7ff5540a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 0x558b2aee0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56ea20640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aed7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 0x558b2aee0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56506/0 (socket says 172.21.15.8:56506) 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 -- 172.21.15.8:0/2854065496 learned_addr learned my addr 172.21.15.8:0/2854065496 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56da1e640 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2aee2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56ea20640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5540a2230 0x7ff5540a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff55f7fe640 1 -- 172.21.15.8:0/1125731090 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7ff558014070 con 0x7ff568101090 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 -- 172.21.15.8:0/2854065496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 msgr2=0x558b2aed7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aed7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 -- 172.21.15.8:0/2854065496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 msgr2=0x558b2aee2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2aee2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 -- 172.21.15.8:0/2854065496 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b2ae01060 con 0x558b2aed7b70 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56ea20640 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aed7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.055+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 0x558b2aee0210 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff564009e80 tx=0x7ff564011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d82522fd7bb03db server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff56ea20640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5540a2230 0x7ff5540a46f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff560005f00 tx=0x7ff560009280 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff55cbfe640 1 -- 172.21.15.8:0/2854065496 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564019630 con 0x558b2aed7b70 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff55cbfe640 1 -- 172.21.15.8:0/2854065496 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff564019c70 con 0x558b2aed7b70 2024-02-19T16:30:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff55cbfe640 1 -- 172.21.15.8:0/2854065496 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400ec40 con 0x558b2aed7b70 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 -- 172.21.15.8:0/2854065496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 msgr2=0x558b2aee0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 0x558b2aee0210 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff564009e80 tx=0x7ff564011810 comp rx=0 tx=0).stop 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 -- 172.21.15.8:0/2854065496 shutdown_connections 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2aee2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7b70 0x558b2aee0210 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 --2- 172.21.15.8:0/2854065496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aed7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.056+0000 7ff573057740 1 -- 172.21.15.8:0/2854065496 >> 172.21.15.8:0/2854065496 conn(0x558b2aed1980 msgr2=0x558b2aed1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 -- 172.21.15.8:0/2854065496 shutdown_connections 2024-02-19T16:30:16.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 -- 172.21.15.8:0/2854065496 wait complete. 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 -- start start 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 Processor -- start 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7250 0x558b2aedd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aede1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ae3f020 con 0x558b2aee0750 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ab12430 con 0x558b2aed7250 2024-02-19T16:30:16.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2aacee00 con 0x558b2aed7b70 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff573057740 10 client.0 _ll_register_callbacks cb 0x558b2ade6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff56ea20640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aede1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff56ea20640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aede1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56512/0 (socket says 172.21.15.8:56512) 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff56ea20640 1 -- 172.21.15.8:0/2078639282 learned_addr learned my addr 172.21.15.8:0/2078639282 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7250 0x558b2aedd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.057+0000 7ff56da1e640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56e21f640 1 -- 172.21.15.8:0/2078639282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 msgr2=0x558b2aeddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56e21f640 1 -- 172.21.15.8:0/2078639282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 msgr2=0x558b2aede1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aede1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56e21f640 1 -- 172.21.15.8:0/2078639282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff564009ae0 con 0x558b2aed7250 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56ea20640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aede1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7250 0x558b2aedd770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5640097b0 tx=0x7ff56400e9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aeae222f99fb191 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff551ffb640 1 -- 172.21.15.8:0/2078639282 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564005240 con 0x558b2aed7250 2024-02-19T16:30:16.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b2ab3e7e0 con 0x558b2aed7250 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff551ffb640 1 -- 172.21.15.8:0/2078639282 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7ff56400ee80 con 0x558b2aed7250 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff551ffb640 1 -- 172.21.15.8:0/2078639282 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5640053a0 con 0x558b2aed7250 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff573057740 10 client.27011 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558b2ae3d6a0 con 0x558b2aed7250 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.058+0000 7ff551ffb640 1 -- 172.21.15.8:0/2078639282 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b2ab3e7e0 con 0x558b2aed7250 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff551ffb640 1 -- 172.21.15.8:0/2078639282 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff564029020 con 0x558b2aed7250 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff573057740 10 client.27011 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff573057740 10 client.27011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:30:16.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff573057740 20 client.27011 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff573057740 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2aef0a80 0x558b2aef2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff573057740 4 client.27011 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b2ab7b720 con 0x558b2aef0a80 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff56da1e640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2aef0a80 0x558b2aef2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff551ffb640 10 client.27011 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.059+0000 7ff56da1e640 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2aef0a80 0x558b2aef2e70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec6bb171f2f26599 server_cookie=934434b8f8b44b14 in_seq=0 out_seq=0 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff551ffb640 1 -- 172.21.15.8:0/2078639282 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558b2ab7b720 con 0x558b2aef0a80 2024-02-19T16:30:16.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff551ffb640 10 client.27011 handle_command_reply: tid=0 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 client.27011 shutdown 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7250 msgr2=0x558b2aedd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7250 0x558b2aedd770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5640097b0 tx=0x7ff56400e9c0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 shutdown_connections 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2aef0a80 0x558b2aef2e70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aede1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff551ffb640 0 client.27011 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:16.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 --2- 172.21.15.8:0/2078639282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7250 0x558b2aedd770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 >> 172.21.15.8:0/2078639282 conn(0x558b2aed1980 msgr2=0x558b2aed36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 shutdown_connections 2024-02-19T16:30:16.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 1 -- 172.21.15.8:0/2078639282 wait complete. 2024-02-19T16:30:16.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.060+0000 7ff573057740 20 client.27011 trim_cache size 0 max 16384 2024-02-19T16:30:16.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.072+0000 7ff573057740 1 Processor -- start 2024-02-19T16:30:16.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.072+0000 7ff573057740 1 -- start start 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff573057740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aedfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff573057740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7b70 0x558b2aede010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff573057740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aed32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff573057740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2adc4130 con 0x558b2aee0750 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff573057740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2aed6650 con 0x558b2aed7b70 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff573057740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2aef0880 con 0x558b2aed7250 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56ea20640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aed32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aedfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56ea20640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aed32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56524/0 (socket says 172.21.15.8:56524) 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56ea20640 1 -- 172.21.15.8:0/2391492075 learned_addr learned my addr 172.21.15.8:0/2391492075 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56da1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7b70 0x558b2aede010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 -- 172.21.15.8:0/2391492075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7b70 msgr2=0x558b2aede010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7b70 0x558b2aede010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 -- 172.21.15.8:0/2391492075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 msgr2=0x558b2aed32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aed32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 -- 172.21.15.8:0/2391492075 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b2add9c20 con 0x558b2aed7250 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56ea20640 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aed32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff56e21f640 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aedfea0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff564009e80 tx=0x7ff564004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e82451edb43ef3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.073+0000 7ff553fff640 1 -- 172.21.15.8:0/2391492075 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564004be0 con 0x558b2aed7250 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff553fff640 1 -- 172.21.15.8:0/2391492075 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff56400ea30 con 0x558b2aed7250 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff553fff640 1 -- 172.21.15.8:0/2391492075 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5640197d0 con 0x558b2aed7250 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 -- 172.21.15.8:0/2391492075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 msgr2=0x558b2aedfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aedfea0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff564009e80 tx=0x7ff564004db0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 -- 172.21.15.8:0/2391492075 shutdown_connections 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aee0750 0x558b2aed32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aed7b70 0x558b2aede010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 --2- 172.21.15.8:0/2391492075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7250 0x558b2aedfea0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 -- 172.21.15.8:0/2391492075 >> 172.21.15.8:0/2391492075 conn(0x558b2aed1980 msgr2=0x558b2aed2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 -- 172.21.15.8:0/2391492075 shutdown_connections 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.074+0000 7ff573057740 1 -- 172.21.15.8:0/2391492075 wait complete. 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 -- start start 2024-02-19T16:30:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 Processor -- start 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 0x558b2aed9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeda0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2af472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2af47800 con 0x558b2aed7250 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2af47940 con 0x558b2aee0750 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2af47b00 con 0x558b2aed7b70 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff573057740 10 client.0 _ll_register_callbacks cb 0x7ff56401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:16.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 0x558b2aed9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 0x558b2aed9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56532/0 (socket says 172.21.15.8:56532) 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 -- 172.21.15.8:0/694950701 learned_addr learned my addr 172.21.15.8:0/694950701 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56da1e640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeda0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56ea20640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2af472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 -- 172.21.15.8:0/694950701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 msgr2=0x558b2aeda0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeda0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 -- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 msgr2=0x558b2af472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2af472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 -- 172.21.15.8:0/694950701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff560030070 con 0x558b2aed7250 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff56e21f640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 0x558b2aed9bb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff564014040 tx=0x7ff564004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6bacd8fc94c78b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.075+0000 7ff551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff551ffb640 1 -- 172.21.15.8:0/694950701 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564004be0 con 0x558b2aed7250 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff551ffb640 1 -- 172.21.15.8:0/694950701 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7ff564000e40 con 0x558b2aed7250 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b2abca890 con 0x558b2aed7250 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff551ffb640 1 -- 172.21.15.8:0/694950701 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff56400e960 con 0x558b2aed7250 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 10 client.17262 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558b2ae3d6a0 con 0x558b2aed7250 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff551ffb640 1 -- 172.21.15.8:0/694950701 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b2abca890 con 0x558b2aed7250 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff551ffb640 1 -- 172.21.15.8:0/694950701 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff564029070 con 0x558b2aed7250 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 10 client.17262 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 10 client.17262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 20 client.17262 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2af4eef0 0x558b2af512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 4 client.17262 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:30:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b2aaea700 con 0x558b2af4eef0 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.076+0000 7ff56da1e640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2af4eef0 0x558b2af512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff551ffb640 10 client.17262 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff56da1e640 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2af4eef0 0x558b2af512e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bc0493b4b10b6f2 server_cookie=3a4553b5d1e113ae in_seq=0 out_seq=0 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff551ffb640 1 -- 172.21.15.8:0/694950701 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b2aaea700 con 0x558b2af4eef0 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff551ffb640 10 client.17262 handle_command_reply: tid=0 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 client.17262 shutdown 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 msgr2=0x558b2aed9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 0x558b2aed9bb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff564014040 tx=0x7ff564004a70 comp rx=0 tx=0).stop 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 shutdown_connections 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x558b2af4eef0 0x558b2af512e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b2aee0750 0x558b2af472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b2aed7b70 0x558b2aeda0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff551ffb640 0 client.17262 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 --2- 172.21.15.8:0/694950701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b2aed7250 0x558b2aed9bb0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 >> 172.21.15.8:0/694950701 conn(0x558b2aed1980 msgr2=0x558b2aee8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 shutdown_connections 2024-02-19T16:30:16.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 1 -- 172.21.15.8:0/694950701 wait complete. 2024-02-19T16:30:16.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.077+0000 7ff573057740 20 client.17262 trim_cache size 0 max 16384 2024-02-19T16:30:16.090 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:30:16.090 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:15 smithi156 ceph-mon[36502]: pgmap v227: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:16.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5540a2230 msgr2=0x7ff5540a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5540a2230 0x7ff5540a46f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff560005f00 tx=0x7ff560009280 comp rx=0 tx=0).stop 2024-02-19T16:30:16.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 msgr2=0x7ff56819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56819cbf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e9f0 tx=0x7ff55800eeb0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 shutdown_connections 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5681064b0 0x7ff568196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff568101090 0x7ff56819cbf0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5540a2230 0x7ff5540a46f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 --2- 172.21.15.8:0/1125731090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5681006d0 0x7ff56819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 >> 172.21.15.8:0/1125731090 conn(0x7ff5680773d0 msgr2=0x7ff5680ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 shutdown_connections 2024-02-19T16:30:16.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.078+0000 7ff570483640 1 -- 172.21.15.8:0/1125731090 wait complete. 2024-02-19T16:30:16.094 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:30:16.094 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:30:16.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 -- 172.21.15.8:0/1050110223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20100fa0 msgr2=0x7fad20104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 --2- 172.21.15.8:0/1050110223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20100fa0 0x7fad20104fe0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fad14009930 tx=0x7fad1402bc30 comp rx=0 tx=0).stop 2024-02-19T16:30:16.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 -- 172.21.15.8:0/1050110223 shutdown_connections 2024-02-19T16:30:16.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 --2- 172.21.15.8:0/1050110223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201056c0 0x7fad2010d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 --2- 172.21.15.8:0/1050110223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20100fa0 0x7fad20104fe0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 --2- 172.21.15.8:0/1050110223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad201005f0 0x7fad201009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 -- 172.21.15.8:0/1050110223 >> 172.21.15.8:0/1050110223 conn(0x7fad200774c0 msgr2=0x7fad200778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 -- 172.21.15.8:0/1050110223 shutdown_connections 2024-02-19T16:30:16.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.198+0000 7fad2592e640 1 -- 172.21.15.8:0/1050110223 wait complete. 2024-02-19T16:30:16.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 Processor -- start 2024-02-19T16:30:16.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 -- start start 2024-02-19T16:30:16.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 0x7fad2019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad201056c0 0x7fad2019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 0x7fad20197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20198350 con 0x7fad20197960 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad201984c0 con 0x7fad201005f0 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad2592e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20198680 con 0x7fad201056c0 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad1effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 0x7fad2019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.199+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 0x7fad20197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 0x7fad2019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:34168/0 (socket says 172.21.15.8:34168) 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 0x7fad20197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56548/0 (socket says 172.21.15.8:56548) 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 -- 172.21.15.8:0/3938828557 learned_addr learned my addr 172.21.15.8:0/3938828557 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad201056c0 0x7fad2019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 -- 172.21.15.8:0/3938828557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad201056c0 msgr2=0x7fad2019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad201056c0 0x7fad2019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 -- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 msgr2=0x7fad20197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 0x7fad20197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 -- 172.21.15.8:0/3938828557 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009560 con 0x7fad201005f0 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 0x7fad20197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad1effd640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 0x7fad2019d7b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad1000e9f0 tx=0x7fad1000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad2492c640 1 -- 172.21.15.8:0/3938828557 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10009720 con 0x7fad201005f0 2024-02-19T16:30:16.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad2492c640 1 -- 172.21.15.8:0/3938828557 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad1000cd80 con 0x7fad201005f0 2024-02-19T16:30:16.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad201a4670 con 0x7fad201005f0 2024-02-19T16:30:16.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad2492c640 1 -- 172.21.15.8:0/3938828557 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10010690 con 0x7fad201005f0 2024-02-19T16:30:16.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.200+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201a4b00 con 0x7fad201005f0 2024-02-19T16:30:16.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.202+0000 7fad2492c640 1 -- 172.21.15.8:0/3938828557 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fad100046c0 con 0x7fad201005f0 2024-02-19T16:30:16.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 Processor -- start 2024-02-19T16:30:16.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 -- start start 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580939630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 0x564580942210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580944b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645808f4960 con 0x564580939b70 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645808a8310 con 0x564580942750 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad285e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645808a5840 con 0x564580939250 2024-02-19T16:30:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad2492c640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fad080a2230 0x7fad080a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad1effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 0x564580942210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad1effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 0x564580942210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56564/0 (socket says 172.21.15.8:56564) 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580944b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad1effd640 1 -- 172.21.15.8:0/3929224304 learned_addr learned my addr 172.21.15.8:0/3929224304 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fad080a2230 0x7fad080a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580939630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.203+0000 7fad2492c640 1 -- 172.21.15.8:0/3938828557 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fad10014070 con 0x7fad201005f0 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1e7fc640 1 -- 172.21.15.8:0/3929224304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 msgr2=0x564580939630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580939630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1e7fc640 1 -- 172.21.15.8:0/3929224304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 msgr2=0x564580942210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 0x564580942210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1e7fc640 1 -- 172.21.15.8:0/3929224304 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564580863060 con 0x564580942750 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1effd640 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 0x564580942210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fad080a2230 0x7fad080a46f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad18009fd0 tx=0x7fad18009f60 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580944b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad1402c170 tx=0x7fad1402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b326081c3f5ed6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad05ffb640 1 -- 172.21.15.8:0/3929224304 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14004800 con 0x564580942750 2024-02-19T16:30:16.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad05ffb640 1 -- 172.21.15.8:0/3929224304 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad14004960 con 0x564580942750 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.204+0000 7fad05ffb640 1 -- 172.21.15.8:0/3929224304 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14005050 con 0x564580942750 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 -- 172.21.15.8:0/3929224304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 msgr2=0x564580944b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580944b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad1402c170 tx=0x7fad1402c5d0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 -- 172.21.15.8:0/3929224304 shutdown_connections 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580944b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939b70 0x564580942210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 --2- 172.21.15.8:0/3929224304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580939630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 -- 172.21.15.8:0/3929224304 >> 172.21.15.8:0/3929224304 conn(0x564580933980 msgr2=0x564580933d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 -- 172.21.15.8:0/3929224304 shutdown_connections 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 -- 172.21.15.8:0/3929224304 wait complete. 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 -- start start 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.205+0000 7fad285e0740 1 Processor -- start 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939250 0x56458093f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458093fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645808a1020 con 0x564580942750 2024-02-19T16:30:16.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580574430 con 0x564580939250 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580530e00 con 0x564580939b70 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad285e0740 10 client.0 _ll_register_callbacks cb 0x564580848cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458093fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56566/0 (socket says 172.21.15.8:56566) 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939250 0x56458093f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458093fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:33000/0 (socket says 172.21.15.8:33000) 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 -- 172.21.15.8:0/1860569863 learned_addr learned my addr 172.21.15.8:0/1860569863 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 -- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 msgr2=0x56458093fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458093fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 -- 172.21.15.8:0/1860569863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939250 msgr2=0x56458093f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939250 0x56458093f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 -- 172.21.15.8:0/1860569863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad20069b00 con 0x564580942750 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1effd640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939250 0x56458093f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458093fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809401f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fad1801f900 tx=0x7fad18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582c494f26524086 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.206+0000 7facfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7facfeffd640 1 -- 172.21.15.8:0/1860569863 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800fe40 con 0x564580942750 2024-02-19T16:30:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7facfeffd640 1 -- 172.21.15.8:0/1860569863 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fad18004b20 con 0x564580942750 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645805a07e0 con 0x564580942750 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7facfeffd640 1 -- 172.21.15.8:0/1860569863 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad18004ce0 con 0x564580942750 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 10 client.17274 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56458089f6a0 con 0x564580942750 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7facfeffd640 1 -- 172.21.15.8:0/1860569863 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645805a07e0 con 0x564580942750 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7facfeffd640 1 -- 172.21.15.8:0/1860569863 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad18028030 con 0x564580942750 2024-02-19T16:30:16.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 10 client.17274 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 20 client.17274 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x564580954f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 4 client.17274 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645805dd720 con 0x564580952b40 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.207+0000 7fad1effd640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x564580954f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.208+0000 7fad1effd640 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x564580954f60 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cb6ba15ce332b0 server_cookie=83b5039b332dae72 in_seq=0 out_seq=0 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.208+0000 7facfeffd640 10 client.17274 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.208+0000 7facfeffd640 1 -- 172.21.15.8:0/1860569863 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5645805dd720 con 0x564580952b40 2024-02-19T16:30:16.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.208+0000 7facfeffd640 10 client.17274 handle_command_reply: tid=0 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.208+0000 7fad285e0740 1 client.17274 shutdown 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 msgr2=0x5645809401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809401f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fad1801f900 tx=0x7fad18004a40 comp rx=0 tx=0).stop 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 shutdown_connections 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809401f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x564580954f60 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458093fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7facfeffd640 0 client.17274 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:16.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 --2- 172.21.15.8:0/1860569863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939250 0x56458093f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 >> 172.21.15.8:0/1860569863 conn(0x564580933980 msgr2=0x5645809356d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 shutdown_connections 2024-02-19T16:30:16.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 1 -- 172.21.15.8:0/1860569863 wait complete. 2024-02-19T16:30:16.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.209+0000 7fad285e0740 20 client.17274 trim_cache size 0 max 16384 2024-02-19T16:30:16.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 Processor -- start 2024-02-19T16:30:16.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 -- start start 2024-02-19T16:30:16.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 0x564580941ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458092ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580940010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580826130 con 0x564580939250 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580934980 con 0x564580942750 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.221+0000 7fad285e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458093a5f0 con 0x564580939b70 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 0x564580941ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 0x564580941ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56572/0 (socket says 172.21.15.8:56572) 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 -- 172.21.15.8:0/472466427 learned_addr learned my addr 172.21.15.8:0/472466427 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580940010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458092ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 -- 172.21.15.8:0/472466427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 msgr2=0x56458092ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458092ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 -- 172.21.15.8:0/472466427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 msgr2=0x564580940010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580940010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 -- 172.21.15.8:0/472466427 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458083bc20 con 0x564580939250 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad1effd640 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 0x564580941ea0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fad10004ae0 tx=0x7fad100d0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e1c485dad4bf6d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad057fa640 1 -- 172.21.15.8:0/472466427 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100a35c0 con 0x564580939250 2024-02-19T16:30:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad057fa640 1 -- 172.21.15.8:0/472466427 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad10099120 con 0x564580939250 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.222+0000 7fad057fa640 1 -- 172.21.15.8:0/472466427 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100ce640 con 0x564580939250 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 -- 172.21.15.8:0/472466427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 msgr2=0x564580941ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 0x564580941ea0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fad10004ae0 tx=0x7fad100d0b40 comp rx=0 tx=0).stop 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 -- 172.21.15.8:0/472466427 shutdown_connections 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580942750 0x564580940010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939b70 0x56458092ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 --2- 172.21.15.8:0/472466427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580939250 0x564580941ea0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 -- 172.21.15.8:0/472466427 >> 172.21.15.8:0/472466427 conn(0x564580933980 msgr2=0x564580932b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 -- 172.21.15.8:0/472466427 shutdown_connections 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 -- 172.21.15.8:0/472466427 wait complete. 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 -- start start 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.223+0000 7fad285e0740 1 Processor -- start 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580946160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939b70 0x5645809466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645809ad950 con 0x564580942750 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645809ada90 con 0x564580939b70 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645809adbd0 con 0x564580939250 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 10 client.0 _ll_register_callbacks cb 0x7fad18009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56576/0 (socket says 172.21.15.8:56576) 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580946160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 -- 172.21.15.8:0/2670649166 learned_addr learned my addr 172.21.15.8:0/2670649166 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:16.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1e7fc640 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939b70 0x5645809466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 -- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 msgr2=0x564580946160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580946160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 -- 172.21.15.8:0/2670649166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939b70 msgr2=0x5645809466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939b70 0x5645809466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 -- 172.21.15.8:0/2670649166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad100aee70 con 0x564580942750 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad1f7fe640 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809ad410 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fad18005510 tx=0x7fad1800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae257129ffbdf3f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7facfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:16.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7facfeffd640 1 -- 172.21.15.8:0/2670649166 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1801f6f0 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7facfeffd640 1 -- 172.21.15.8:0/2670649166 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fad18005bd0 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.224+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56458092f0f0 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7facfeffd640 1 -- 172.21.15.8:0/2670649166 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad1800fe40 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 10 client.17280 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5645808618c0 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7facfeffd640 1 -- 172.21.15.8:0/2670649166 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56458092f0f0 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7facfeffd640 1 -- 172.21.15.8:0/2670649166 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad18028040 con 0x564580942750 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 10 client.17280 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:16.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 20 client.17280 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x5645809b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 4 client.17280 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fad14004960 con 0x564580952b40 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.225+0000 7fad1effd640 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x5645809b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7facfeffd640 10 client.17280 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad1effd640 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x5645809b1390 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bb5696d979cad6 server_cookie=2cb0d0b665ea6965 in_seq=0 out_seq=0 2024-02-19T16:30:16.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7facfeffd640 1 -- 172.21.15.8:0/2670649166 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fad14004960 con 0x564580952b40 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7facfeffd640 10 client.17280 handle_command_reply: tid=0 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 client.17280 shutdown 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 msgr2=0x5645809ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809ad410 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fad18005510 tx=0x7fad1800e9e0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 shutdown_connections 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564580942750 0x5645809ad410 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564580939b70 0x5645809466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x564580952b40 0x5645809b1390 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 --2- 172.21.15.8:0/2670649166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564580939250 0x564580946160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7facfeffd640 0 client.17280 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:16.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 >> 172.21.15.8:0/2670649166 conn(0x564580933980 msgr2=0x564580934b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 shutdown_connections 2024-02-19T16:30:16.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 1 -- 172.21.15.8:0/2670649166 wait complete. 2024-02-19T16:30:16.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.226+0000 7fad285e0740 20 client.17280 trim_cache size 0 max 16384 2024-02-19T16:30:16.238 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:30:16.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fad080a2230 msgr2=0x7fad080a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fad080a2230 0x7fad080a46f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad18009fd0 tx=0x7fad18009f60 comp rx=0 tx=0).stop 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 msgr2=0x7fad2019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 0x7fad2019d7b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad1000e9f0 tx=0x7fad1000edf0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 shutdown_connections 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad20197960 0x7fad20197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fad080a2230 0x7fad080a46f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad201056c0 0x7fad2019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 --2- 172.21.15.8:0/3938828557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad201005f0 0x7fad2019d7b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:16.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 >> 172.21.15.8:0/3938828557 conn(0x7fad200774c0 msgr2=0x7fad200fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:16.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 shutdown_connections 2024-02-19T16:30:16.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:16.227+0000 7fad2592e640 1 -- 172.21.15.8:0/3938828557 wait complete. 2024-02-19T16:30:17.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:16 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2619233189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:30:17.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:16 smithi008 ceph-mon[32573]: osdmap e100: 12 total, 12 up, 12 in 2024-02-19T16:30:17.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:16 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2619233189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:30:17.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:16 smithi191 ceph-mon[37203]: osdmap e100: 12 total, 12 up, 12 in 2024-02-19T16:30:17.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:16 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2619233189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:30:17.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:16 smithi156 ceph-mon[36502]: osdmap e100: 12 total, 12 up, 12 in 2024-02-19T16:30:17.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:30:17.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:30:17.243 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:30:17.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 -- 172.21.15.8:0/1640070047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c06a400 msgr2=0x7ff50c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 --2- 172.21.15.8:0/1640070047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c06a400 0x7ff50c10e340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009cd0 tx=0x7ff4fc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 -- 172.21.15.8:0/1640070047 shutdown_connections 2024-02-19T16:30:17.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 --2- 172.21.15.8:0/1640070047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c06a400 0x7ff50c10e340 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 --2- 172.21.15.8:0/1640070047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff50c069a60 0x7ff50c069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 --2- 172.21.15.8:0/1640070047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff50c107510 0x7ff50c1078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 -- 172.21.15.8:0/1640070047 >> 172.21.15.8:0/1640070047 conn(0x7ff50c0773d0 msgr2=0x7ff50c0777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 -- 172.21.15.8:0/1640070047 shutdown_connections 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.350+0000 7ff5117e9640 1 -- 172.21.15.8:0/1640070047 wait complete. 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 Processor -- start 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 -- start start 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 0x7ff50c1007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff50c06a400 0x7ff50c100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff50c107510 0x7ff50c101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a5e70 con 0x7ff50c069a60 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a5fe0 con 0x7ff50c06a400 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff5117e9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a61a0 con 0x7ff50c107510 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff50affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 0x7ff50c1007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff50affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 0x7ff50c1007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56602/0 (socket says 172.21.15.8:56602) 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.351+0000 7ff50affd640 1 -- 172.21.15.8:0/4239561673 learned_addr learned my addr 172.21.15.8:0/4239561673 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff50c06a400 0x7ff50c100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff50c107510 0x7ff50c101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50affd640 1 -- 172.21.15.8:0/4239561673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff50c107510 msgr2=0x7ff50c101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50affd640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff50c107510 0x7ff50c101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50affd640 1 -- 172.21.15.8:0/4239561673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff50c06a400 msgr2=0x7ff50c100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50affd640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff50c06a400 0x7ff50c100ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50affd640 1 -- 172.21.15.8:0/4239561673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc009900 con 0x7ff50c069a60 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff50affd640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 0x7ff50c1007a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9f0 tx=0x7ff50400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff503fff640 1 -- 172.21.15.8:0/4239561673 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504009720 con 0x7ff50c069a60 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff503fff640 1 -- 172.21.15.8:0/4239561673 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff50400cd80 con 0x7ff50c069a60 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c1a6480 con 0x7ff50c069a60 2024-02-19T16:30:17.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff503fff640 1 -- 172.21.15.8:0/4239561673 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504010690 con 0x7ff50c069a60 2024-02-19T16:30:17.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.352+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c1a6940 con 0x7ff50c069a60 2024-02-19T16:30:17.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 Processor -- start 2024-02-19T16:30:17.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff503fff640 1 -- 172.21.15.8:0/4239561673 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff5040046c0 con 0x7ff50c069a60 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 -- start start 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a178b0 0x563e72a19d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7297cf90 con 0x563e72a0ed30 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7297a4c0 con 0x563e72a178b0 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.355+0000 7ff5143c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72975ca0 con 0x563e72a0e380 2024-02-19T16:30:17.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56616/0 (socket says 172.21.15.8:56616) 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 -- 172.21.15.8:0/1537176888 learned_addr learned my addr 172.21.15.8:0/1537176888 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a178b0 0x563e72a19d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff503fff640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff4f80a2230 0x7ff4f80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 -- 172.21.15.8:0/1537176888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 msgr2=0x563e72a0e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a0e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 -- 172.21.15.8:0/1537176888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a178b0 msgr2=0x563e72a19d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a178b0 0x563e72a19d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 -- 172.21.15.8:0/1537176888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e729380b0 con 0x563e72a0ed30 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a178b0 0x563e72a19d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff4f80a2230 0x7ff4f80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff503fff640 1 -- 172.21.15.8:0/4239561673 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7ff504014070 con 0x7ff50c069a60 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50affd640 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a172e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff5040cba90 tx=0x7ff504090490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc18b684848d5dee server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff4f80a2230 0x7ff4f80a46f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005cc0 tx=0x7ff4fc0095c0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff5017fa640 1 -- 172.21.15.8:0/1537176888 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50409a900 con 0x563e72a0ed30 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff5017fa640 1 -- 172.21.15.8:0/1537176888 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff50409a720 con 0x563e72a0ed30 2024-02-19T16:30:17.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.356+0000 7ff5017fa640 1 -- 172.21.15.8:0/1537176888 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5040cbda0 con 0x563e72a0ed30 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 -- 172.21.15.8:0/1537176888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 msgr2=0x563e72a172e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a172e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff5040cba90 tx=0x7ff504090490 comp rx=0 tx=0).stop 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 -- 172.21.15.8:0/1537176888 shutdown_connections 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a178b0 0x563e72a19d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a172e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1537176888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 -- 172.21.15.8:0/1537176888 >> 172.21.15.8:0/1537176888 conn(0x563e72a08840 msgr2=0x563e72a08c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 -- 172.21.15.8:0/1537176888 shutdown_connections 2024-02-19T16:30:17.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 -- 172.21.15.8:0/1537176888 wait complete. 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.357+0000 7ff5143c9740 1 -- start start 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 Processor -- start 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a14790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0ed30 0x563e72a14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 0x563e72a15210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7291af40 con 0x563e72a178b0 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e726494b0 con 0x563e72a0ed30 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e728fb670 con 0x563e72a0e380 2024-02-19T16:30:17.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff5143c9740 10 client.0 _ll_register_callbacks cb 0x563e72968f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 0x563e72a15210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a14790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 0x563e72a15210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56630/0 (socket says 172.21.15.8:56630) 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 -- 172.21.15.8:0/3041566908 learned_addr learned my addr 172.21.15.8:0/3041566908 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0ed30 0x563e72a14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 -- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 msgr2=0x563e72a14790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a14790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 -- 172.21.15.8:0/3041566908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0ed30 msgr2=0x563e72a14cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0ed30 0x563e72a14cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 -- 172.21.15.8:0/3041566908 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c1a6940 con 0x563e72a178b0 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 0x563e72a15210 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc004af0 tx=0x7ff4fc02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3d3100cfc72f13 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.358+0000 7ff4f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff4f27fc640 1 -- 172.21.15.8:0/3041566908 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc032ac0 con 0x563e72a178b0 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e7294ff40 con 0x563e72a178b0 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff4f27fc640 1 -- 172.21.15.8:0/3041566908 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7ff4fc034e40 con 0x563e72a178b0 2024-02-19T16:30:17.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff4f27fc640 1 -- 172.21.15.8:0/3041566908 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4fc003970 con 0x563e72a178b0 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 10 client.17292 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563e72974320 con 0x563e72a178b0 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff4f27fc640 1 -- 172.21.15.8:0/3041566908 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563e7294ff40 con 0x563e72a178b0 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff4f27fc640 1 -- 172.21.15.8:0/3041566908 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4fc049050 con 0x563e72a178b0 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 10 client.17292 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 20 client.17292 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a27c50 0x563e72a2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 4 client.17292 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:17.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e7296c310 con 0x563e72a27c50 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.359+0000 7ff50affd640 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a27c50 0x563e72a2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.360+0000 7ff50affd640 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a27c50 0x563e72a2a070 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d215239f8b5d24c server_cookie=530d733663d98d0b in_seq=0 out_seq=0 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.360+0000 7ff4f27fc640 10 client.17292 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff4f27fc640 1 -- 172.21.15.8:0/3041566908 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563e7296c310 con 0x563e72a27c50 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff4f27fc640 10 client.17292 handle_command_reply: tid=0 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 client.17292 shutdown 2024-02-19T16:30:17.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 msgr2=0x563e72a15210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 0x563e72a15210 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc004af0 tx=0x7ff4fc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 shutdown_connections 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a178b0 0x563e72a15210 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0ed30 0x563e72a14cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a27c50 0x563e72a2a070 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 --2- 172.21.15.8:0/3041566908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0e380 0x563e72a14790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 >> 172.21.15.8:0/3041566908 conn(0x563e72a08840 msgr2=0x563e72a0a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff4f27fc640 0 client.17292 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 shutdown_connections 2024-02-19T16:30:17.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 1 -- 172.21.15.8:0/3041566908 wait complete. 2024-02-19T16:30:17.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.361+0000 7ff5143c9740 20 client.17292 trim_cache size 0 max 16384 2024-02-19T16:30:17.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 Processor -- start 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 -- start start 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 0x563e72a16ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a15030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0ed30 0x563e72a22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72a10100 con 0x563e72a08840 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72a0de50 con 0x563e72a0e380 2024-02-19T16:30:17.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff5143c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72a0d640 con 0x563e72a0ed30 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff50affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 0x563e72a16ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.377+0000 7ff50affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 0x563e72a16ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56644/0 (socket says 172.21.15.8:56644) 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 -- 172.21.15.8:0/818474200 learned_addr learned my addr 172.21.15.8:0/818474200 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a15030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0ed30 0x563e72a22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 -- 172.21.15.8:0/818474200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0ed30 msgr2=0x563e72a22cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0ed30 0x563e72a22cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 -- 172.21.15.8:0/818474200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 msgr2=0x563e72a15030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a15030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 -- 172.21.15.8:0/818474200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e72910c00 con 0x563e72a08840 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a15030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0ed30 0x563e72a22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff50affd640 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 0x563e72a16ec0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff5040cc120 tx=0x7ff504004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0ea7ac9c0a5d71 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff500ff9640 1 -- 172.21.15.8:0/818474200 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5040ce8d0 con 0x563e72a08840 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff500ff9640 1 -- 172.21.15.8:0/818474200 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff50409a900 con 0x563e72a08840 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.378+0000 7ff500ff9640 1 -- 172.21.15.8:0/818474200 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504097430 con 0x563e72a08840 2024-02-19T16:30:17.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- 172.21.15.8:0/818474200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 msgr2=0x563e72a16ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 0x563e72a16ec0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff5040cc120 tx=0x7ff504004b30 comp rx=0 tx=0).stop 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- 172.21.15.8:0/818474200 shutdown_connections 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a0ed30 0x563e72a22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a15030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- 172.21.15.8:0/818474200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a08840 0x563e72a16ec0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- 172.21.15.8:0/818474200 >> 172.21.15.8:0/818474200 conn(0x563e72a0f480 msgr2=0x563e72a078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- 172.21.15.8:0/818474200 shutdown_connections 2024-02-19T16:30:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- 172.21.15.8:0/818474200 wait complete. 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- start start 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 Processor -- start 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a08840 0x563e72a11610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a11b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a7c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72a7ca10 con 0x563e72a0ed30 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72a7cb80 con 0x563e72a0e380 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e72a7cd40 con 0x563e72a08840 2024-02-19T16:30:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff5143c9740 10 client.0 _ll_register_callbacks cb 0x7ff504005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a7c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a7c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56660/0 (socket says 172.21.15.8:56660) 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff50b7fe640 1 -- 172.21.15.8:0/1357703892 learned_addr learned my addr 172.21.15.8:0/1357703892 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff50a7fc640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a11b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.379+0000 7ff50affd640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a08840 0x563e72a11610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff50b7fe640 1 -- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a08840 msgr2=0x563e72a11610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a08840 0x563e72a11610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff50b7fe640 1 -- 172.21.15.8:0/1357703892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 msgr2=0x563e72a11b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a11b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff50b7fe640 1 -- 172.21.15.8:0/1357703892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e729380b0 con 0x563e72a0ed30 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff50b7fe640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a7c4d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005d50 tx=0x7ff4fc0328c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53364548d35fb2a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff4f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff4f27fc640 1 -- 172.21.15.8:0/1357703892 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc032b70 con 0x563e72a0ed30 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff4f27fc640 1 -- 172.21.15.8:0/1357703892 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7ff4fc04cdf0 con 0x563e72a0ed30 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e7299b900 con 0x563e72a0ed30 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff4f27fc640 1 -- 172.21.15.8:0/1357703892 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4fc061470 con 0x563e72a0ed30 2024-02-19T16:30:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff5143c9740 10 client.17298 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563e72936910 con 0x563e72a0ed30 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff4f27fc640 1 -- 172.21.15.8:0/1357703892 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563e7299b900 con 0x563e72a0ed30 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff4f27fc640 1 -- 172.21.15.8:0/1357703892 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4fc05f050 con 0x563e72a0ed30 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff5143c9740 10 client.17298 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff5143c9740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.380+0000 7ff5143c9740 20 client.17298 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.381+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a178b0 0x563e72a861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.381+0000 7ff5143c9740 4 client.17298 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.381+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563e72675860 con 0x563e72a178b0 2024-02-19T16:30:17.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.381+0000 7ff50affd640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a178b0 0x563e72a861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.381+0000 7ff50affd640 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a178b0 0x563e72a861f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4af04b0ca00b4b1 server_cookie=bbf564ab6775aa10 in_seq=0 out_seq=0 2024-02-19T16:30:17.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.381+0000 7ff4f27fc640 10 client.17298 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff4f27fc640 1 -- 172.21.15.8:0/1357703892 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563e72675860 con 0x563e72a178b0 2024-02-19T16:30:17.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff4f27fc640 10 client.17298 handle_command_reply: tid=0 2024-02-19T16:30:17.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 client.17298 shutdown 2024-02-19T16:30:17.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 msgr2=0x563e72a7c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a7c4d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005d50 tx=0x7ff4fc0328c0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 shutdown_connections 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563e72a0ed30 0x563e72a7c4d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e72a0e380 0x563e72a11b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563e72a178b0 0x563e72a861f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 --2- 172.21.15.8:0/1357703892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e72a08840 0x563e72a11610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 >> 172.21.15.8:0/1357703892 conn(0x563e72a0f480 msgr2=0x563e72a2e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff4f27fc640 0 client.17298 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.393 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 shutdown_connections 2024-02-19T16:30:17.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 1 -- 172.21.15.8:0/1357703892 wait complete. 2024-02-19T16:30:17.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.382+0000 7ff5143c9740 20 client.17298 trim_cache size 0 max 16384 2024-02-19T16:30:17.394 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:30:17.394 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:30:17.394 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "91232e55-7deb-407f-bb55-3e6c463d5f2c", 2024-02-19T16:30:17.394 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:30:17.395 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:30:17.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff4f80a2230 msgr2=0x7ff4f80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff4f80a2230 0x7ff4f80a46f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005cc0 tx=0x7ff4fc0095c0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 msgr2=0x7ff50c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 0x7ff50c1007a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9f0 tx=0x7ff50400edf0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 shutdown_connections 2024-02-19T16:30:17.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff4f80a2230 0x7ff4f80a46f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff50c107510 0x7ff50c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff50c06a400 0x7ff50c100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 --2- 172.21.15.8:0/4239561673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff50c069a60 0x7ff50c1007a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 >> 172.21.15.8:0/4239561673 conn(0x7ff50c0773d0 msgr2=0x7ff50c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 shutdown_connections 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.383+0000 7ff5117e9640 1 -- 172.21.15.8:0/4239561673 wait complete. 2024-02-19T16:30:17.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:30:17.398 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 -- 172.21.15.8:0/870601349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f612806a520 msgr2=0x7f612810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 --2- 172.21.15.8:0/870601349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f612806a520 0x7f612810e310 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6120009cd0 tx=0x7f612002c180 comp rx=0 tx=0).stop 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 -- 172.21.15.8:0/870601349 shutdown_connections 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 --2- 172.21.15.8:0/870601349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f612806a520 0x7f612810e310 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 --2- 172.21.15.8:0/870601349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6128069b60 0x7f6128069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 --2- 172.21.15.8:0/870601349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6128107600 0x7f61281079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.506 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 -- 172.21.15.8:0/870601349 >> 172.21.15.8:0/870601349 conn(0x7f61280771d0 msgr2=0x7f61280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 -- 172.21.15.8:0/870601349 shutdown_connections 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.502+0000 7f612eff7640 1 -- 172.21.15.8:0/870601349 wait complete. 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 Processor -- start 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 -- start start 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 0x7f612819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f612806a520 0x7f612819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6128107600 0x7f6128197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128197ee0 con 0x7f6128069b60 2024-02-19T16:30:17.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128198050 con 0x7f6128107600 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612eff7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128198210 con 0x7f612806a520 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612cd93640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 0x7f612819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612cd93640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 0x7f612819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56680/0 (socket says 172.21.15.8:56680) 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612cd93640 1 -- 172.21.15.8:0/603321873 learned_addr learned my addr 172.21.15.8:0/603321873 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f6127fff640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f612806a520 0x7f612819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.503+0000 7f612d594640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6128107600 0x7f6128197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612cd93640 1 -- 172.21.15.8:0/603321873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f612806a520 msgr2=0x7f612819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612cd93640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f612806a520 0x7f612819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612cd93640 1 -- 172.21.15.8:0/603321873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6128107600 msgr2=0x7f6128197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612cd93640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6128107600 0x7f6128197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612cd93640 1 -- 172.21.15.8:0/603321873 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120009900 con 0x7f6128069b60 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612cd93640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 0x7f612819d6d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9f0 tx=0x7f611c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f6125ffb640 1 -- 172.21.15.8:0/603321873 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c009700 con 0x7f6128069b60 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61281984f0 con 0x7f6128069b60 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f6125ffb640 1 -- 172.21.15.8:0/603321873 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f611c00cd80 con 0x7f6128069b60 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f6125ffb640 1 -- 172.21.15.8:0/603321873 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c018920 con 0x7f6128069b60 2024-02-19T16:30:17.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.504+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61281a4700 con 0x7f6128069b60 2024-02-19T16:30:17.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6125ffb640 1 -- 172.21.15.8:0/603321873 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f611c018a80 con 0x7f6128069b60 2024-02-19T16:30:17.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 Processor -- start 2024-02-19T16:30:17.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 -- start start 2024-02-19T16:30:17.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c38a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c393270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c395ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c2f9280 con 0x56213c3937b0 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c2f67b0 con 0x56213c38abd0 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6131bd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c2f1f90 con 0x56213c38a2b0 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6127fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c395ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6125ffb640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61180a2230 0x7f61180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f612cd93640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c393270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6127fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c395ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56684/0 (socket says 172.21.15.8:56684) 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f6127fff640 1 -- 172.21.15.8:0/1744556150 learned_addr learned my addr 172.21.15.8:0/1744556150 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.507+0000 7f612d594640 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c38a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f612d594640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61180a2230 0x7f61180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6127fff640 1 -- 172.21.15.8:0/1744556150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 msgr2=0x56213c38a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6127fff640 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c38a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6127fff640 1 -- 172.21.15.8:0/1744556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 msgr2=0x56213c393270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6127fff640 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c393270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6127fff640 1 -- 172.21.15.8:0/1744556150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213c2b0570 con 0x56213c3937b0 2024-02-19T16:30:17.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6125ffb640 1 -- 172.21.15.8:0/603321873 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f611c014070 con 0x7f6128069b60 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6127fff640 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c395ba0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c60 tx=0x7f6114011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1af07626d94909 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f612d594640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61180a2230 0x7f61180a46f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6120005cc0 tx=0x7f612002c180 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6112ffd640 1 -- 172.21.15.8:0/1744556150 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114004230 con 0x56213c3937b0 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6112ffd640 1 -- 172.21.15.8:0/1744556150 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6114004390 con 0x56213c3937b0 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.508+0000 7f6112ffd640 1 -- 172.21.15.8:0/1744556150 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61140196d0 con 0x56213c3937b0 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 -- 172.21.15.8:0/1744556150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 msgr2=0x56213c395ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c395ba0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c60 tx=0x7f6114011810 comp rx=0 tx=0).stop 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 -- 172.21.15.8:0/1744556150 shutdown_connections 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c395ba0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c393270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 --2- 172.21.15.8:0/1744556150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c38a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 -- 172.21.15.8:0/1744556150 >> 172.21.15.8:0/1744556150 conn(0x56213c384990 msgr2=0x56213c384d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 -- 172.21.15.8:0/1744556150 shutdown_connections 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 -- 172.21.15.8:0/1744556150 wait complete. 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 -- start start 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.509+0000 7f6131bd1740 1 Processor -- start 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c393180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c390510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c390b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213bfc5430 con 0x56213c3937b0 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213bf81e00 con 0x56213c38abd0 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c276fd0 con 0x56213c38a2b0 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6131bd1740 10 client.0 _ll_register_callbacks cb 0x56213c2ccfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c390b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c390b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56694/0 (socket says 172.21.15.8:56694) 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 -- 172.21.15.8:0/3165384217 learned_addr learned my addr 172.21.15.8:0/3165384217 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612cd93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c393180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6127fff640 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c390510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 -- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 msgr2=0x56213c393180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c393180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 -- 172.21.15.8:0/3165384217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 msgr2=0x56213c390510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c390510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 -- 172.21.15.8:0/3165384217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6114009890 con 0x56213c3937b0 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f612d594640 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c390b20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6120036000 tx=0x7f61200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5094a50fdb703fe1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6107fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6107fff640 1 -- 172.21.15.8:0/3165384217 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120003cb0 con 0x56213c3937b0 2024-02-19T16:30:17.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.510+0000 7f6107fff640 1 -- 172.21.15.8:0/3165384217 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f6120003e10 con 0x56213c3937b0 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56213bff17e0 con 0x56213c3937b0 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6107fff640 1 -- 172.21.15.8:0/3165384217 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6120004040 con 0x56213c3937b0 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 10 client.17310 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56213c2f0610 con 0x56213c3937b0 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6107fff640 1 -- 172.21.15.8:0/3165384217 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56213bff17e0 con 0x56213c3937b0 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6107fff640 1 -- 172.21.15.8:0/3165384217 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f612003e040 con 0x56213c3937b0 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 10 client.17310 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:17.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 20 client.17310 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c3a39a0 0x56213c3a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 4 client.17310 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56213c02e720 con 0x56213c3a39a0 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.511+0000 7f612cd93640 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c3a39a0 0x56213c3a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.512+0000 7f612cd93640 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c3a39a0 0x56213c3a5dc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e976c375c25d6213 server_cookie=a2284ee180082d8 in_seq=0 out_seq=0 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.512+0000 7f6107fff640 10 client.17310 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6107fff640 1 -- 172.21.15.8:0/3165384217 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56213c02e720 con 0x56213c3a39a0 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6107fff640 10 client.17310 handle_command_reply: tid=0 2024-02-19T16:30:17.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 client.17310 shutdown 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 msgr2=0x56213c390b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c390b20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6120036000 tx=0x7f61200095c0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 shutdown_connections 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c390b20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38abd0 0x56213c390510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c3a39a0 0x56213c3a5dc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3165384217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38a2b0 0x56213c393180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6107fff640 0 client.17310 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 >> 172.21.15.8:0/3165384217 conn(0x56213c384990 msgr2=0x56213c386730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 shutdown_connections 2024-02-19T16:30:17.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 1 -- 172.21.15.8:0/3165384217 wait complete. 2024-02-19T16:30:17.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.513+0000 7f6131bd1740 20 client.17310 trim_cache size 0 max 16384 2024-02-19T16:30:17.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 Processor -- start 2024-02-19T16:30:17.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 -- start start 2024-02-19T16:30:17.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 0x56213c392c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c393180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c39eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c389cf0 con 0x56213c3937b0 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c386cf0 con 0x56213c38a2b0 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6131bd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c384160 con 0x56213c38abd0 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612d594640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c39eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 0x56213c392c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 0x56213c392c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:34322/0 (socket says 172.21.15.8:34322) 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612d594640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c39eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56700/0 (socket says 172.21.15.8:56700) 2024-02-19T16:30:17.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 -- 172.21.15.8:0/3915459997 learned_addr learned my addr 172.21.15.8:0/3915459997 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f6127fff640 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c393180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 -- 172.21.15.8:0/3915459997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 msgr2=0x56213c393180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c393180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 -- 172.21.15.8:0/3915459997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 msgr2=0x56213c39eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c39eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612cd93640 1 -- 172.21.15.8:0/3915459997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213c28cac0 con 0x56213c38a2b0 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.532+0000 7f612d594640 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c39eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:17.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.533+0000 7f612cd93640 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 0x56213c392c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f611c097330 tx=0x7f611c0d0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ff177290bf662 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.533+0000 7f61127fc640 1 -- 172.21.15.8:0/3915459997 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b2070 con 0x56213c38a2b0 2024-02-19T16:30:17.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.533+0000 7f61127fc640 1 -- 172.21.15.8:0/3915459997 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f611c09ef70 con 0x56213c38a2b0 2024-02-19T16:30:17.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.533+0000 7f61127fc640 1 -- 172.21.15.8:0/3915459997 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c010ca0 con 0x56213c38a2b0 2024-02-19T16:30:17.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- 172.21.15.8:0/3915459997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 msgr2=0x56213c392c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 0x56213c392c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f611c097330 tx=0x7f611c0d0040 comp rx=0 tx=0).stop 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- 172.21.15.8:0/3915459997 shutdown_connections 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c3937b0 0x56213c39eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c393180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- 172.21.15.8:0/3915459997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c38a2b0 0x56213c392c40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- 172.21.15.8:0/3915459997 >> 172.21.15.8:0/3915459997 conn(0x56213c384990 msgr2=0x56213c383a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- 172.21.15.8:0/3915459997 shutdown_connections 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- 172.21.15.8:0/3915459997 wait complete. 2024-02-19T16:30:17.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- start start 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 Processor -- start 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 0x56213c38e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c394b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c3937b0 0x56213c395050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c395590 con 0x56213c38a2b0 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c395700 con 0x56213c3937b0 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213c3958c0 con 0x56213c38abd0 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.534+0000 7f6131bd1740 10 client.0 _ll_register_callbacks cb 0x7f6114005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:17.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 0x56213c38e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 0x56213c38e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56716/0 (socket says 172.21.15.8:56716) 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 -- 172.21.15.8:0/2761044592 learned_addr learned my addr 172.21.15.8:0/2761044592 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6127fff640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c394b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612d594640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c3937b0 0x56213c395050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 -- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 msgr2=0x56213c394b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c394b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 -- 172.21.15.8:0/2761044592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c3937b0 msgr2=0x56213c395050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c3937b0 0x56213c395050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 -- 172.21.15.8:0/2761044592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56213c2f0610 con 0x56213c38a2b0 2024-02-19T16:30:17.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f612cd93640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 0x56213c38e040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f611c009380 tx=0x7f611c09b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791791b810630ba4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6107fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6107fff640 1 -- 172.21.15.8:0/2761044592 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b2030 con 0x56213c38a2b0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6107fff640 1 -- 172.21.15.8:0/2761044592 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f611c0b3070 con 0x56213c38a2b0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6114004390 con 0x56213c38a2b0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6107fff640 1 -- 172.21.15.8:0/2761044592 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f611c0991d0 con 0x56213c38a2b0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6131bd1740 10 client.17316 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.535+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f611c09e040 con 0x56213c38a2b0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6107fff640 1 -- 172.21.15.8:0/2761044592 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6114004390 con 0x56213c38a2b0 2024-02-19T16:30:17.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6107fff640 1 -- 172.21.15.8:0/2761044592 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f611c0b3b50 con 0x56213c38a2b0 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6131bd1740 10 client.17316 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6131bd1740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6131bd1740 20 client.17316 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6131bd1740 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c400510 0x56213c402900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6131bd1740 4 client.17316 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56213c07d890 con 0x56213c400510 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6127fff640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c400510 0x56213c402900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6127fff640 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c400510 0x56213c402900 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888e7f95e47cfafa server_cookie=86b8dae2897382b5 in_seq=0 out_seq=0 2024-02-19T16:30:17.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.536+0000 7f6107fff640 10 client.17316 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6107fff640 1 -- 172.21.15.8:0/2761044592 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x56213c07d890 con 0x56213c400510 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6107fff640 10 client.17316 handle_command_reply: tid=0 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 client.17316 shutdown 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 msgr2=0x56213c38e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 0x56213c38e040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f611c009380 tx=0x7f611c09b050 comp rx=0 tx=0).stop 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 shutdown_connections 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213c3937b0 0x56213c395050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56213c400510 0x56213c402900 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56213c38abd0 0x56213c394b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6107fff640 0 client.17316 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 --2- 172.21.15.8:0/2761044592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56213c38a2b0 0x56213c38e040 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 >> 172.21.15.8:0/2761044592 conn(0x56213c384990 msgr2=0x56213c385ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 shutdown_connections 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 1 -- 172.21.15.8:0/2761044592 wait complete. 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.537+0000 7f6131bd1740 20 client.17316 trim_cache size 0 max 16384 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:30:17.547 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2823 inodes in the stack)", 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: "91232e55-7deb-407f-bb55-3e6c463d5f2c": { 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "91232e55-7deb-407f-bb55-3e6c463d5f2c", 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:30:17.548 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:30:17.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61180a2230 msgr2=0x7f61180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61180a2230 0x7f61180a46f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6120005cc0 tx=0x7f612002c180 comp rx=0 tx=0).stop 2024-02-19T16:30:17.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 msgr2=0x7f612819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:17.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 0x7f612819d6d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9f0 tx=0x7f611c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 shutdown_connections 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6128107600 0x7f6128197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f61180a2230 0x7f61180a46f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f612806a520 0x7f612819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 --2- 172.21.15.8:0/603321873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6128069b60 0x7f612819d6d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 >> 172.21.15.8:0/603321873 conn(0x7f61280771d0 msgr2=0x7f61280ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 shutdown_connections 2024-02-19T16:30:17.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:17.538+0000 7f612eff7640 1 -- 172.21.15.8:0/603321873 wait complete. 2024-02-19T16:30:17.551 INFO:tasks.cephfs.filesystem:scrub status for tag:91232e55-7deb-407f-bb55-3e6c463d5f2c - {'path': '/', 'tag': '91232e55-7deb-407f-bb55-3e6c463d5f2c', 'options': 'recursive,force'} 2024-02-19T16:30:18.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: pgmap v229: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:30:18.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:18.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.772 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:18 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: pgmap v229: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:30:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:18 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: pgmap v229: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:30:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:30:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:18.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:18.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:18.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:18 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:19.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:19 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:19.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:19 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:19.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:19 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:19.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:19 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:19.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:19 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:19.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:19 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:20.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:20 smithi008 ceph-mon[32573]: pgmap v230: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 229 B/s wr, 7 op/s 2024-02-19T16:30:20.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:20 smithi191 ceph-mon[37203]: pgmap v230: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 229 B/s wr, 7 op/s 2024-02-19T16:30:20.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:20 smithi156 ceph-mon[36502]: pgmap v230: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 229 B/s wr, 7 op/s 2024-02-19T16:30:22.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:22 smithi008 ceph-mon[32573]: pgmap v231: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:22 smithi191 ceph-mon[37203]: pgmap v231: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:22 smithi156 ceph-mon[36502]: pgmap v231: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:24.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:24 smithi008 ceph-mon[32573]: pgmap v232: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:24.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:24 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:30:24.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:24 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:30:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:24 smithi191 ceph-mon[37203]: pgmap v232: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:24 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:30:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:24 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:30:24.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:24 smithi156 ceph-mon[36502]: pgmap v232: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:30:24.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:24 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:30:24.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:24 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:30:25.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:25 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:30:25.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:25 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:30:25.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:25 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:30:26.756 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:26 smithi008 ceph-mon[32573]: pgmap v233: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:30:26.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:26 smithi191 ceph-mon[37203]: pgmap v233: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:30:26.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:26 smithi156 ceph-mon[36502]: pgmap v233: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:30:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:28 smithi008 ceph-mon[32573]: pgmap v234: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:30:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:28 smithi191 ceph-mon[37203]: pgmap v234: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:30:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:28 smithi156 ceph-mon[36502]: pgmap v234: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:30:30.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:30 smithi008 ceph-mon[32573]: pgmap v235: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:30:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:30 smithi191 ceph-mon[37203]: pgmap v235: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:30:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:30 smithi156 ceph-mon[36502]: pgmap v235: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:30:32.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:32 smithi008 ceph-mon[32573]: pgmap v236: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:32.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:32 smithi191 ceph-mon[37203]: pgmap v236: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:32.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:32 smithi156 ceph-mon[36502]: pgmap v236: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:34 smithi008 ceph-mon[32573]: pgmap v237: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:34 smithi191 ceph-mon[37203]: pgmap v237: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:34 smithi156 ceph-mon[36502]: pgmap v237: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:36.772 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:36 smithi008 ceph-mon[32573]: pgmap v238: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:36.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:36 smithi191 ceph-mon[37203]: pgmap v238: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:36 smithi156 ceph-mon[36502]: pgmap v238: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:38 smithi008 ceph-mon[32573]: pgmap v239: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:38 smithi191 ceph-mon[37203]: pgmap v239: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:38.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:38 smithi156 ceph-mon[36502]: pgmap v239: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:40.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:40 smithi008 ceph-mon[32573]: pgmap v240: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:40 smithi191 ceph-mon[37203]: pgmap v240: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:40 smithi156 ceph-mon[36502]: pgmap v240: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:42.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:42 smithi008 ceph-mon[32573]: pgmap v241: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:42 smithi191 ceph-mon[37203]: pgmap v241: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:42 smithi156 ceph-mon[36502]: pgmap v241: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:44.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:44 smithi008 ceph-mon[32573]: pgmap v242: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:44 smithi191 ceph-mon[37203]: pgmap v242: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:44 smithi156 ceph-mon[36502]: pgmap v242: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:46.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:46 smithi008 ceph-mon[32573]: pgmap v243: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:46 smithi191 ceph-mon[37203]: pgmap v243: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:46 smithi156 ceph-mon[36502]: pgmap v243: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:47.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:30:47.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:30:47.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 -- 172.21.15.8:0/2060141300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc20806a480 msgr2=0x7fc20810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 --2- 172.21.15.8:0/2060141300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc20806a480 0x7fc20810e330 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc200009930 tx=0x7fc20002bc30 comp rx=0 tx=0).stop 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 -- 172.21.15.8:0/2060141300 shutdown_connections 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 --2- 172.21.15.8:0/2060141300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc20806a480 0x7fc20810e330 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 --2- 172.21.15.8:0/2060141300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc208069ac0 0x7fc208069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 --2- 172.21.15.8:0/2060141300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc208107520 0x7fc208107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 -- 172.21.15.8:0/2060141300 >> 172.21.15.8:0/2060141300 conn(0x7fc2080773d0 msgr2=0x7fc2080777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 -- 172.21.15.8:0/2060141300 shutdown_connections 2024-02-19T16:30:47.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 -- 172.21.15.8:0/2060141300 wait complete. 2024-02-19T16:30:47.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 Processor -- start 2024-02-19T16:30:47.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.669+0000 7fc20d68f640 1 -- start start 2024-02-19T16:30:47.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc20d68f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc208069ac0 0x7fc20819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc20d68f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 0x7fc20819dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc20d68f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc208107520 0x7fc2081978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc20d68f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208197f10 con 0x7fc208107520 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc20d68f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208198080 con 0x7fc208069ac0 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc20d68f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208198240 con 0x7fc20806a480 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2077fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc208107520 0x7fc2081978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 0x7fc20819dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 0x7fc20819dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:48330/0 (socket says 172.21.15.8:48330) 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 -- 172.21.15.8:0/3762482750 learned_addr learned my addr 172.21.15.8:0/3762482750 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc206ffd640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc208069ac0 0x7fc20819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 -- 172.21.15.8:0/3762482750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc208069ac0 msgr2=0x7fc20819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc208069ac0 0x7fc20819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 -- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc208107520 msgr2=0x7fc2081978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc208107520 0x7fc2081978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 -- 172.21.15.8:0/3762482750 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f8009840 con 0x7fc20806a480 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.670+0000 7fc2067fc640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 0x7fc20819dc50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e9f0 tx=0x7fc1fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.671+0000 7fc1effff640 1 -- 172.21.15.8:0/3762482750 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc009700 con 0x7fc20806a480 2024-02-19T16:30:47.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.671+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc200009560 con 0x7fc20806a480 2024-02-19T16:30:47.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.671+0000 7fc1effff640 1 -- 172.21.15.8:0/3762482750 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc00cd80 con 0x7fc20806a480 2024-02-19T16:30:47.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.671+0000 7fc1effff640 1 -- 172.21.15.8:0/3762482750 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc018920 con 0x7fc20806a480 2024-02-19T16:30:47.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.671+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc208198710 con 0x7fc20806a480 2024-02-19T16:30:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.673+0000 7fc1effff640 1 -- 172.21.15.8:0/3762482750 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fc1fc018a80 con 0x7fc20806a480 2024-02-19T16:30:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.673+0000 7fc210268740 1 Processor -- start 2024-02-19T16:30:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.673+0000 7fc210268740 1 -- start start 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc210268740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad8062b0 0x5574ad806690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc210268740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad80f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc210268740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad811ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc210268740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad775280 con 0x5574ad80f7b0 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc210268740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad7727b0 con 0x5574ad8062b0 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc210268740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad76df90 con 0x5574ad806bd0 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad811ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2077fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad8062b0 0x5574ad806690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad811ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39366/0 (socket says 172.21.15.8:39366) 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 -- 172.21.15.8:0/3081716041 learned_addr learned my addr 172.21.15.8:0/3081716041 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc1effff640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc1f00a2230 0x7fc1f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc206ffd640 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad80f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2077fe640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc1f00a2230 0x7fc1f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 -- 172.21.15.8:0/3081716041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 msgr2=0x5574ad80f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad80f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 -- 172.21.15.8:0/3081716041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad8062b0 msgr2=0x5574ad806690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad8062b0 0x5574ad806690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc1effff640 1 -- 172.21.15.8:0/3762482750 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fc1fc014070 con 0x7fc20806a480 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 -- 172.21.15.8:0/3081716041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ad72c570 con 0x5574ad80f7b0 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2077fe640 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad8062b0 0x5574ad806690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc2067fc640 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad811ba0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc09abd0 tx=0x7fc1fc010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73fd3b621cd245c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.674+0000 7fc1ed7fa640 1 -- 172.21.15.8:0/3081716041 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc09aed0 con 0x5574ad80f7b0 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc1ed7fa640 1 -- 172.21.15.8:0/3081716041 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc0d3030 con 0x5574ad80f7b0 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc1ed7fa640 1 -- 172.21.15.8:0/3081716041 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc05f130 con 0x5574ad80f7b0 2024-02-19T16:30:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 -- 172.21.15.8:0/3081716041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 msgr2=0x5574ad811ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc2077fe640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc1f00a2230 0x7fc1f00a46f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc200005940 tx=0x7fc20002bc30 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad811ba0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc09abd0 tx=0x7fc1fc010f50 comp rx=0 tx=0).stop 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 -- 172.21.15.8:0/3081716041 shutdown_connections 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad811ba0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad80f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 --2- 172.21.15.8:0/3081716041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad8062b0 0x5574ad806690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 -- 172.21.15.8:0/3081716041 >> 172.21.15.8:0/3081716041 conn(0x5574ad800990 msgr2=0x5574ad800d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 -- 172.21.15.8:0/3081716041 shutdown_connections 2024-02-19T16:30:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.675+0000 7fc210268740 1 -- 172.21.15.8:0/3081716041 wait complete. 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 -- start start 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 Processor -- start 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad8062b0 0x5574ad80f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad80ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad441430 con 0x5574ad80f7b0 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad3fde00 con 0x5574ad806bd0 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad6f2fd0 con 0x5574ad8062b0 2024-02-19T16:30:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc210268740 10 client.0 _ll_register_callbacks cb 0x5574ad748fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad80ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad8062b0 0x5574ad80f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad80ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39382/0 (socket says 172.21.15.8:39382) 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 -- 172.21.15.8:0/1159158622 learned_addr learned my addr 172.21.15.8:0/1159158622 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2067fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc206ffd640 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad8062b0 0x5574ad80f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:48350/0 (socket says 172.21.15.8:48350) 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 -- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad8062b0 msgr2=0x5574ad80f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad8062b0 0x5574ad80f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 -- 172.21.15.8:0/1159158622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 msgr2=0x5574ad80c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.676+0000 7fc2077fe640 1 -- 172.21.15.8:0/1159158622 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc208198710 con 0x5574ad80f7b0 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc2077fe640 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad80ca40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc200005ab0 tx=0x7fc200009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ce6b155077ac21 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc1eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc1eaff5640 1 -- 172.21.15.8:0/1159158622 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200005000 con 0x5574ad80f7b0 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc1eaff5640 1 -- 172.21.15.8:0/1159158622 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fc200005160 con 0x5574ad80f7b0 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574ad46d7e0 con 0x5574ad80f7b0 2024-02-19T16:30:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc1eaff5640 1 -- 172.21.15.8:0/1159158622 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc200005320 con 0x5574ad80f7b0 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 10 client.17328 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5574ad76c610 con 0x5574ad80f7b0 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc1eaff5640 1 -- 172.21.15.8:0/1159158622 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5574ad46d7e0 con 0x5574ad80f7b0 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc1eaff5640 1 -- 172.21.15.8:0/1159158622 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc20003c070 con 0x5574ad80f7b0 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 10 client.17328 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 20 client.17328 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad81f910 0x5574ad821d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 4 client.17328 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.677+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574ad4aa720 con 0x5574ad81f910 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.678+0000 7fc206ffd640 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad81f910 0x5574ad821d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.678+0000 7fc1eaff5640 10 client.17328 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.678+0000 7fc206ffd640 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad81f910 0x5574ad821d30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce173895f4ae813f server_cookie=85fa0b54ae15d8e5 in_seq=0 out_seq=0 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.678+0000 7fc1eaff5640 1 -- 172.21.15.8:0/1159158622 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5574ad4aa720 con 0x5574ad81f910 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.678+0000 7fc1eaff5640 10 client.17328 handle_command_reply: tid=0 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.678+0000 7fc210268740 1 client.17328 shutdown 2024-02-19T16:30:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 msgr2=0x5574ad80ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad80ca40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc200005ab0 tx=0x7fc200009260 comp rx=0 tx=0).stop 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 shutdown_connections 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad80f7b0 0x5574ad80ca40 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad81f910 0x5574ad821d30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 --2- 172.21.15.8:0/1159158622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad8062b0 0x5574ad80f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 >> 172.21.15.8:0/1159158622 conn(0x5574ad800990 msgr2=0x5574ad802730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc1eaff5640 0 client.17328 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 shutdown_connections 2024-02-19T16:30:47.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 1 -- 172.21.15.8:0/1159158622 wait complete. 2024-02-19T16:30:47.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.679+0000 7fc210268740 20 client.17328 trim_cache size 0 max 16384 2024-02-19T16:30:47.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 Processor -- start 2024-02-19T16:30:47.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 -- start start 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad80ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad80f7b0 0x5574ad81ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad80aa10 con 0x5574ad8062b0 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad811ec0 con 0x5574ad806bd0 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc210268740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad80cd60 con 0x5574ad80f7b0 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad80ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad80ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39388/0 (socket says 172.21.15.8:39388) 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 -- 172.21.15.8:0/1112028877 learned_addr learned my addr 172.21.15.8:0/1112028877 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc2067fc640 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc2077fe640 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad80f7b0 0x5574ad81ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 -- 172.21.15.8:0/1112028877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad80f7b0 msgr2=0x5574ad81ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad80f7b0 0x5574ad81ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 -- 172.21.15.8:0/1112028877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 msgr2=0x5574ad80f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc206ffd640 1 -- 172.21.15.8:0/1112028877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ad708ac0 con 0x5574ad8062b0 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.695+0000 7fc2077fe640 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad80f7b0 0x5574ad81ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc206ffd640 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad80ebf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005fd0 tx=0x7fc1f8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1a54caa3043970 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc1ecff9640 1 -- 172.21.15.8:0/1112028877 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8016030 con 0x5574ad8062b0 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc1ecff9640 1 -- 172.21.15.8:0/1112028877 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1f8017040 con 0x5574ad8062b0 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc1ecff9640 1 -- 172.21.15.8:0/1112028877 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80042a0 con 0x5574ad8062b0 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 -- 172.21.15.8:0/1112028877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 msgr2=0x5574ad80ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad80ebf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005fd0 tx=0x7fc1f8009500 comp rx=0 tx=0).stop 2024-02-19T16:30:47.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 -- 172.21.15.8:0/1112028877 shutdown_connections 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad80f7b0 0x5574ad81ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad806bd0 0x5574ad80f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 --2- 172.21.15.8:0/1112028877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad80ebf0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 -- 172.21.15.8:0/1112028877 >> 172.21.15.8:0/1112028877 conn(0x5574ad800990 msgr2=0x5574ad7ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 -- 172.21.15.8:0/1112028877 shutdown_connections 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.696+0000 7fc210268740 1 -- 172.21.15.8:0/1112028877 wait complete. 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 -- start start 2024-02-19T16:30:47.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 Processor -- start 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad8114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad812600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad80f7b0 0x5574ad812b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad813100 con 0x5574ad8062b0 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad813270 con 0x5574ad80f7b0 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad813430 con 0x5574ad806bd0 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc210268740 10 client.0 _ll_register_callbacks cb 0x7fc200009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad8114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad8114e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39390/0 (socket says 172.21.15.8:39390) 2024-02-19T16:30:47.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 -- 172.21.15.8:0/2973309936 learned_addr learned my addr 172.21.15.8:0/2973309936 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc2067fc640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad812600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc2077fe640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad80f7b0 0x5574ad812b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 -- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 msgr2=0x5574ad812600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad812600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 -- 172.21.15.8:0/2973309936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad80f7b0 msgr2=0x5574ad812b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad80f7b0 0x5574ad812b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 -- 172.21.15.8:0/2973309936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f8010550 con 0x5574ad8062b0 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc206ffd640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad8114e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc1f801ecb0 tx=0x7fc1f801e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227091390edacc1a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.697+0000 7fc1eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc1eaff5640 1 -- 172.21.15.8:0/2973309936 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8016070 con 0x5574ad8062b0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc1eaff5640 1 -- 172.21.15.8:0/2973309936 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fc1f8004ab0 con 0x5574ad8062b0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1fc0d3030 con 0x5574ad8062b0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc1eaff5640 1 -- 172.21.15.8:0/2973309936 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f8004c70 con 0x5574ad8062b0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 10 client.17334 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5574ad72e6a0 con 0x5574ad8062b0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc1eaff5640 1 -- 172.21.15.8:0/2973309936 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc0d3030 con 0x5574ad8062b0 2024-02-19T16:30:47.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc1eaff5640 1 -- 172.21.15.8:0/2973309936 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1f8005170 con 0x5574ad8062b0 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 10 client.17334 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 20 client.17334 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad87c5a0 0x5574ad87e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 4 client.17334 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574ad4f9890 con 0x5574ad87c5a0 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.698+0000 7fc2067fc640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad87c5a0 0x5574ad87e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc1eaff5640 10 client.17334 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:47.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc2067fc640 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad87c5a0 0x5574ad87e990 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41022c6ce79af02 server_cookie=51b6867f859779c5 in_seq=0 out_seq=0 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc1eaff5640 1 -- 172.21.15.8:0/2973309936 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5574ad4f9890 con 0x5574ad87c5a0 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc1eaff5640 10 client.17334 handle_command_reply: tid=0 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 client.17334 shutdown 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 msgr2=0x5574ad8114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad8114e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc1f801ecb0 tx=0x7fc1f801e0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 shutdown_connections 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5574ad80f7b0 0x5574ad812b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5574ad87c5a0 0x5574ad87e990 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574ad806bd0 0x5574ad812600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc1eaff5640 0 client.17334 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:47.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 --2- 172.21.15.8:0/2973309936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5574ad8062b0 0x5574ad8114e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 >> 172.21.15.8:0/2973309936 conn(0x5574ad800990 msgr2=0x5574ad7ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 shutdown_connections 2024-02-19T16:30:47.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 1 -- 172.21.15.8:0/2973309936 wait complete. 2024-02-19T16:30:47.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.699+0000 7fc210268740 20 client.17334 trim_cache size 0 max 16384 2024-02-19T16:30:47.711 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:30:47.712 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:30:47.712 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:30:47.712 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:30:47.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc1f00a2230 msgr2=0x7fc1f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc1f00a2230 0x7fc1f00a46f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc200005940 tx=0x7fc20002bc30 comp rx=0 tx=0).stop 2024-02-19T16:30:47.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 msgr2=0x7fc20819dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 0x7fc20819dc50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e9f0 tx=0x7fc1fc00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 shutdown_connections 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc208107520 0x7fc2081978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fc1f00a2230 0x7fc1f00a46f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20806a480 0x7fc20819dc50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 --2- 172.21.15.8:0/3762482750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc208069ac0 0x7fc20819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 >> 172.21.15.8:0/3762482750 conn(0x7fc2080773d0 msgr2=0x7fc2080ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 shutdown_connections 2024-02-19T16:30:47.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.700+0000 7fc20d68f640 1 -- 172.21.15.8:0/3762482750 wait complete. 2024-02-19T16:30:47.715 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:30:47.715 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:30:47.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 -- 172.21.15.8:0/358741582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1141003b0 msgr2=0x7fb11410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 --2- 172.21.15.8:0/358741582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1141003b0 0x7fb11410e340 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ca0 tx=0x7fb10402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 -- 172.21.15.8:0/358741582 shutdown_connections 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 --2- 172.21.15.8:0/358741582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1141003b0 0x7fb11410e340 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 --2- 172.21.15.8:0/358741582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1140ff9f0 0x7fb1140ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 --2- 172.21.15.8:0/358741582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb114107580 0x7fb114107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 -- 172.21.15.8:0/358741582 >> 172.21.15.8:0/358741582 conn(0x7fb114076980 msgr2=0x7fb1140fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 -- 172.21.15.8:0/358741582 shutdown_connections 2024-02-19T16:30:47.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 -- 172.21.15.8:0/358741582 wait complete. 2024-02-19T16:30:47.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.816+0000 7fb1196ff640 1 Processor -- start 2024-02-19T16:30:47.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 -- start start 2024-02-19T16:30:47.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1140ff9f0 0x7fb11419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1141003b0 0x7fb11419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 0x7fb114197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114197ee0 con 0x7fb114107580 2024-02-19T16:30:47.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114198050 con 0x7fb1141003b0 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1196ff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114198210 con 0x7fb1140ff9f0 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 0x7fb114197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 0x7fb114197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39416/0 (socket says 172.21.15.8:39416) 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 -- 172.21.15.8:0/157606880 learned_addr learned my addr 172.21.15.8:0/157606880 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb112ffd640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1140ff9f0 0x7fb11419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1127fc640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1141003b0 0x7fb11419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 -- 172.21.15.8:0/157606880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1140ff9f0 msgr2=0x7fb11419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1140ff9f0 0x7fb11419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 -- 172.21.15.8:0/157606880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1141003b0 msgr2=0x7fb11419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1141003b0 0x7fb11419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 -- 172.21.15.8:0/157606880 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009900 con 0x7fb114107580 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb1137fe640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 0x7fb114197840 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb104005e30 tx=0x7fb104035aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.817+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104035ea0 con 0x7fb114107580 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.818+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb114198490 con 0x7fb114107580 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.818+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb104003e50 con 0x7fb114107580 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.818+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004830 con 0x7fb114107580 2024-02-19T16:30:47.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.818+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1141a4700 con 0x7fb114107580 2024-02-19T16:30:47.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.818+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb114102d10 con 0x7fb114107580 2024-02-19T16:30:47.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.819+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb10403b040 con 0x7fb114107580 2024-02-19T16:30:47.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.820+0000 7fb0fbfff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb0fc0a2230 0x7fb0fc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:47.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.820+0000 7fb112ffd640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb0fc0a2230 0x7fb0fc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:47.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.820+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb1040f0cc0 con 0x7fb114107580 2024-02-19T16:30:47.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.821+0000 7fb112ffd640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb0fc0a2230 0x7fb0fc0a46f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009bd0 tx=0x7fb10c005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:47.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:47.825+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb1040b9660 con 0x7fb114107580 2024-02-19T16:30:48.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.051+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1141098d0 con 0x7fb114107580 2024-02-19T16:30:48.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.051+0000 7fb0fbfff640 1 -- 172.21.15.8:0/157606880 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fb1040bcd20 con 0x7fb114107580 2024-02-19T16:30:48.053 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:30:48.053 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:30:48.053 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:30:48.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb0fc0a2230 msgr2=0x7fb0fc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb0fc0a2230 0x7fb0fc0a46f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009bd0 tx=0x7fb10c005f90 comp rx=0 tx=0).stop 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 msgr2=0x7fb114197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 0x7fb114197840 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb104005e30 tx=0x7fb104035aa0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 shutdown_connections 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb114107580 0x7fb114197840 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1141003b0 0x7fb11419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb0fc0a2230 0x7fb0fc0a46f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 --2- 172.21.15.8:0/157606880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1140ff9f0 0x7fb11419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 >> 172.21.15.8:0/157606880 conn(0x7fb114076980 msgr2=0x7fb1140fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 shutdown_connections 2024-02-19T16:30:48.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.054+0000 7fb1196ff640 1 -- 172.21.15.8:0/157606880 wait complete. 2024-02-19T16:30:48.066 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:30:48.066 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:30:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 -- 172.21.15.8:0/1289302193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f773010bad0 msgr2=0x7f773010df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 --2- 172.21.15.8:0/1289302193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f773010bad0 0x7f773010df00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7728009ca0 tx=0x7f772802c180 comp rx=0 tx=0).stop 2024-02-19T16:30:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 -- 172.21.15.8:0/1289302193 shutdown_connections 2024-02-19T16:30:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 --2- 172.21.15.8:0/1289302193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f773010bad0 0x7f773010df00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 --2- 172.21.15.8:0/1289302193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7730103020 0x7f773010b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 --2- 172.21.15.8:0/1289302193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7730102700 0x7f7730102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 -- 172.21.15.8:0/1289302193 >> 172.21.15.8:0/1289302193 conn(0x7f7730076770 msgr2=0x7f7730076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 -- 172.21.15.8:0/1289302193 shutdown_connections 2024-02-19T16:30:48.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 -- 172.21.15.8:0/1289302193 wait complete. 2024-02-19T16:30:48.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.169+0000 7f773651e640 1 Processor -- start 2024-02-19T16:30:48.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 -- start start 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 0x7f7730100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7730103020 0x7f7730100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773010bad0 0x7f7730100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77301a5e90 con 0x7f7730102700 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77301a6000 con 0x7f773010bad0 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f773651e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77301a61c0 con 0x7f7730103020 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 0x7f7730100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 0x7f7730100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39432/0 (socket says 172.21.15.8:39432) 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 -- 172.21.15.8:0/842832997 learned_addr learned my addr 172.21.15.8:0/842832997 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772f7fe640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7730103020 0x7f7730100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f7734abb640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773010bad0 0x7f7730100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 -- 172.21.15.8:0/842832997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7730103020 msgr2=0x7f7730100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7730103020 0x7f7730100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 -- 172.21.15.8:0/842832997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773010bad0 msgr2=0x7f7730100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773010bad0 0x7f7730100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 -- 172.21.15.8:0/842832997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718009490 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.170+0000 7f772ffff640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 0x7f7730100350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f772000e9f0 tx=0x7f772000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.171+0000 7f772d7fa640 1 -- 172.21.15.8:0/842832997 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720009700 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.171+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7728009900 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.171+0000 7f772d7fa640 1 -- 172.21.15.8:0/842832997 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f772000cd80 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.171+0000 7f772d7fa640 1 -- 172.21.15.8:0/842832997 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720018920 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.171+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77301a6820 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f772d7fa640 1 -- 172.21.15.8:0/842832997 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f7720018a80 con 0x7f7730102700 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 Processor -- start 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 -- start start 2024-02-19T16:30:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fab6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fabf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fabf750 0x55e02fac1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f772d7fa640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f771c0a2230 0x7f771c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fa71960 con 0x55e02fab6b70 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fa25310 con 0x55e02fabf750 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f77391d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fa22840 con 0x55e02fab6250 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f772ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fabf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f7734abb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fab6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f772ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fabf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39434/0 (socket says 172.21.15.8:39434) 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f772ffff640 1 -- 172.21.15.8:0/2574160906 learned_addr learned my addr 172.21.15.8:0/2574160906 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.173+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fabf750 0x55e02fac1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f7734abb640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f771c0a2230 0x7f771c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772d7fa640 1 -- 172.21.15.8:0/842832997 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7720014070 con 0x7f7730102700 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772ffff640 1 -- 172.21.15.8:0/2574160906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 msgr2=0x55e02fab6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772ffff640 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fab6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772ffff640 1 -- 172.21.15.8:0/2574160906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fabf750 msgr2=0x55e02fac1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772ffff640 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fabf750 0x55e02fac1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772ffff640 1 -- 172.21.15.8:0/2574160906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02f9e0060 con 0x55e02fab6b70 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fabf750 0x55e02fac1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f772ffff640 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fabf210 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f772009ae60 tx=0x7f772006e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61abefe537b3fa5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f7734abb640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f771c0a2230 0x7f771c0a46f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7728005cc0 tx=0x7f7728005c50 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f77267fc640 1 -- 172.21.15.8:0/2574160906 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772009ef70 con 0x55e02fab6b70 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f77267fc640 1 -- 172.21.15.8:0/2574160906 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77200a0210 con 0x55e02fab6b70 2024-02-19T16:30:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.174+0000 7f77267fc640 1 -- 172.21.15.8:0/2574160906 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200cbe00 con 0x55e02fab6b70 2024-02-19T16:30:48.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 -- 172.21.15.8:0/2574160906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 msgr2=0x55e02fabf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fabf210 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f772009ae60 tx=0x7f772006e000 comp rx=0 tx=0).stop 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 -- 172.21.15.8:0/2574160906 shutdown_connections 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fabf750 0x55e02fac1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fabf210 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 --2- 172.21.15.8:0/2574160906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fab6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 -- 172.21.15.8:0/2574160906 >> 172.21.15.8:0/2574160906 conn(0x55e02fab0980 msgr2=0x55e02fab0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 -- 172.21.15.8:0/2574160906 shutdown_connections 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 -- 172.21.15.8:0/2574160906 wait complete. 2024-02-19T16:30:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 -- start start 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.175+0000 7f77391d2740 1 Processor -- start 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02fabcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fa1e020 con 0x55e02fabf750 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f6f1430 con 0x55e02fab6b70 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f6ade00 con 0x55e02fab6250 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 10 client.0 _ll_register_callbacks cb 0x55e02f9c5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f772ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f772f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02fabcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39442/0 (socket says 172.21.15.8:39442) 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 -- 172.21.15.8:0/746085625 learned_addr learned my addr 172.21.15.8:0/746085625 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 -- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 msgr2=0x55e02fabc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 -- 172.21.15.8:0/746085625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 msgr2=0x55e02fabcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02fabcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 -- 172.21.15.8:0/746085625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77301a6820 con 0x55e02fabf750 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f772f7fe640 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02fabcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f7734abb640 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd250 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7728009ca0 tx=0x7f772802c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aa83c14eb779ee server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f770f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f770f7fe640 1 -- 172.21.15.8:0/746085625 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728032990 con 0x55e02fabf750 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f770f7fe640 1 -- 172.21.15.8:0/746085625 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7728032af0 con 0x55e02fabf750 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.176+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e02f71d7e0 con 0x55e02fabf750 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f770f7fe640 1 -- 172.21.15.8:0/746085625 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7728032cb0 con 0x55e02fabf750 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 10 client.17352 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e02fa1c6a0 con 0x55e02fabf750 2024-02-19T16:30:48.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f770f7fe640 1 -- 172.21.15.8:0/746085625 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e02f71d7e0 con 0x55e02fabf750 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f770f7fe640 1 -- 172.21.15.8:0/746085625 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77280050c0 con 0x55e02fabf750 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 10 client.17352 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 20 client.17352 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02facfc80 0x55e02fad20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 4 client.17352 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e02f75a720 con 0x55e02facfc80 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f772ffff640 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02facfc80 0x55e02fad20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f772ffff640 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02facfc80 0x55e02fad20a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bce7015a507a829e server_cookie=6520368bb73ad6cc in_seq=0 out_seq=0 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.177+0000 7f770f7fe640 10 client.17352 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f770f7fe640 1 -- 172.21.15.8:0/746085625 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e02f75a720 con 0x55e02facfc80 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f770f7fe640 10 client.17352 handle_command_reply: tid=0 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 client.17352 shutdown 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 msgr2=0x55e02fabd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd250 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7728009ca0 tx=0x7f772802c180 comp rx=0 tx=0).stop 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 shutdown_connections 2024-02-19T16:30:48.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02facfc80 0x55e02fad20a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd250 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02fabcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f770f7fe640 0 client.17352 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 --2- 172.21.15.8:0/746085625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 >> 172.21.15.8:0/746085625 conn(0x55e02fab0980 msgr2=0x55e02fab26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 shutdown_connections 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 1 -- 172.21.15.8:0/746085625 wait complete. 2024-02-19T16:30:48.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.178+0000 7f77391d2740 20 client.17352 trim_cache size 0 max 16384 2024-02-19T16:30:48.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 Processor -- start 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 -- start start 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02faaba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f9a3130 con 0x55e02fabf750 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02facc650 con 0x55e02fab6b70 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f77391d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02faabf50 con 0x55e02fab6250 2024-02-19T16:30:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39454/0 (socket says 172.21.15.8:39454) 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 -- 172.21.15.8:0/512490932 learned_addr learned my addr 172.21.15.8:0/512490932 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f772ffff640 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f772f7fe640 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02faaba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 -- 172.21.15.8:0/512490932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 msgr2=0x55e02fabef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 -- 172.21.15.8:0/512490932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 msgr2=0x55e02faaba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02faaba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 -- 172.21.15.8:0/512490932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02f9b8c20 con 0x55e02fabf750 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.191+0000 7f7734abb640 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd070 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f772802c6c0 tx=0x7f772802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8187bfd068673288 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f7725ffb640 1 -- 172.21.15.8:0/512490932 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728003bf0 con 0x55e02fabf750 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f7725ffb640 1 -- 172.21.15.8:0/512490932 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7728005000 con 0x55e02fabf750 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f7725ffb640 1 -- 172.21.15.8:0/512490932 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728032d80 con 0x55e02fabf750 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 -- 172.21.15.8:0/512490932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 msgr2=0x55e02fabd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd070 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f772802c6c0 tx=0x7f772802c920 comp rx=0 tx=0).stop 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 -- 172.21.15.8:0/512490932 shutdown_connections 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fabf750 0x55e02fabd070 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6b70 0x55e02faaba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 --2- 172.21.15.8:0/512490932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fab6250 0x55e02fabef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 -- 172.21.15.8:0/512490932 >> 172.21.15.8:0/512490932 conn(0x55e02fab0980 msgr2=0x55e02faafb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 -- 172.21.15.8:0/512490932 shutdown_connections 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 -- 172.21.15.8:0/512490932 wait complete. 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.192+0000 7f77391d2740 1 -- start start 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 Processor -- start 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6250 0x55e02fab9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fab9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fabf750 0x55e02fb24500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fb24a40 con 0x55e02fab6b70 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fb24b80 con 0x55e02fab6250 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02fb24d40 con 0x55e02fabf750 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f77391d2740 10 client.0 _ll_register_callbacks cb 0x7f7718005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fab9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fab9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39466/0 (socket says 172.21.15.8:39466) 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 -- 172.21.15.8:0/2614386012 learned_addr learned my addr 172.21.15.8:0/2614386012 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f7734abb640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fabf750 0x55e02fb24500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772ffff640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6250 0x55e02fab9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 -- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fabf750 msgr2=0x55e02fb24500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fabf750 0x55e02fb24500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 -- 172.21.15.8:0/2614386012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6250 msgr2=0x55e02fab9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6250 0x55e02fab9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 -- 172.21.15.8:0/2614386012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e02fa1c6a0 con 0x55e02fab6b70 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fab9c50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x55e02fab1b10 tx=0x7f771800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cec512d09db4664 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.193+0000 7f770f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f770f7fe640 1 -- 172.21.15.8:0/2614386012 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718017040 con 0x55e02fab6b70 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f770f7fe640 1 -- 172.21.15.8:0/2614386012 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7718004250 con 0x55e02fab6b70 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e02faac0f0 con 0x55e02fab6b70 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f770f7fe640 1 -- 172.21.15.8:0/2614386012 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7718010ea0 con 0x55e02fab6b70 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 10 client.17358 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e02f9de8c0 con 0x55e02fab6b70 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f770f7fe640 1 -- 172.21.15.8:0/2614386012 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e02faac0f0 con 0x55e02fab6b70 2024-02-19T16:30:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f770f7fe640 1 -- 172.21.15.8:0/2614386012 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f771801c480 con 0x55e02fab6b70 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 10 client.17358 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 10 client.17358 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 20 client.17358 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02fb2c6c0 0x55e02fb2eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 4 client.17358 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f77200a0210 con 0x55e02fb2c6c0 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.194+0000 7f772ffff640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02fb2c6c0 0x55e02fb2eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f772ffff640 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02fb2c6c0 0x55e02fb2eae0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25bb4e73414a303e server_cookie=28e0ebe469bacac8 in_seq=0 out_seq=0 2024-02-19T16:30:48.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f770f7fe640 10 client.17358 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f770f7fe640 1 -- 172.21.15.8:0/2614386012 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f77200a0210 con 0x55e02fb2c6c0 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f770f7fe640 10 client.17358 handle_command_reply: tid=0 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 client.17358 shutdown 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 msgr2=0x55e02fab9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fab9c50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x55e02fab1b10 tx=0x7f771800f790 comp rx=0 tx=0).stop 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 shutdown_connections 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55e02fb2c6c0 0x55e02fb2eae0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e02fabf750 0x55e02fb24500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e02fab6b70 0x55e02fab9c50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f770f7fe640 0 client.17358 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 --2- 172.21.15.8:0/2614386012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e02fab6250 0x55e02fab9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 >> 172.21.15.8:0/2614386012 conn(0x55e02fab0980 msgr2=0x55e02fab1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 shutdown_connections 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 1 -- 172.21.15.8:0/2614386012 wait complete. 2024-02-19T16:30:48.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.195+0000 7f77391d2740 20 client.17358 trim_cache size 0 max 16384 2024-02-19T16:30:48.209 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:30:48.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f771c0a2230 msgr2=0x7f771c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f771c0a2230 0x7f771c0a46f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7728005cc0 tx=0x7f7728005c50 comp rx=0 tx=0).stop 2024-02-19T16:30:48.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 msgr2=0x7f7730100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 0x7f7730100350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f772000e9f0 tx=0x7f772000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 shutdown_connections 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773010bad0 0x7f7730100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f771c0a2230 0x7f771c0a46f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7730103020 0x7f7730100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 --2- 172.21.15.8:0/842832997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7730102700 0x7f7730100350 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 >> 172.21.15.8:0/842832997 conn(0x7f7730076770 msgr2=0x7f77300fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 shutdown_connections 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.196+0000 7f773651e640 1 -- 172.21.15.8:0/842832997 wait complete. 2024-02-19T16:30:48.211 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:30:48.212 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:30:48.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 -- 172.21.15.8:0/2148104000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254103080 msgr2=0x7f925410b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 --2- 172.21.15.8:0/2148104000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254103080 0x7f925410b560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9248009860 tx=0x7f924802bc00 comp rx=0 tx=0).stop 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 -- 172.21.15.8:0/2148104000 shutdown_connections 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 --2- 172.21.15.8:0/2148104000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 --2- 172.21.15.8:0/2148104000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254103080 0x7f925410b560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 --2- 172.21.15.8:0/2148104000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254102760 0x7f9254102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 -- 172.21.15.8:0/2148104000 >> 172.21.15.8:0/2148104000 conn(0x7f9254077550 msgr2=0x7f9254077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 -- 172.21.15.8:0/2148104000 shutdown_connections 2024-02-19T16:30:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 -- 172.21.15.8:0/2148104000 wait complete. 2024-02-19T16:30:48.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 Processor -- start 2024-02-19T16:30:48.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.315+0000 7f9258b8c640 1 -- start start 2024-02-19T16:30:48.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254102760 0x7f9254199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254103080 0x7f925419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925419ec30 con 0x7f925410bb30 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925419eda0 con 0x7f9254103080 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925419ef60 con 0x7f9254102760 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39490/0 (socket says 172.21.15.8:39490) 2024-02-19T16:30:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 -- 172.21.15.8:0/52956290 learned_addr learned my addr 172.21.15.8:0/52956290 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f925259c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254102760 0x7f9254199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9251d9b640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254103080 0x7f925419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 -- 172.21.15.8:0/52956290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254102760 msgr2=0x7f9254199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254102760 0x7f9254199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 -- 172.21.15.8:0/52956290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254103080 msgr2=0x7f925419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254103080 0x7f925419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 -- 172.21.15.8:0/52956290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009900 con 0x7f925410bb30 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9252d9d640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925419e630 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f924c00efc0 tx=0x7f924c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f92437fe640 1 -- 172.21.15.8:0/52956290 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c003970 con 0x7f925410bb30 2024-02-19T16:30:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9248009490 con 0x7f925410bb30 2024-02-19T16:30:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f92437fe640 1 -- 172.21.15.8:0/52956290 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f924c004020 con 0x7f925410bb30 2024-02-19T16:30:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f92437fe640 1 -- 172.21.15.8:0/52956290 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00b9b0 con 0x7f925410bb30 2024-02-19T16:30:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.316+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92541a68e0 con 0x7f925410bb30 2024-02-19T16:30:48.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f92437fe640 1 -- 172.21.15.8:0/52956290 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f924c00a040 con 0x7f925410bb30 2024-02-19T16:30:48.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 Processor -- start 2024-02-19T16:30:48.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 -- start start 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216250 0x55a19c216630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c21f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c21f750 0x55a19c221b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c1d1960 con 0x55a19c216b70 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c185310 con 0x55a19c21f750 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925b76c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c182840 con 0x55a19c216250 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f92437fe640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f923c0a2230 0x7f923c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c21f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c21f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39492/0 (socket says 172.21.15.8:39492) 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f925259c640 1 -- 172.21.15.8:0/394797489 learned_addr learned my addr 172.21.15.8:0/394797489 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216250 0x55a19c216630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f9251d9b640 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c21f750 0x55a19c221b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.319+0000 7f9252d9d640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f923c0a2230 0x7f923c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f92437fe640 1 -- 172.21.15.8:0/52956290 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f924c0ce450 con 0x7f925410bb30 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f925259c640 1 -- 172.21.15.8:0/394797489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216250 msgr2=0x55a19c216630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f925259c640 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216250 0x55a19c216630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f925259c640 1 -- 172.21.15.8:0/394797489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c21f750 msgr2=0x55a19c221b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f925259c640 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c21f750 0x55a19c221b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f925259c640 1 -- 172.21.15.8:0/394797489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a19c140060 con 0x55a19c216b70 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f9251d9b640 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216250 0x55a19c216630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f925259c640 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c21f210 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9244009cf0 tx=0x7f9244011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73617b433c42ea9c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f9252d9d640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f923c0a2230 0x7f923c0a46f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f924c09a740 tx=0x7f924c069be0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f9240ff9640 1 -- 172.21.15.8:0/394797489 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244009ea0 con 0x55a19c216b70 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f9240ff9640 1 -- 172.21.15.8:0/394797489 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9244004230 con 0x55a19c216b70 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.320+0000 7f9240ff9640 1 -- 172.21.15.8:0/394797489 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244019800 con 0x55a19c216b70 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 -- 172.21.15.8:0/394797489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 msgr2=0x55a19c21f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c21f210 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9244009cf0 tx=0x7f9244011810 comp rx=0 tx=0).stop 2024-02-19T16:30:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 -- 172.21.15.8:0/394797489 shutdown_connections 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c21f750 0x55a19c221b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c21f210 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 --2- 172.21.15.8:0/394797489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216250 0x55a19c216630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 -- 172.21.15.8:0/394797489 >> 172.21.15.8:0/394797489 conn(0x55a19c210980 msgr2=0x55a19c210d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 -- 172.21.15.8:0/394797489 shutdown_connections 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 -- 172.21.15.8:0/394797489 wait complete. 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 -- start start 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.321+0000 7f925b76c740 1 Processor -- start 2024-02-19T16:30:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216b70 0x55a19c21ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 0x55a19c21d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c17e020 con 0x55a19c21f750 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19be51430 con 0x55a19c216250 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19be0de00 con 0x55a19c216b70 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 10 client.0 _ll_register_callbacks cb 0x55a19c125cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 0x55a19c21d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 0x55a19c21d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39494/0 (socket says 172.21.15.8:39494) 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 -- 172.21.15.8:0/2886224800 learned_addr learned my addr 172.21.15.8:0/2886224800 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925259c640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216b70 0x55a19c21ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 -- 172.21.15.8:0/2886224800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216b70 msgr2=0x55a19c21ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216b70 0x55a19c21ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 -- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 msgr2=0x55a19c21c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 -- 172.21.15.8:0/2886224800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9244009950 con 0x55a19c21f750 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9251d9b640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 0x55a19c21d230 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f924802c140 tx=0x7f9248005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde35e193ce30fd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:48.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9239ffb640 1 -- 172.21.15.8:0/2886224800 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924802cbe0 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9239ffb640 1 -- 172.21.15.8:0/2886224800 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f92480035f0 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a19be7d7e0 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f9239ffb640 1 -- 172.21.15.8:0/2886224800 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9248034a60 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 10 client.17370 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.322+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a19c17c6a0 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f9239ffb640 1 -- 172.21.15.8:0/2886224800 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a19be7d7e0 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f9239ffb640 1 -- 172.21.15.8:0/2886224800 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f924804d030 con 0x55a19c21f750 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925b76c740 10 client.17370 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925b76c740 10 client.17370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925b76c740 20 client.17370 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925b76c740 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c22fb30 0x55a19c231f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925b76c740 4 client.17370 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a19beba720 con 0x55a19c22fb30 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925259c640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c22fb30 0x55a19c231f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f9239ffb640 10 client.17370 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.323+0000 7f925259c640 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c22fb30 0x55a19c231f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8efa568f6ee4c6c server_cookie=1cdb86b52d643038 in_seq=0 out_seq=0 2024-02-19T16:30:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f9239ffb640 1 -- 172.21.15.8:0/2886224800 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a19beba720 con 0x55a19c22fb30 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f9239ffb640 10 client.17370 handle_command_reply: tid=0 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 client.17370 shutdown 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 msgr2=0x55a19c21d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 0x55a19c21d230 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f924802c140 tx=0x7f9248005f90 comp rx=0 tx=0).stop 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 shutdown_connections 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c22fb30 0x55a19c231f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c21f750 0x55a19c21d230 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c216b70 0x55a19c21ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f9239ffb640 0 client.17370 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 --2- 172.21.15.8:0/2886224800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 >> 172.21.15.8:0/2886224800 conn(0x55a19c210980 msgr2=0x55a19c2126d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 shutdown_connections 2024-02-19T16:30:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 1 -- 172.21.15.8:0/2886224800 wait complete. 2024-02-19T16:30:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.324+0000 7f925b76c740 20 client.17370 trim_cache size 0 max 16384 2024-02-19T16:30:48.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 Processor -- start 2024-02-19T16:30:48.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 -- start start 2024-02-19T16:30:48.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c20fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c22aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c103130 con 0x55a19c216b70 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c211780 con 0x55a19c216250 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925b76c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c22c690 con 0x55a19c21f750 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c20fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c20fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39508/0 (socket says 172.21.15.8:39508) 2024-02-19T16:30:48.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 -- 172.21.15.8:0/1378809246 learned_addr learned my addr 172.21.15.8:0/1378809246 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f9251d9b640 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c22aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f9251d9b640 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 -- 172.21.15.8:0/1378809246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 msgr2=0x55a19c22aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c22aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 -- 172.21.15.8:0/1378809246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 msgr2=0x55a19c21eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 -- 172.21.15.8:0/1378809246 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a19c118c20 con 0x55a19c216b70 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f9251d9b640 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:48.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.337+0000 7f925259c640 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c20fa00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9244004070 tx=0x7f9244003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b13ce0cb647f0f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f923bfff640 1 -- 172.21.15.8:0/1378809246 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244005610 con 0x55a19c216b70 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f923bfff640 1 -- 172.21.15.8:0/1378809246 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92440097a0 con 0x55a19c216b70 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f923bfff640 1 -- 172.21.15.8:0/1378809246 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924402b9b0 con 0x55a19c216b70 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 -- 172.21.15.8:0/1378809246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 msgr2=0x55a19c20fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c20fa00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9244004070 tx=0x7f9244003f10 comp rx=0 tx=0).stop 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 -- 172.21.15.8:0/1378809246 shutdown_connections 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c22aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216b70 0x55a19c20fa00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 --2- 172.21.15.8:0/1378809246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216250 0x55a19c21eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 -- 172.21.15.8:0/1378809246 >> 172.21.15.8:0/1378809246 conn(0x55a19c210980 msgr2=0x55a19c20f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 -- 172.21.15.8:0/1378809246 shutdown_connections 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 -- 172.21.15.8:0/1378809246 wait complete. 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 -- start start 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.338+0000 7f925b76c740 1 Processor -- start 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 0x55a19c2231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216b70 0x55a19c2236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c28a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c28a8f0 con 0x55a19c216250 2024-02-19T16:30:48.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c28aa30 con 0x55a19c216b70 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a19c28ab70 con 0x55a19c21f750 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 10 client.0 _ll_register_callbacks cb 0x7f9244005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 0x55a19c2231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 0x55a19c2231a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39510/0 (socket says 172.21.15.8:39510) 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 -- 172.21.15.8:0/505176206 learned_addr learned my addr 172.21.15.8:0/505176206 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925259c640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216b70 0x55a19c2236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c28a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 -- 172.21.15.8:0/505176206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 msgr2=0x55a19c28a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c28a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 -- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216b70 msgr2=0x55a19c2236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216b70 0x55a19c2236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 -- 172.21.15.8:0/505176206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a19c140060 con 0x55a19c216250 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925259c640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216b70 0x55a19c2236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9251d9b640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 0x55a19c2231a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9248009250 tx=0x7f924802c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bcf4fb45fd943a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9239ffb640 1 -- 172.21.15.8:0/505176206 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924802cbe0 con 0x55a19c216250 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f9239ffb640 1 -- 172.21.15.8:0/505176206 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f9248042b80 con 0x55a19c216250 2024-02-19T16:30:48.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.339+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9244004230 con 0x55a19c216250 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f9239ffb640 1 -- 172.21.15.8:0/505176206 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9248042d40 con 0x55a19c216250 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 10 client.17376 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a19c13e8c0 con 0x55a19c216250 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f9239ffb640 1 -- 172.21.15.8:0/505176206 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9244004230 con 0x55a19c216250 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f9239ffb640 1 -- 172.21.15.8:0/505176206 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f924804d050 con 0x55a19c216250 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 10 client.17376 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 10 client.17376 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 20 client.17376 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c28c040 0x55a19c28e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 4 client.17376 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a19bf09890 con 0x55a19c28c040 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925259c640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c28c040 0x55a19c28e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f9239ffb640 10 client.17376 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.340+0000 7f925259c640 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c28c040 0x55a19c28e460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b36a5871480efb5 server_cookie=d67f861a427fbfac in_seq=0 out_seq=0 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9239ffb640 1 -- 172.21.15.8:0/505176206 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a19bf09890 con 0x55a19c28c040 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9239ffb640 10 client.17376 handle_command_reply: tid=0 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 client.17376 shutdown 2024-02-19T16:30:48.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 msgr2=0x55a19c2231a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 0x55a19c2231a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9248009250 tx=0x7f924802c380 comp rx=0 tx=0).stop 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 shutdown_connections 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55a19c28c040 0x55a19c28e460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a19c21f750 0x55a19c28a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9239ffb640 0 client.17376 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a19c216b70 0x55a19c2236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 --2- 172.21.15.8:0/505176206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a19c216250 0x55a19c2231a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 >> 172.21.15.8:0/505176206 conn(0x55a19c210980 msgr2=0x55a19c21cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 shutdown_connections 2024-02-19T16:30:48.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 1 -- 172.21.15.8:0/505176206 wait complete. 2024-02-19T16:30:48.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f925b76c740 20 client.17376 trim_cache size 0 max 16384 2024-02-19T16:30:48.354 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:30:48.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f923c0a2230 msgr2=0x7f923c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f923c0a2230 0x7f923c0a46f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f924c09a740 tx=0x7f924c069be0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 msgr2=0x7f925419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925419e630 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f924c00efc0 tx=0x7f924c00c800 comp rx=0 tx=0).stop 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 shutdown_connections 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f925410bb30 0x7f925419e630 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254103080 0x7f925419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f923c0a2230 0x7f923c0a46f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 --2- 172.21.15.8:0/52956290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9254102760 0x7f9254199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 >> 172.21.15.8:0/52956290 conn(0x7f9254077550 msgr2=0x7f92541079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 shutdown_connections 2024-02-19T16:30:48.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.341+0000 7f9258b8c640 1 -- 172.21.15.8:0/52956290 wait complete. 2024-02-19T16:30:48.357 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:30:48.357 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:30:48.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 -- 172.21.15.8:0/4080094141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70101e60 msgr2=0x7f0c7010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 --2- 172.21.15.8:0/4080094141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70101e60 0x7f0c7010e320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009ca0 tx=0x7f0c6802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 -- 172.21.15.8:0/4080094141 shutdown_connections 2024-02-19T16:30:48.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 --2- 172.21.15.8:0/4080094141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70101e60 0x7f0c7010e320 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 --2- 172.21.15.8:0/4080094141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 0x7f0c70101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 --2- 172.21.15.8:0/4080094141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70107560 0x7f0c70107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 -- 172.21.15.8:0/4080094141 >> 172.21.15.8:0/4080094141 conn(0x7f0c700788f0 msgr2=0x7f0c70078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 -- 172.21.15.8:0/4080094141 shutdown_connections 2024-02-19T16:30:48.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.460+0000 7f0c75c46640 1 -- 172.21.15.8:0/4080094141 wait complete. 2024-02-19T16:30:48.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 Processor -- start 2024-02-19T16:30:48.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 -- start start 2024-02-19T16:30:48.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 0x7f0c7019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70101e60 0x7f0c7019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 0x7f0c70197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70197ee0 con 0x7f0c70107560 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70198050 con 0x7f0c701014a0 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c75c46640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70198210 con 0x7f0c70101e60 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 0x7f0c70197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 0x7f0c7019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70101e60 0x7f0c7019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 0x7f0c70197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39524/0 (socket says 172.21.15.8:39524) 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 -- 172.21.15.8:0/2091543943 learned_addr learned my addr 172.21.15.8:0/2091543943 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 -- 172.21.15.8:0/2091543943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70101e60 msgr2=0x7f0c7019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70101e60 0x7f0c7019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 -- 172.21.15.8:0/2091543943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 msgr2=0x7f0c7019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 0x7f0c7019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6ffff640 1 -- 172.21.15.8:0/2091543943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64009490 con 0x7f0c70107560 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 0x7f0c7019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.461+0000 7f0c6effd640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70101e60 0x7f0c7019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.462+0000 7f0c6ffff640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 0x7f0c70197840 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005e30 tx=0x7f0c68035aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.462+0000 7f0c6cff9640 1 -- 172.21.15.8:0/2091543943 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68035ea0 con 0x7f0c70107560 2024-02-19T16:30:48.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.462+0000 7f0c6cff9640 1 -- 172.21.15.8:0/2091543943 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c68003e50 con 0x7f0c70107560 2024-02-19T16:30:48.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.462+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c60009490 con 0x7f0c70107560 2024-02-19T16:30:48.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.462+0000 7f0c6cff9640 1 -- 172.21.15.8:0/2091543943 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68004830 con 0x7f0c70107560 2024-02-19T16:30:48.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.462+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c68009900 con 0x7f0c70107560 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 Processor -- start 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6cff9640 1 -- 172.21.15.8:0/2091543943 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f0c6803b040 con 0x7f0c70107560 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 -- start start 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e56ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 0x55a81e577620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e579f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e525960 con 0x55a81e56ef80 2024-02-19T16:30:48.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e4d9310 con 0x55a81e577b60 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c7881a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e4d6840 con 0x55a81e56e660 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 0x55a81e577620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e56ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 0x55a81e577620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39532/0 (socket says 172.21.15.8:39532) 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3161399186 learned_addr learned my addr 172.21.15.8:0/3161399186 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e579f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6cff9640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0c580a2230 0x7f0c580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3161399186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 msgr2=0x55a81e56ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e56ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.465+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3161399186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 msgr2=0x55a81e579f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e579f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3161399186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81e494060 con 0x55a81e56ef80 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c6ffff640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0c580a2230 0x7f0c580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c6cff9640 1 -- 172.21.15.8:0/2091543943 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f0c680f0c80 con 0x7f0c70107560 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 0x55a81e577620 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0c60011750 tx=0x7f0c60011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1775992bf588ea9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c55ffb640 1 -- 172.21.15.8:0/3161399186 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000f9a0 con 0x55a81e56ef80 2024-02-19T16:30:48.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c55ffb640 1 -- 172.21.15.8:0/3161399186 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c60019430 con 0x55a81e56ef80 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c55ffb640 1 -- 172.21.15.8:0/3161399186 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000e5b0 con 0x55a81e56ef80 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c7881a740 1 -- 172.21.15.8:0/3161399186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 msgr2=0x55a81e577620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c6ffff640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0c580a2230 0x7f0c580a46f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c680b9d50 tx=0x7f0c68097040 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.466+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 0x55a81e577620 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0c60011750 tx=0x7f0c60011c10 comp rx=0 tx=0).stop 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- 172.21.15.8:0/3161399186 shutdown_connections 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e579f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56ef80 0x55a81e577620 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3161399186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e56ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- 172.21.15.8:0/3161399186 >> 172.21.15.8:0/3161399186 conn(0x55a81e568d90 msgr2=0x55a81e569170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- 172.21.15.8:0/3161399186 shutdown_connections 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- 172.21.15.8:0/3161399186 wait complete. 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- start start 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 Processor -- start 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e574bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e575130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e575670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e4d2020 con 0x55a81e56e660 2024-02-19T16:30:48.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e1a5430 con 0x55a81e577b60 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e161e00 con 0x55a81e56ef80 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.467+0000 7f0c7881a740 10 client.0 _ll_register_callbacks cb 0x55a81e479cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e574bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e574bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39548/0 (socket says 172.21.15.8:39548) 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6effd640 1 -- 172.21.15.8:0/3226002526 learned_addr learned my addr 172.21.15.8:0/3226002526 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e575130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e575670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3226002526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 msgr2=0x55a81e575670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e575670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3226002526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 msgr2=0x55a81e574bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e574bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 -- 172.21.15.8:0/3226002526 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c60011440 con 0x55a81e56ef80 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e574bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e575130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000f790 tx=0x7f0c6000fbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a69e7bf9434bb12 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.468+0000 7f0c4effd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c4effd640 1 -- 172.21.15.8:0/3226002526 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60019bd0 con 0x55a81e56ef80 2024-02-19T16:30:48.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a81e1d17e0 con 0x55a81e56ef80 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c4effd640 1 -- 172.21.15.8:0/3226002526 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f0c60005040 con 0x55a81e56ef80 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c4effd640 1 -- 172.21.15.8:0/3226002526 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c60005200 con 0x55a81e56ef80 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 10 client.27247 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a81e4d06a0 con 0x55a81e56ef80 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c4effd640 1 -- 172.21.15.8:0/3226002526 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a81e1d17e0 con 0x55a81e56ef80 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c4effd640 1 -- 172.21.15.8:0/3226002526 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c60024420 con 0x55a81e56ef80 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 10 client.27247 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 20 client.27247 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:48.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e587e90 0x55a81e58a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 4 client.27247 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a81e20e720 con 0x55a81e587e90 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.469+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e587e90 0x55a81e58a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c4effd640 10 client.27247 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e587e90 0x55a81e58a2b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358b04889d81e32d server_cookie=9a67abee2ef5285e in_seq=0 out_seq=0 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c4effd640 1 -- 172.21.15.8:0/3226002526 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a81e20e720 con 0x55a81e587e90 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c4effd640 10 client.27247 handle_command_reply: tid=0 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 client.27247 shutdown 2024-02-19T16:30:48.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 msgr2=0x55a81e575130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e575130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000f790 tx=0x7f0c6000fbd0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 shutdown_connections 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e575670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e587e90 0x55a81e58a2b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e575130 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3226002526 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e574bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 >> 172.21.15.8:0/3226002526 conn(0x55a81e568d90 msgr2=0x55a81e56a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.470+0000 7f0c4effd640 0 client.27247 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:48.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.471+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 shutdown_connections 2024-02-19T16:30:48.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.471+0000 7f0c7881a740 1 -- 172.21.15.8:0/3226002526 wait complete. 2024-02-19T16:30:48.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.471+0000 7f0c7881a740 20 client.27247 trim_cache size 0 max 16384 2024-02-19T16:30:48.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 Processor -- start 2024-02-19T16:30:48.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 -- start start 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e575490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e56ef80 0x55a81e582ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 0x55a81e583420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e457130 con 0x55a81e577b60 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e583990 con 0x55a81e56ef80 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c7881a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e5605c0 con 0x55a81e56e660 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 0x55a81e583420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 0x55a81e583420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39554/0 (socket says 172.21.15.8:39554) 2024-02-19T16:30:48.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c6effd640 1 -- 172.21.15.8:0/1178990882 learned_addr learned my addr 172.21.15.8:0/1178990882 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.483+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e56ef80 0x55a81e582ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e575490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 -- 172.21.15.8:0/1178990882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 msgr2=0x55a81e575490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e575490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 -- 172.21.15.8:0/1178990882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e56ef80 msgr2=0x55a81e582ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e56ef80 0x55a81e582ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 -- 172.21.15.8:0/1178990882 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81e46cc20 con 0x55a81e577b60 2024-02-19T16:30:48.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c6effd640 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 0x55a81e583420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55a81e58ab10 tx=0x7f0c64005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1b0f31629c7e28 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c557fa640 1 -- 172.21.15.8:0/1178990882 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6402b030 con 0x55a81e577b60 2024-02-19T16:30:48.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.484+0000 7f0c557fa640 1 -- 172.21.15.8:0/1178990882 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c6402c040 con 0x55a81e577b60 2024-02-19T16:30:48.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c557fa640 1 -- 172.21.15.8:0/1178990882 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64016040 con 0x55a81e577b60 2024-02-19T16:30:48.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 -- 172.21.15.8:0/1178990882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 msgr2=0x55a81e583420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 0x55a81e583420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55a81e58ab10 tx=0x7f0c64005f90 comp rx=0 tx=0).stop 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 -- 172.21.15.8:0/1178990882 shutdown_connections 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e577b60 0x55a81e583420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e56ef80 0x55a81e582ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 --2- 172.21.15.8:0/1178990882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56e660 0x55a81e575490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 -- 172.21.15.8:0/1178990882 >> 172.21.15.8:0/1178990882 conn(0x55a81e568d90 msgr2=0x55a81e58f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 -- 172.21.15.8:0/1178990882 shutdown_connections 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.485+0000 7f0c7881a740 1 -- 172.21.15.8:0/1178990882 wait complete. 2024-02-19T16:30:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 -- start start 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 Processor -- start 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e572930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e57c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e57cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e57d5e0 con 0x55a81e56e660 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e57d750 con 0x55a81e577b60 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81e5e3770 con 0x55a81e56ef80 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c7881a740 10 client.0 _ll_register_callbacks cb 0x7f0c6001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e572930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e572930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39568/0 (socket says 172.21.15.8:39568) 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 -- 172.21.15.8:0/3752525300 learned_addr learned my addr 172.21.15.8:0/3752525300 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e57c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e57cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 -- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 msgr2=0x55a81e57c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e57c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 -- 172.21.15.8:0/3752525300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 msgr2=0x55a81e57cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e57cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.486+0000 7f0c6effd640 1 -- 172.21.15.8:0/3752525300 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c64019040 con 0x55a81e56e660 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c6effd640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e572930 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0c64004910 tx=0x7f0c640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4d56d4f7ca763a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c4effd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c4effd640 1 -- 172.21.15.8:0/3752525300 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64016030 con 0x55a81e56e660 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c60019430 con 0x55a81e56e660 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c4effd640 1 -- 172.21.15.8:0/3752525300 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f0c64000e40 con 0x55a81e56e660 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c4effd640 1 -- 172.21.15.8:0/3752525300 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c6402b070 con 0x55a81e56e660 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 10 client.17394 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a81e4928c0 con 0x55a81e56e660 2024-02-19T16:30:48.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c4effd640 1 -- 172.21.15.8:0/3752525300 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c60019430 con 0x55a81e56e660 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c4effd640 1 -- 172.21.15.8:0/3752525300 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c64022440 con 0x55a81e56e660 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 10 client.17394 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 20 client.17394 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e5e4d70 0x55a81e5e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 4 client.17394 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.487+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a81e25d890 con 0x55a81e5e4d70 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e5e4d70 0x55a81e5e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c6f7fe640 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e5e4d70 0x55a81e5e7190 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9edc0d5c869b9b5 server_cookie=2810a9d01bb73285 in_seq=0 out_seq=0 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c4effd640 10 client.17394 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c4effd640 1 -- 172.21.15.8:0/3752525300 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a81e25d890 con 0x55a81e5e4d70 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c4effd640 10 client.17394 handle_command_reply: tid=0 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 client.17394 shutdown 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 msgr2=0x55a81e572930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e572930 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0c64004910 tx=0x7f0c640057f0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 shutdown_connections 2024-02-19T16:30:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a81e577b60 0x55a81e57cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a81e5e4d70 0x55a81e5e7190 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a81e56ef80 0x55a81e57c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 --2- 172.21.15.8:0/3752525300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a81e56e660 0x55a81e572930 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c4effd640 0 client.17394 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 >> 172.21.15.8:0/3752525300 conn(0x55a81e568d90 msgr2=0x55a81e569d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 shutdown_connections 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.488+0000 7f0c7881a740 1 -- 172.21.15.8:0/3752525300 wait complete. 2024-02-19T16:30:48.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c7881a740 20 client.17394 trim_cache size 0 max 16384 2024-02-19T16:30:48.500 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:30:48.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0c580a2230 msgr2=0x7f0c580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0c580a2230 0x7f0c580a46f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c680b9d50 tx=0x7f0c68097040 comp rx=0 tx=0).stop 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 msgr2=0x7f0c70197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 0x7f0c70197840 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005e30 tx=0x7f0c68035aa0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 shutdown_connections 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c70107560 0x7f0c70197840 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f0c580a2230 0x7f0c580a46f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c70101e60 0x7f0c7019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 --2- 172.21.15.8:0/2091543943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c701014a0 0x7f0c7019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:48.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 >> 172.21.15.8:0/2091543943 conn(0x7f0c700788f0 msgr2=0x7f0c701000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:48.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 shutdown_connections 2024-02-19T16:30:48.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:48.489+0000 7f0c75c46640 1 -- 172.21.15.8:0/2091543943 wait complete. 2024-02-19T16:30:48.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:48 smithi008 ceph-mon[32573]: pgmap v244: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:48.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:48 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/157606880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:30:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:48 smithi191 ceph-mon[37203]: pgmap v244: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:48 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/157606880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:30:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:48 smithi156 ceph-mon[36502]: pgmap v244: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:48 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/157606880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:30:49.503 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:30:49.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:30:49.504 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:30:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/1341529453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 msgr2=0x7f6b541009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/1341529453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b541009f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009930 tx=0x7f6b4c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:30:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/1341529453 shutdown_connections 2024-02-19T16:30:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/1341529453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 0x7f6b5410d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/1341529453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b54100fc0 0x7f6b54105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/1341529453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b541009f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/1341529453 >> 172.21.15.8:0/1341529453 conn(0x7f6b54076770 msgr2=0x7f6b54076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/1341529453 shutdown_connections 2024-02-19T16:30:49.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.622+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/1341529453 wait complete. 2024-02-19T16:30:49.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 Processor -- start 2024-02-19T16:30:49.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 -- start start 2024-02-19T16:30:49.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b5419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 0x7f6b5419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b54197a20 0x7f6b54197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b54198410 con 0x7f6b54100610 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b54198580 con 0x7f6b541056e0 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b5b2ec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b54198740 con 0x7f6b54197a20 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b59088640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b5419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b59088640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b5419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39596/0 (socket says 172.21.15.8:39596) 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b59088640 1 -- 172.21.15.8:0/2125401448 learned_addr learned my addr 172.21.15.8:0/2125401448 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b58887640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 0x7f6b5419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.623+0000 7f6b59889640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b54197a20 0x7f6b54197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b59088640 1 -- 172.21.15.8:0/2125401448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b54197a20 msgr2=0x7f6b54197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b59088640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b54197a20 0x7f6b54197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b59088640 1 -- 172.21.15.8:0/2125401448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 msgr2=0x7f6b5419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b59088640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 0x7f6b5419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b59088640 1 -- 172.21.15.8:0/2125401448 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c009560 con 0x7f6b54100610 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b58887640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 0x7f6b5419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b59088640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b5419d870 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c000c00 tx=0x7f6b4c02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b527fc640 1 -- 172.21.15.8:0/2125401448 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0042d0 con 0x7f6b54100610 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b541a4730 con 0x7f6b54100610 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b527fc640 1 -- 172.21.15.8:0/2125401448 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c004430 con 0x7f6b54100610 2024-02-19T16:30:49.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b527fc640 1 -- 172.21.15.8:0/2125401448 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c003c50 con 0x7f6b54100610 2024-02-19T16:30:49.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.624+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b541a4be0 con 0x7f6b54100610 2024-02-19T16:30:49.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.626+0000 7f6b527fc640 1 -- 172.21.15.8:0/2125401448 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f6b4c004df0 con 0x7f6b54100610 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 Processor -- start 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 -- start start 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca23f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b527fc640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6b3c0a2230 0x7f6b3c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 0x556fca2482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca2488b0 0x556fca24ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca1adf90 con 0x556fca23fd30 2024-02-19T16:30:49.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca1ab4c0 con 0x556fca23f380 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca1a6ca0 con 0x556fca2488b0 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b59088640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6b3c0a2230 0x7f6b3c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 0x556fca2482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 0x556fca2482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39604/0 (socket says 172.21.15.8:39604) 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 -- 172.21.15.8:0/3750416134 learned_addr learned my addr 172.21.15.8:0/3750416134 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b59889640 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca23f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b59889640 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca2488b0 0x556fca24ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b527fc640 1 -- 172.21.15.8:0/2125401448 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f6b4c0f0a90 con 0x7f6b54100610 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 -- 172.21.15.8:0/3750416134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca2488b0 msgr2=0x556fca24ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca2488b0 0x556fca24ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 -- 172.21.15.8:0/3750416134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 msgr2=0x556fca23f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca23f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.627+0000 7f6b58887640 1 -- 172.21.15.8:0/3750416134 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fca1690b0 con 0x556fca23fd30 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b59088640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6b3c0a2230 0x7f6b3c0a46f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0b9590 tx=0x7f6b4c0036f0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b58887640 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 0x556fca2482e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6b48011470 tx=0x7f6b48011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31186eb3b1f4e0a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b3b7fe640 1 -- 172.21.15.8:0/3750416134 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48004310 con 0x556fca23fd30 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b3b7fe640 1 -- 172.21.15.8:0/3750416134 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b48004470 con 0x556fca23fd30 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b3b7fe640 1 -- 172.21.15.8:0/3750416134 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800f9c0 con 0x556fca23fd30 2024-02-19T16:30:49.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b5df9b740 1 -- 172.21.15.8:0/3750416134 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 msgr2=0x556fca2482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.628+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 0x556fca2482e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6b48011470 tx=0x7f6b48011930 comp rx=0 tx=0).stop 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- 172.21.15.8:0/3750416134 shutdown_connections 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca2488b0 0x556fca24ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca23fd30 0x556fca2482e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/3750416134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca23f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- 172.21.15.8:0/3750416134 >> 172.21.15.8:0/3750416134 conn(0x556fca239840 msgr2=0x556fca239c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- 172.21.15.8:0/3750416134 shutdown_connections 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- 172.21.15.8:0/3750416134 wait complete. 2024-02-19T16:30:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- start start 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 Processor -- start 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca2458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca245de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca246320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca14bf40 con 0x556fca2488b0 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc9e7a4b0 con 0x556fca23f380 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca12c670 con 0x556fca23fd30 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.629+0000 7f6b5df9b740 10 client.0 _ll_register_callbacks cb 0x556fca199f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca246320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b58887640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca245de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b58887640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca245de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:48590/0 (socket says 172.21.15.8:48590) 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca246320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39616/0 (socket says 172.21.15.8:39616) 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b58887640 1 -- 172.21.15.8:0/492325766 learned_addr learned my addr 172.21.15.8:0/492325766 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca2458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 -- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 msgr2=0x556fca245de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca245de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 -- 172.21.15.8:0/492325766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 msgr2=0x556fca2458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca2458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 -- 172.21.15.8:0/492325766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b48009c40 con 0x556fca2488b0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b58887640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca245de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b59889640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca246320 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009fd0 tx=0x7f6b44009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ce0353db445b01 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b38ff9640 1 -- 172.21.15.8:0/492325766 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44025070 con 0x556fca2488b0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b38ff9640 1 -- 172.21.15.8:0/492325766 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f6b44004b20 con 0x556fca2488b0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fca180f40 con 0x556fca2488b0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b38ff9640 1 -- 172.21.15.8:0/492325766 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b44004ce0 con 0x556fca2488b0 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b5df9b740 10 client.17406 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.630+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556fca1a5320 con 0x556fca2488b0 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b38ff9640 1 -- 172.21.15.8:0/492325766 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556fca180f40 con 0x556fca2488b0 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b38ff9640 1 -- 172.21.15.8:0/492325766 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b44028030 con 0x556fca2488b0 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b5df9b740 10 client.17406 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b5df9b740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b5df9b740 20 client.17406 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca258f50 0x556fca25b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b5df9b740 4 client.17406 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fca19d310 con 0x556fca258f50 2024-02-19T16:30:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b58887640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca258f50 0x556fca25b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b58887640 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca258f50 0x556fca25b370 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d79801291a9d2ea server_cookie=ae55b728418536d in_seq=0 out_seq=0 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.631+0000 7f6b38ff9640 10 client.17406 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b38ff9640 1 -- 172.21.15.8:0/492325766 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556fca19d310 con 0x556fca258f50 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b38ff9640 10 client.17406 handle_command_reply: tid=0 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 client.17406 shutdown 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 msgr2=0x556fca246320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca246320 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009fd0 tx=0x7f6b44009df0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 shutdown_connections 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca246320 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca258f50 0x556fca25b370 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca245de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/492325766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca2458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b38ff9640 0 client.17406 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 >> 172.21.15.8:0/492325766 conn(0x556fca239840 msgr2=0x556fca23b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 shutdown_connections 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.632+0000 7f6b5df9b740 1 -- 172.21.15.8:0/492325766 wait complete. 2024-02-19T16:30:49.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.633+0000 7f6b5df9b740 20 client.17406 trim_cache size 0 max 16384 2024-02-19T16:30:49.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 Processor -- start 2024-02-19T16:30:49.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 -- start start 2024-02-19T16:30:49.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca247fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca253de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca254320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca2406b0 con 0x556fca2488b0 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca233da0 con 0x556fca23f380 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca2414d0 con 0x556fca23fd30 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b59889640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca254320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b59889640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca254320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39620/0 (socket says 172.21.15.8:39620) 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b59889640 1 -- 172.21.15.8:0/1717790964 learned_addr learned my addr 172.21.15.8:0/1717790964 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b58887640 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca253de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.649+0000 7f6b59889640 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca247fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b59889640 1 -- 172.21.15.8:0/1717790964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 msgr2=0x556fca253de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b59889640 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca253de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b59889640 1 -- 172.21.15.8:0/1717790964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 msgr2=0x556fca247fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b59889640 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca247fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b59889640 1 -- 172.21.15.8:0/1717790964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fca141c00 con 0x556fca2488b0 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b59889640 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca254320 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400f6f0 tx=0x7f6b44014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13622774d4b8df3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b3affd640 1 -- 172.21.15.8:0/1717790964 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44025070 con 0x556fca2488b0 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b3affd640 1 -- 172.21.15.8:0/1717790964 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b44012a60 con 0x556fca2488b0 2024-02-19T16:30:49.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b3affd640 1 -- 172.21.15.8:0/1717790964 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440040c0 con 0x556fca2488b0 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b5df9b740 1 -- 172.21.15.8:0/1717790964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 msgr2=0x556fca254320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.650+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca254320 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400f6f0 tx=0x7f6b44014040 comp rx=0 tx=0).stop 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- 172.21.15.8:0/1717790964 shutdown_connections 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca254320 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca253de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/1717790964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca247fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- 172.21.15.8:0/1717790964 >> 172.21.15.8:0/1717790964 conn(0x556fca239840 msgr2=0x556fca244170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- 172.21.15.8:0/1717790964 shutdown_connections 2024-02-19T16:30:49.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- 172.21.15.8:0/1717790964 wait complete. 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- start start 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 Processor -- start 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca242130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca242670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca242bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca2b7cc0 con 0x556fca2488b0 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca2b7e30 con 0x556fca23f380 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fca2b7ff0 con 0x556fca23fd30 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b5df9b740 10 client.0 _ll_register_callbacks cb 0x7f6b4400f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:49.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.651+0000 7f6b59889640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca242bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b58887640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca242670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca242bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39630/0 (socket says 172.21.15.8:39630) 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 -- 172.21.15.8:0/512093708 learned_addr learned my addr 172.21.15.8:0/512093708 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca242130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 -- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 msgr2=0x556fca242670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca242670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 -- 172.21.15.8:0/512093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 msgr2=0x556fca242130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca242130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 -- 172.21.15.8:0/512093708 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fca1a5320 con 0x556fca2488b0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b59889640 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca242bb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6b44035040 tx=0x7f6b4400e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc342a1718d5918 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b38ff9640 1 -- 172.21.15.8:0/512093708 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44025040 con 0x556fca2488b0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b38ff9640 1 -- 172.21.15.8:0/512093708 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f6b440126a0 con 0x556fca2488b0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fca1cc900 con 0x556fca2488b0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b38ff9640 1 -- 172.21.15.8:0/512093708 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b44003b80 con 0x556fca2488b0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b5df9b740 10 client.17412 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556fca167910 con 0x556fca2488b0 2024-02-19T16:30:49.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.652+0000 7f6b38ff9640 1 -- 172.21.15.8:0/512093708 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556fca1cc900 con 0x556fca2488b0 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b38ff9640 1 -- 172.21.15.8:0/512093708 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b44046080 con 0x556fca2488b0 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b5df9b740 10 client.17412 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b5df9b740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b5df9b740 20 client.17412 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca2b9460 0x556fca2bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b5df9b740 4 client.17412 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556fc9ea6860 con 0x556fca2b9460 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b58887640 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca2b9460 0x556fca2bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b38ff9640 10 client.17412 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.653+0000 7f6b58887640 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca2b9460 0x556fca2bb880 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40669c2d9f95b64f server_cookie=9d4c6fab953fb89d in_seq=0 out_seq=0 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.654+0000 7f6b38ff9640 1 -- 172.21.15.8:0/512093708 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556fc9ea6860 con 0x556fca2b9460 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.654+0000 7f6b38ff9640 10 client.17412 handle_command_reply: tid=0 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.654+0000 7f6b5df9b740 1 client.17412 shutdown 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 msgr2=0x556fca242bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca242bb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6b44035040 tx=0x7f6b4400e910 comp rx=0 tx=0).stop 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 shutdown_connections 2024-02-19T16:30:49.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556fca2488b0 0x556fca242bb0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556fca2b9460 0x556fca2bb880 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556fca23fd30 0x556fca242670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 --2- 172.21.15.8:0/512093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556fca23f380 0x556fca242130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 >> 172.21.15.8:0/512093708 conn(0x556fca239840 msgr2=0x556fca23aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b38ff9640 0 client.17412 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 shutdown_connections 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 1 -- 172.21.15.8:0/512093708 wait complete. 2024-02-19T16:30:49.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5df9b740 20 client.17412 trim_cache size 0 max 16384 2024-02-19T16:30:49.666 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:30:49.666 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:30:49.666 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "efb7e6e1-1a75-443a-b79b-89ee50879c13", 2024-02-19T16:30:49.666 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:30:49.666 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6b3c0a2230 msgr2=0x7f6b3c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6b3c0a2230 0x7f6b3c0a46f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0b9590 tx=0x7f6b4c0036f0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 msgr2=0x7f6b5419d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.655+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b5419d870 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c000c00 tx=0x7f6b4c02c5f0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 shutdown_connections 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f6b3c0a2230 0x7f6b3c0a46f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b54197a20 0x7f6b54197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b541056e0 0x7f6b5419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 --2- 172.21.15.8:0/2125401448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b54100610 0x7f6b5419d870 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 >> 172.21.15.8:0/2125401448 conn(0x7f6b54076770 msgr2=0x7f6b54069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 shutdown_connections 2024-02-19T16:30:49.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.656+0000 7f6b5b2ec640 1 -- 172.21.15.8:0/2125401448 wait complete. 2024-02-19T16:30:49.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:30:49.669 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:30:49.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 -- 172.21.15.8:0/2050726582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 msgr2=0x7fe49c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 --2- 172.21.15.8:0/2050726582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c1009c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe490009930 tx=0x7fe49002bc30 comp rx=0 tx=0).stop 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 -- 172.21.15.8:0/2050726582 shutdown_connections 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 --2- 172.21.15.8:0/2050726582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c1056b0 0x7fe49c10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 --2- 172.21.15.8:0/2050726582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe49c100f90 0x7fe49c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 --2- 172.21.15.8:0/2050726582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c1009c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 -- 172.21.15.8:0/2050726582 >> 172.21.15.8:0/2050726582 conn(0x7fe49c0775a0 msgr2=0x7fe49c0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 -- 172.21.15.8:0/2050726582 shutdown_connections 2024-02-19T16:30:49.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.775+0000 7fe4a320e640 1 -- 172.21.15.8:0/2050726582 wait complete. 2024-02-19T16:30:49.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 Processor -- start 2024-02-19T16:30:49.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 -- start start 2024-02-19T16:30:49.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe49c1056b0 0x7fe49c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c199430 0x7fe49c1a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1a4080 con 0x7fe49c1005e0 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1a41f0 con 0x7fe49c1056b0 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a320e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1a43b0 con 0x7fe49c199430 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39644/0 (socket says 172.21.15.8:39644) 2024-02-19T16:30:49.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 -- 172.21.15.8:0/971023255 learned_addr learned my addr 172.21.15.8:0/971023255 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe49bfff640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe49c1056b0 0x7fe49c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c199430 0x7fe49c1a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 -- 172.21.15.8:0/971023255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c199430 msgr2=0x7fe49c1a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c199430 0x7fe49c1a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 -- 172.21.15.8:0/971023255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe49c1056b0 msgr2=0x7fe49c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe49c1056b0 0x7fe49c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.776+0000 7fe4a0faa640 1 -- 172.21.15.8:0/971023255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe490009560 con 0x7fe49c1005e0 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.777+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c198960 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe490009230 tx=0x7fe49002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.777+0000 7fe499ffb640 1 -- 172.21.15.8:0/971023255 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490004350 con 0x7fe49c1005e0 2024-02-19T16:30:49.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.777+0000 7fe499ffb640 1 -- 172.21.15.8:0/971023255 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4900044b0 con 0x7fe49c1005e0 2024-02-19T16:30:49.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.777+0000 7fe499ffb640 1 -- 172.21.15.8:0/971023255 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490003d00 con 0x7fe49c1005e0 2024-02-19T16:30:49.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.777+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c1a4630 con 0x7fe49c1005e0 2024-02-19T16:30:49.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.777+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c1a4b90 con 0x7fe49c1005e0 2024-02-19T16:30:49.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.779+0000 7fe499ffb640 1 -- 172.21.15.8:0/971023255 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe490004df0 con 0x7fe49c1005e0 2024-02-19T16:30:49.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 Processor -- start 2024-02-19T16:30:49.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 -- start start 2024-02-19T16:30:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254202b0 0x557625420690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 0x557625429270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254297b0 0x55762542bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762538f280 con 0x557625420bd0 2024-02-19T16:30:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762538c7b0 con 0x5576254202b0 2024-02-19T16:30:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557625387f90 con 0x5576254297b0 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe499ffb640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe48c0a22b0 0x7fe48c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 0x557625429270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a17ab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254202b0 0x557625420690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 0x557625429270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39652/0 (socket says 172.21.15.8:39652) 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 -- 172.21.15.8:0/2232591348 learned_addr learned my addr 172.21.15.8:0/2232591348 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe49bfff640 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254297b0 0x55762542bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe48c0a22b0 0x7fe48c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe499ffb640 1 -- 172.21.15.8:0/971023255 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe4900f0390 con 0x7fe49c1005e0 2024-02-19T16:30:49.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 -- 172.21.15.8:0/2232591348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254297b0 msgr2=0x55762542bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254297b0 0x55762542bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 -- 172.21.15.8:0/2232591348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254202b0 msgr2=0x557625420690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254202b0 0x557625420690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a0faa640 1 -- 172.21.15.8:0/2232591348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557625346570 con 0x557625420bd0 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.780+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254202b0 0x557625420690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 0x557625429270 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe4900ed6d0 tx=0x7fe4900037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1d4e661f988237 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe48c0a22b0 0x7fe48c0a4770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe494005fd0 tx=0x7fe494009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe486ffd640 1 -- 172.21.15.8:0/2232591348 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900bd130 con 0x557625420bd0 2024-02-19T16:30:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe486ffd640 1 -- 172.21.15.8:0/2232591348 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4900bca30 con 0x557625420bd0 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe486ffd640 1 -- 172.21.15.8:0/2232591348 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900edb20 con 0x557625420bd0 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe4a5deb740 1 -- 172.21.15.8:0/2232591348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 msgr2=0x557625429270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.781+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 0x557625429270 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe4900ed6d0 tx=0x7fe4900037d0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- 172.21.15.8:0/2232591348 shutdown_connections 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254297b0 0x55762542bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557625420bd0 0x557625429270 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/2232591348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254202b0 0x557625420690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- 172.21.15.8:0/2232591348 >> 172.21.15.8:0/2232591348 conn(0x55762541a990 msgr2=0x55762541ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- 172.21.15.8:0/2232591348 shutdown_connections 2024-02-19T16:30:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- 172.21.15.8:0/2232591348 wait complete. 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- start start 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 Processor -- start 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625429090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625426420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x557625426960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762505b430 con 0x5576254297b0 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557625017e00 con 0x557625420bd0 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762530cfd0 con 0x5576254202b0 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a5deb740 10 client.0 _ll_register_callbacks cb 0x557625362fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.782+0000 7fe4a17ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x557625426960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x557625426960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39662/0 (socket says 172.21.15.8:39662) 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625429090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 -- 172.21.15.8:0/613170130 learned_addr learned my addr 172.21.15.8:0/613170130 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe49bfff640 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625426420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 -- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 msgr2=0x557625429090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625429090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 -- 172.21.15.8:0/613170130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 msgr2=0x557625426420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625426420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 -- 172.21.15.8:0/613170130 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c1a4b90 con 0x5576254297b0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x557625426960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe494011fd0 tx=0x7fe494004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7b203c91174c06 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe47bfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe47bfff640 1 -- 172.21.15.8:0/613170130 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400fe70 con 0x5576254297b0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe47bfff640 1 -- 172.21.15.8:0/613170130 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe494021b90 con 0x5576254297b0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576250877e0 con 0x5576254297b0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe47bfff640 1 -- 172.21.15.8:0/613170130 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe494021d90 con 0x5576254297b0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a5deb740 10 client.17424 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557625386610 con 0x5576254297b0 2024-02-19T16:30:49.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.783+0000 7fe47bfff640 1 -- 172.21.15.8:0/613170130 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5576250877e0 con 0x5576254297b0 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe47bfff640 1 -- 172.21.15.8:0/613170130 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe494029410 con 0x5576254297b0 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a5deb740 10 client.17424 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a5deb740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a5deb740 20 client.17424 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576254396b0 0x55762543bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a5deb740 4 client.17424 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576250c4720 con 0x5576254396b0 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576254396b0 0x55762543bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576254396b0 0x55762543bad0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e957ef8b7302df1 server_cookie=4e2421369371d401 in_seq=0 out_seq=0 2024-02-19T16:30:49.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.784+0000 7fe47bfff640 10 client.17424 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe47bfff640 1 -- 172.21.15.8:0/613170130 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5576250c4720 con 0x5576254396b0 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe47bfff640 10 client.17424 handle_command_reply: tid=0 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 client.17424 shutdown 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 msgr2=0x557625426960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x557625426960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe494011fd0 tx=0x7fe494004a40 comp rx=0 tx=0).stop 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 shutdown_connections 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x557625426960 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625426420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576254396b0 0x55762543bad0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/613170130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625429090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe47bfff640 0 client.17424 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 >> 172.21.15.8:0/613170130 conn(0x55762541a990 msgr2=0x55762541c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 shutdown_connections 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 1 -- 172.21.15.8:0/613170130 wait complete. 2024-02-19T16:30:49.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.787+0000 7fe4a5deb740 20 client.17424 trim_cache size 0 max 16384 2024-02-19T16:30:49.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.804+0000 7fe4a5deb740 1 Processor -- start 2024-02-19T16:30:49.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.804+0000 7fe4a5deb740 1 -- start start 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 0x557625428b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557625420bd0 0x557625429090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254297b0 0x5576254348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762541b790 con 0x5576254202b0 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557625426cc0 con 0x5576254297b0 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576254162b0 con 0x557625420bd0 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 0x557625428b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 0x557625428b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39676/0 (socket says 172.21.15.8:39676) 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 -- 172.21.15.8:0/1800820762 learned_addr learned my addr 172.21.15.8:0/1800820762 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254297b0 0x5576254348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe49bfff640 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557625420bd0 0x557625429090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 -- 172.21.15.8:0/1800820762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557625420bd0 msgr2=0x557625429090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557625420bd0 0x557625429090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 -- 172.21.15.8:0/1800820762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254297b0 msgr2=0x5576254348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254297b0 0x5576254348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 -- 172.21.15.8:0/1800820762 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557625322ac0 con 0x5576254202b0 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe49bfff640 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557625420bd0 0x557625429090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 0x557625428b50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe4900bc6e0 tx=0x7fe4900ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6e45f479ad1983 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.805+0000 7fe4867fc640 1 -- 172.21.15.8:0/1800820762 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900c5250 con 0x5576254202b0 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4867fc640 1 -- 172.21.15.8:0/1800820762 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4900bd130 con 0x5576254202b0 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4867fc640 1 -- 172.21.15.8:0/1800820762 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900b90a0 con 0x5576254202b0 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 -- 172.21.15.8:0/1800820762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 msgr2=0x557625428b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 0x557625428b50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe4900bc6e0 tx=0x7fe4900ef040 comp rx=0 tx=0).stop 2024-02-19T16:30:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 -- 172.21.15.8:0/1800820762 shutdown_connections 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576254297b0 0x5576254348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557625420bd0 0x557625429090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/1800820762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254202b0 0x557625428b50 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 -- 172.21.15.8:0/1800820762 >> 172.21.15.8:0/1800820762 conn(0x55762541a990 msgr2=0x557625419a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 -- 172.21.15.8:0/1800820762 shutdown_connections 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.806+0000 7fe4a5deb740 1 -- 172.21.15.8:0/1800820762 wait complete. 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 -- start start 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 Processor -- start 2024-02-19T16:30:49.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625423bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625424130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x55762542a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762542add0 con 0x5576254297b0 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762542af40 con 0x557625420bd0 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762542b100 con 0x5576254202b0 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a5deb740 10 client.0 _ll_register_callbacks cb 0x7fe494019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a17ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x55762542a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a17ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x55762542a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39690/0 (socket says 172.21.15.8:39690) 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a17ab640 1 -- 172.21.15.8:0/3811475752 learned_addr learned my addr 172.21.15.8:0/3811475752 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe49bfff640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625424130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.807+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625423bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a17ab640 1 -- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 msgr2=0x557625423bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625423bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a17ab640 1 -- 172.21.15.8:0/3811475752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 msgr2=0x557625424130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625424130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a17ab640 1 -- 172.21.15.8:0/3811475752 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557625346570 con 0x5576254297b0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe49bfff640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625424130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a17ab640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x55762542a700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe494009bb0 tx=0x7fe494029640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7040ae9d8f622f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe47bfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe47bfff640 1 -- 172.21.15.8:0/3811475752 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494029ba0 con 0x5576254297b0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4900bca30 con 0x5576254297b0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe47bfff640 1 -- 172.21.15.8:0/3811475752 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe49400fe70 con 0x5576254297b0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe47bfff640 1 -- 172.21.15.8:0/3811475752 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe494021b90 con 0x5576254297b0 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a5deb740 10 client.17430 fetch_fsmap learned FSMap version 30 2024-02-19T16:30:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5576253486a0 con 0x5576254297b0 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.808+0000 7fe47bfff640 1 -- 172.21.15.8:0/3811475752 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4900bca30 con 0x5576254297b0 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe47bfff640 1 -- 172.21.15.8:0/3811475752 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe494031030 con 0x5576254297b0 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a5deb740 10 client.17430 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a5deb740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a5deb740 20 client.17430 populate_metadata read hostname 'smithi008' 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557625496090 0x557625498480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a5deb740 4 client.17430 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557625113890 con 0x557625496090 2024-02-19T16:30:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557625496090 0x557625498480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:30:49.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe47bfff640 10 client.17430 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.809+0000 7fe4a0faa640 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557625496090 0x557625498480 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39262df90e9072e5 server_cookie=6d60d8f8ad0799c5 in_seq=0 out_seq=0 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe47bfff640 1 -- 172.21.15.8:0/3811475752 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x557625113890 con 0x557625496090 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe47bfff640 10 client.17430 handle_command_reply: tid=0 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 client.17430 shutdown 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 msgr2=0x55762542a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x55762542a700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe494009bb0 tx=0x7fe494029640 comp rx=0 tx=0).stop 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 shutdown_connections 2024-02-19T16:30:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576254297b0 0x55762542a700 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557625420bd0 0x557625424130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x557625496090 0x557625498480 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 --2- 172.21.15.8:0/3811475752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576254202b0 0x557625423bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe47bfff640 0 client.17430 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 >> 172.21.15.8:0/3811475752 conn(0x55762541a990 msgr2=0x557625425cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 shutdown_connections 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 1 -- 172.21.15.8:0/3811475752 wait complete. 2024-02-19T16:30:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.810+0000 7fe4a5deb740 20 client.17430 trim_cache size 0 max 16384 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2970 inodes in the stack)", 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: "efb7e6e1-1a75-443a-b79b-89ee50879c13": { 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "efb7e6e1-1a75-443a-b79b-89ee50879c13", 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:30:49.822 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:30:49.823 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:30:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe48c0a22b0 msgr2=0x7fe48c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe48c0a22b0 0x7fe48c0a4770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe494005fd0 tx=0x7fe494009500 comp rx=0 tx=0).stop 2024-02-19T16:30:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 msgr2=0x7fe49c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:30:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c198960 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe490009230 tx=0x7fe49002c650 comp rx=0 tx=0).stop 2024-02-19T16:30:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 shutdown_connections 2024-02-19T16:30:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe48c0a22b0 0x7fe48c0a4770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c199430 0x7fe49c1a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe49c1056b0 0x7fe49c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 --2- 172.21.15.8:0/971023255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe49c1005e0 0x7fe49c198960 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:30:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 >> 172.21.15.8:0/971023255 conn(0x7fe49c0775a0 msgr2=0x7fe49c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:30:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 shutdown_connections 2024-02-19T16:30:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:30:49.811+0000 7fe4a320e640 1 -- 172.21.15.8:0/971023255 wait complete. 2024-02-19T16:30:49.825 INFO:tasks.cephfs.filesystem:scrub status for tag:efb7e6e1-1a75-443a-b79b-89ee50879c13 - {'path': '/', 'tag': 'efb7e6e1-1a75-443a-b79b-89ee50879c13', 'options': 'recursive,force'} 2024-02-19T16:30:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:50 smithi008 ceph-mon[32573]: pgmap v245: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:50 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:30:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:50 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:50 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:50.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:50 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:50.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:50 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:50 smithi191 ceph-mon[37203]: pgmap v245: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:50 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:30:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:50 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:50 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:50.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:50 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:50.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:50 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:50 smithi156 ceph-mon[36502]: pgmap v245: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:30:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:50 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:30:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:50 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:50 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:50 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:50 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:51.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:51.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:30:51.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:51 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:51.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:51.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:30:51.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:51 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:51 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:30:52.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:52 smithi008 ceph-mon[32573]: pgmap v246: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:52.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:52 smithi191 ceph-mon[37203]: pgmap v246: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:52.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:52 smithi156 ceph-mon[36502]: pgmap v246: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:54.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:54 smithi008 ceph-mon[32573]: pgmap v247: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:54 smithi191 ceph-mon[37203]: pgmap v247: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:54.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:54 smithi156 ceph-mon[36502]: pgmap v247: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:30:56.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:56 smithi008 ceph-mon[32573]: pgmap v248: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:30:56.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:56 smithi191 ceph-mon[37203]: pgmap v248: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:30:56.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:56 smithi156 ceph-mon[36502]: pgmap v248: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:30:58.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:58 smithi008 ceph-mon[32573]: pgmap v249: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:30:58.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:58 smithi191 ceph-mon[37203]: pgmap v249: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:30:58.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:58 smithi156 ceph-mon[36502]: pgmap v249: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:30:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:59 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:30:59.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:30:59 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:30:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:59 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:30:59.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:30:59 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:30:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:59 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:30:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:30:59 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:31:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:00 smithi008 ceph-mon[32573]: pgmap v250: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:31:00.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:00 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:31:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:00 smithi191 ceph-mon[37203]: pgmap v250: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:31:00.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:00 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:31:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:00 smithi156 ceph-mon[36502]: pgmap v250: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:31:00.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:00 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:31:02.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:02 smithi008 ceph-mon[32573]: pgmap v251: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:02.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:02 smithi191 ceph-mon[37203]: pgmap v251: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:02.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:02 smithi156 ceph-mon[36502]: pgmap v251: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:04 smithi008 ceph-mon[32573]: pgmap v252: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:04 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:31:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:04 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:31:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:04 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:31:04.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:04 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:31:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:04 smithi191 ceph-mon[37203]: pgmap v252: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:04 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:31:04.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:04 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:31:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:04 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:31:04.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:04 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:31:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:04 smithi156 ceph-mon[36502]: pgmap v252: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:04 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:31:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:04 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:31:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:04 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:31:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:04 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:31:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:06 smithi008 ceph-mon[32573]: pgmap v253: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:31:06.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:06 smithi191 ceph-mon[37203]: pgmap v253: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:31:06.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:06 smithi156 ceph-mon[36502]: pgmap v253: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:31:08.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:08 smithi008 ceph-mon[32573]: pgmap v254: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:31:08.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:08 smithi191 ceph-mon[37203]: pgmap v254: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:31:08.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:08 smithi156 ceph-mon[36502]: pgmap v254: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:31:10.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:10 smithi191 ceph-mon[37203]: pgmap v255: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:31:10.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:10 smithi191 ceph-mon[37203]: osdmap e101: 12 total, 12 up, 12 in 2024-02-19T16:31:10.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:10 smithi156 ceph-mon[36502]: pgmap v255: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:31:10.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:10 smithi156 ceph-mon[36502]: osdmap e101: 12 total, 12 up, 12 in 2024-02-19T16:31:11.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:10 smithi008 ceph-mon[32573]: pgmap v255: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:31:11.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:10 smithi008 ceph-mon[32573]: osdmap e101: 12 total, 12 up, 12 in 2024-02-19T16:31:12.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:12 smithi191 ceph-mon[37203]: pgmap v257: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:31:12.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:12 smithi156 ceph-mon[36502]: pgmap v257: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:31:13.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:12 smithi008 ceph-mon[32573]: pgmap v257: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:31:14.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:14 smithi191 ceph-mon[37203]: pgmap v258: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:31:14.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:14 smithi156 ceph-mon[36502]: pgmap v258: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:31:15.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:14 smithi008 ceph-mon[32573]: pgmap v258: 129 pgs: 129 active+clean; 79 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:31:16.821 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:16 smithi008 ceph-mon[32573]: pgmap v259: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:16.822 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:16 smithi008 ceph-mon[32573]: osdmap e102: 12 total, 12 up, 12 in 2024-02-19T16:31:16.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:16 smithi156 ceph-mon[36502]: pgmap v259: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:16.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:16 smithi156 ceph-mon[36502]: osdmap e102: 12 total, 12 up, 12 in 2024-02-19T16:31:17.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:16 smithi191 ceph-mon[37203]: pgmap v259: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:17.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:16 smithi191 ceph-mon[37203]: osdmap e102: 12 total, 12 up, 12 in 2024-02-19T16:31:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:18 smithi156 ceph-mon[36502]: pgmap v261: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:31:19.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:18 smithi008 ceph-mon[32573]: pgmap v261: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:31:19.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:18 smithi191 ceph-mon[37203]: pgmap v261: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:31:19.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:31:19.826 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:31:19.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.941+0000 7fe77b44e640 1 -- 172.21.15.8:0/702978802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 msgr2=0x7fe774069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- 172.21.15.8:0/702978802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe774069f80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe768009930 tx=0x7fe76802bc30 comp rx=0 tx=0).stop 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- 172.21.15.8:0/702978802 shutdown_connections 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- 172.21.15.8:0/702978802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe77406a4c0 0x7fe77410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- 172.21.15.8:0/702978802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe774069f80 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- 172.21.15.8:0/702978802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe774107640 0x7fe774107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- 172.21.15.8:0/702978802 >> 172.21.15.8:0/702978802 conn(0x7fe774077190 msgr2=0x7fe7740775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- 172.21.15.8:0/702978802 shutdown_connections 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- 172.21.15.8:0/702978802 wait complete. 2024-02-19T16:31:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 Processor -- start 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- start start 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe77419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe77406a4c0 0x7fe77419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe774107640 0x7fe7741978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774197ef0 con 0x7fe774069b20 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.942+0000 7fe77b44e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774198060 con 0x7fe77406a4c0 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe77b44e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774198220 con 0x7fe774107640 2024-02-19T16:31:19.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe77419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe77419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36794/0 (socket says 172.21.15.8:36794) 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 -- 172.21.15.8:0/1439410671 learned_addr learned my addr 172.21.15.8:0/1439410671 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7799eb640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe774107640 0x7fe7741978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7789e9640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe77406a4c0 0x7fe77419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 -- 172.21.15.8:0/1439410671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe774107640 msgr2=0x7fe7741978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe774107640 0x7fe7741978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 -- 172.21.15.8:0/1439410671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe77406a4c0 msgr2=0x7fe77419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe77406a4c0 0x7fe77419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 -- 172.21.15.8:0/1439410671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009560 con 0x7fe774069b20 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7789e9640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe77406a4c0 0x7fe77419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7799eb640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe774107640 0x7fe7741978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7791ea640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe77419d6f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe76400ebc0 tx=0x7fe76400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7727fc640 1 -- 172.21.15.8:0/1439410671 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640099b0 con 0x7fe774069b20 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7727fc640 1 -- 172.21.15.8:0/1439410671 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe76400eec0 con 0x7fe774069b20 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe774198500 con 0x7fe774069b20 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe7727fc640 1 -- 172.21.15.8:0/1439410671 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764018b60 con 0x7fe774069b20 2024-02-19T16:31:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.943+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741a4790 con 0x7fe774069b20 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.945+0000 7fe7727fc640 1 -- 172.21.15.8:0/1439410671 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe764018cc0 con 0x7fe774069b20 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.946+0000 7fe77e02e740 1 Processor -- start 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.946+0000 7fe77e02e740 1 -- start start 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd6e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7727fc640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe75c0a2230 0x7fe75c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd79ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe77e02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dcdd280 con 0x56076dd6e2b0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe77e02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dcda7b0 con 0x56076dd777b0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe77e02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dcd5f90 con 0x56076dd6ebd0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd6e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7791ea640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe75c0a2230 0x7fe75c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd6e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36800/0 (socket says 172.21.15.8:36800) 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7799eb640 1 -- 172.21.15.8:0/2523340362 learned_addr learned my addr 172.21.15.8:0/2523340362 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd79ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7727fc640 1 -- 172.21.15.8:0/1439410671 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe764014070 con 0x7fe774069b20 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 -- 172.21.15.8:0/2523340362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 msgr2=0x56076dd79ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd79ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 -- 172.21.15.8:0/2523340362 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 msgr2=0x56076dd6e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd6e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 -- 172.21.15.8:0/2523340362 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076dc94570 con 0x56076dd6ebd0 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7791ea640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe75c0a2230 0x7fe75c0a46f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe76409a060 tx=0x7fe7640185c0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.947+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe768009230 tx=0x7fe76802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=623ba17e68e1d56d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe75b7fe640 1 -- 172.21.15.8:0/2523340362 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76802c7e0 con 0x56076dd6ebd0 2024-02-19T16:31:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe75b7fe640 1 -- 172.21.15.8:0/2523340362 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe768004500 con 0x56076dd6ebd0 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe75b7fe640 1 -- 172.21.15.8:0/2523340362 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680053b0 con 0x56076dd6ebd0 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 -- 172.21.15.8:0/2523340362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 msgr2=0x56076dd77270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe768009230 tx=0x7fe76802c660 comp rx=0 tx=0).stop 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 -- 172.21.15.8:0/2523340362 shutdown_connections 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd79ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77270 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2523340362 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd6e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 -- 172.21.15.8:0/2523340362 >> 172.21.15.8:0/2523340362 conn(0x56076dd68990 msgr2=0x56076dd68d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 -- 172.21.15.8:0/2523340362 shutdown_connections 2024-02-19T16:31:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.948+0000 7fe77e02e740 1 -- 172.21.15.8:0/2523340362 wait complete. 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 -- start start 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 Processor -- start 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd77150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd6ebd0 0x56076dd744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd777b0 0x56076dd74a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076d9a9430 con 0x56076dd6e2b0 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076d965e00 con 0x56076dd6ebd0 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dc5afd0 con 0x56076dd777b0 2024-02-19T16:31:19.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe77e02e740 10 client.0 _ll_register_callbacks cb 0x56076dcb0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd77150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd77150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36814/0 (socket says 172.21.15.8:36814) 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe7799eb640 1 -- 172.21.15.8:0/2975748014 learned_addr learned my addr 172.21.15.8:0/2975748014 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd6ebd0 0x56076dd744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.949+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd777b0 0x56076dd74a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe7799eb640 1 -- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd777b0 msgr2=0x56076dd74a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd777b0 0x56076dd74a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe7799eb640 1 -- 172.21.15.8:0/2975748014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd6ebd0 msgr2=0x56076dd744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd6ebd0 0x56076dd744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe7799eb640 1 -- 172.21.15.8:0/2975748014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77410ec30 con 0x56076dd6e2b0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd77150 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe76c013430 tx=0x7fe76c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aff23581cd740e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe758ff9640 1 -- 172.21.15.8:0/2975748014 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c005020 con 0x56076dd6e2b0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe758ff9640 1 -- 172.21.15.8:0/2975748014 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe76c005180 con 0x56076dd6e2b0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56076da12720 con 0x56076dd6e2b0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe758ff9640 1 -- 172.21.15.8:0/2975748014 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76c005340 con 0x56076dd6e2b0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe77e02e740 10 client.17442 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56076dcd4610 con 0x56076dd6e2b0 2024-02-19T16:31:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.950+0000 7fe758ff9640 1 -- 172.21.15.8:0/2975748014 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56076da12720 con 0x56076dd6e2b0 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe758ff9640 1 -- 172.21.15.8:0/2975748014 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe76c01e7f0 con 0x56076dd6e2b0 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe77e02e740 10 client.17442 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe77e02e740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe77e02e740 20 client.17442 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dd87960 0x56076dd89d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe77e02e740 4 client.17442 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56076da61890 con 0x56076dd87960 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dd87960 0x56076dd89d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dd87960 0x56076dd89d80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a2e7697f82983c server_cookie=16f327a90f954b0b in_seq=0 out_seq=0 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.951+0000 7fe758ff9640 10 client.17442 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe758ff9640 1 -- 172.21.15.8:0/2975748014 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56076da61890 con 0x56076dd87960 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe758ff9640 10 client.17442 handle_command_reply: tid=0 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 client.17442 shutdown 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 msgr2=0x56076dd77150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd77150 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe76c013430 tx=0x7fe76c005f90 comp rx=0 tx=0).stop 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 shutdown_connections 2024-02-19T16:31:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dd87960 0x56076dd89d80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd777b0 0x56076dd74a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe758ff9640 0 client.17442 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd6ebd0 0x56076dd744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2975748014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd77150 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 >> 172.21.15.8:0/2975748014 conn(0x56076dd68990 msgr2=0x56076dd6a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 shutdown_connections 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 1 -- 172.21.15.8:0/2975748014 wait complete. 2024-02-19T16:31:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.952+0000 7fe77e02e740 20 client.17442 trim_cache size 0 max 16384 2024-02-19T16:31:19.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.968+0000 7fe77e02e740 1 Processor -- start 2024-02-19T16:31:19.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.968+0000 7fe77e02e740 1 -- start start 2024-02-19T16:31:19.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd76c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd82b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe77e02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dd843a0 con 0x56076dd6e2b0 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe77e02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dd642b0 con 0x56076dd777b0 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe77e02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dd6df10 con 0x56076dd6ebd0 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd76c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd76c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36826/0 (socket says 172.21.15.8:36826) 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 -- 172.21.15.8:0/2266083471 learned_addr learned my addr 172.21.15.8:0/2266083471 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7789e9640 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd82b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 -- 172.21.15.8:0/2266083471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 msgr2=0x56076dd77150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 -- 172.21.15.8:0/2266083471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 msgr2=0x56076dd82b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd82b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 -- 172.21.15.8:0/2266083471 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076dc70ac0 con 0x56076dd6e2b0 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe7799eb640 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd76c10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009bb0 tx=0x7fe76c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4bb0d0275ce40f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe75affd640 1 -- 172.21.15.8:0/2266083471 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c01eea0 con 0x56076dd6e2b0 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe75affd640 1 -- 172.21.15.8:0/2266083471 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe76c005020 con 0x56076dd6e2b0 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.969+0000 7fe75affd640 1 -- 172.21.15.8:0/2266083471 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0035f0 con 0x56076dd6e2b0 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 -- 172.21.15.8:0/2266083471 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 msgr2=0x56076dd76c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd76c10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009bb0 tx=0x7fe76c0096a0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 -- 172.21.15.8:0/2266083471 shutdown_connections 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd82b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd77150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 --2- 172.21.15.8:0/2266083471 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd76c10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 -- 172.21.15.8:0/2266083471 >> 172.21.15.8:0/2266083471 conn(0x56076dd68990 msgr2=0x56076dd6f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 -- 172.21.15.8:0/2266083471 shutdown_connections 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 -- 172.21.15.8:0/2266083471 wait complete. 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 -- start start 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.970+0000 7fe77e02e740 1 Processor -- start 2024-02-19T16:31:19.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dd78d70 con 0x56076dd6e2b0 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dd78ee0 con 0x56076dd777b0 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076dd790a0 con 0x56076dd6ebd0 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe77e02e740 10 client.0 _ll_register_callbacks cb 0x7fe76c013ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd71a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36832/0 (socket says 172.21.15.8:36832) 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 -- 172.21.15.8:0/4206873811 learned_addr learned my addr 172.21.15.8:0/4206873811 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:19.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7789e9640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 -- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 msgr2=0x56076dd71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd71fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 -- 172.21.15.8:0/4206873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 msgr2=0x56076dd786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 -- 172.21.15.8:0/4206873811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c027040 con 0x56076dd6e2b0 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe7799eb640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd71a60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe76c013ae0 tx=0x7fe76c005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fdc5cc37963e9b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:19.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.971+0000 7fe758ff9640 1 -- 172.21.15.8:0/4206873811 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c01e9d0 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe758ff9640 1 -- 172.21.15.8:0/4206873811 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe76c000e40 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56076dd74310 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe758ff9640 1 -- 172.21.15.8:0/4206873811 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76c005190 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 10 client.17448 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56076dc966a0 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe758ff9640 1 -- 172.21.15.8:0/4206873811 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56076dd74310 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe758ff9640 1 -- 172.21.15.8:0/4206873811 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe76c003ba0 con 0x56076dd6e2b0 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 10 client.17448 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:19.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 10 client.17448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 20 client.17448 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dde3e90 0x56076dde62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 4 client.17448 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56076d9d57e0 con 0x56076dde3e90 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.972+0000 7fe7789e9640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dde3e90 0x56076dde62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe7789e9640 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dde3e90 0x56076dde62b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba20448f506c0c17 server_cookie=a27fe59c6b79e6f7 in_seq=0 out_seq=0 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe758ff9640 10 client.17448 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe758ff9640 1 -- 172.21.15.8:0/4206873811 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56076d9d57e0 con 0x56076dde3e90 2024-02-19T16:31:19.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe758ff9640 10 client.17448 handle_command_reply: tid=0 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 client.17448 shutdown 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 msgr2=0x56076dd71a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd71a60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe76c013ae0 tx=0x7fe76c005930 comp rx=0 tx=0).stop 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 shutdown_connections 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56076dd777b0 0x56076dd786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x56076dde3e90 0x56076dde62b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56076dd6ebd0 0x56076dd71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 --2- 172.21.15.8:0/4206873811 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56076dd6e2b0 0x56076dd71a60 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe758ff9640 0 client.17448 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:19.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 >> 172.21.15.8:0/4206873811 conn(0x56076dd68990 msgr2=0x56076dd671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:19.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 shutdown_connections 2024-02-19T16:31:19.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 1 -- 172.21.15.8:0/4206873811 wait complete. 2024-02-19T16:31:19.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.973+0000 7fe77e02e740 20 client.17448 trim_cache size 0 max 16384 2024-02-19T16:31:19.984 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:31:19.984 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:31:19.985 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:31:19.985 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:31:19.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe75c0a2230 msgr2=0x7fe75c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe75c0a2230 0x7fe75c0a46f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe76409a060 tx=0x7fe7640185c0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 msgr2=0x7fe77419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:19.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe77419d6f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe76400ebc0 tx=0x7fe76400c6a0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 shutdown_connections 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe75c0a2230 0x7fe75c0a46f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe774107640 0x7fe7741978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe77406a4c0 0x7fe77419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 --2- 172.21.15.8:0/1439410671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe774069b20 0x7fe77419d6f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 >> 172.21.15.8:0/1439410671 conn(0x7fe774077190 msgr2=0x7fe7740ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 shutdown_connections 2024-02-19T16:31:19.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:19.974+0000 7fe77b44e640 1 -- 172.21.15.8:0/1439410671 wait complete. 2024-02-19T16:31:19.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:31:19.988 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:31:20.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 -- 172.21.15.8:0/2861488371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704107560 msgr2=0x7ff704107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 --2- 172.21.15.8:0/2861488371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704107560 0x7ff704107940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff700009930 tx=0x7ff70002bc30 comp rx=0 tx=0).stop 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 -- 172.21.15.8:0/2861488371 shutdown_connections 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 --2- 172.21.15.8:0/2861488371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff704101e60 0x7ff70410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 --2- 172.21.15.8:0/2861488371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7041014a0 0x7ff704101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 --2- 172.21.15.8:0/2861488371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704107560 0x7ff704107940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 -- 172.21.15.8:0/2861488371 >> 172.21.15.8:0/2861488371 conn(0x7ff7040fcf90 msgr2=0x7ff7040ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 -- 172.21.15.8:0/2861488371 shutdown_connections 2024-02-19T16:31:20.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 -- 172.21.15.8:0/2861488371 wait complete. 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 Processor -- start 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.089+0000 7ff70b488640 1 -- start start 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041014a0 0x7ff7040737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704101e60 0x7ff704073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 0x7ff704074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7041a5e70 con 0x7ff704107560 2024-02-19T16:31:20.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7041a5fe0 con 0x7ff704101e60 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7041a61a0 con 0x7ff7041014a0 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 0x7ff704074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 0x7ff704074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36850/0 (socket says 172.21.15.8:36850) 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 -- 172.21.15.8:0/558302395 learned_addr learned my addr 172.21.15.8:0/558302395 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709224640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041014a0 0x7ff7040737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff708a23640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704101e60 0x7ff704073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 -- 172.21.15.8:0/558302395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041014a0 msgr2=0x7ff7040737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041014a0 0x7ff7040737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 -- 172.21.15.8:0/558302395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704101e60 msgr2=0x7ff704073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704101e60 0x7ff704073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 -- 172.21.15.8:0/558302395 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff700009560 con 0x7ff704107560 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff709a25640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 0x7ff704074270 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00eb10 tx=0x7ff6fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc004b20 con 0x7ff704107560 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7041a6480 con 0x7ff704107560 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc004c80 con 0x7ff704107560 2024-02-19T16:31:20.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc010650 con 0x7ff704107560 2024-02-19T16:31:20.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.090+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7041a6990 con 0x7ff704107560 2024-02-19T16:31:20.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.091+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff704068730 con 0x7ff704107560 2024-02-19T16:31:20.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.092+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff6fc005020 con 0x7ff704107560 2024-02-19T16:31:20.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.093+0000 7ff6fa7fc640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff6f00a2230 0x7ff6f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.093+0000 7ff709224640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff6f00a2230 0x7ff6f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.093+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7ff6fc014070 con 0x7ff704107560 2024-02-19T16:31:20.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.093+0000 7ff709224640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff6f00a2230 0x7ff6f00a46f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff700005ab0 tx=0x7ff700005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.097+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff6fc097290 con 0x7ff704107560 2024-02-19T16:31:20.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.318+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff704109950 con 0x7ff704107560 2024-02-19T16:31:20.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.319+0000 7ff6fa7fc640 1 -- 172.21.15.8:0/558302395 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff6fc09a950 con 0x7ff704107560 2024-02-19T16:31:20.321 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:31:20.321 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:31:20.321 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:31:20.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff6f00a2230 msgr2=0x7ff6f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff6f00a2230 0x7ff6f00a46f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff700005ab0 tx=0x7ff700005fd0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 msgr2=0x7ff704074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 0x7ff704074270 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00eb10 tx=0x7ff6fc00c2d0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 shutdown_connections 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff704107560 0x7ff704074270 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff704101e60 0x7ff704073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff6f00a2230 0x7ff6f00a46f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 --2- 172.21.15.8:0/558302395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041014a0 0x7ff7040737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 >> 172.21.15.8:0/558302395 conn(0x7ff7040fcf90 msgr2=0x7ff7040fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 shutdown_connections 2024-02-19T16:31:20.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.321+0000 7ff70b488640 1 -- 172.21.15.8:0/558302395 wait complete. 2024-02-19T16:31:20.333 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:31:20.333 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:31:20.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 -- 172.21.15.8:0/774725386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c100770 msgr2=0x7fe37c10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 --2- 172.21.15.8:0/774725386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c100770 0x7fe37c10e370 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe374009ca0 tx=0x7fe37402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 -- 172.21.15.8:0/774725386 shutdown_connections 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 --2- 172.21.15.8:0/774725386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c100770 0x7fe37c10e370 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 --2- 172.21.15.8:0/774725386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 --2- 172.21.15.8:0/774725386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37c1076f0 0x7fe37c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 -- 172.21.15.8:0/774725386 >> 172.21.15.8:0/774725386 conn(0x7fe37c075730 msgr2=0x7fe37c075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 -- 172.21.15.8:0/774725386 shutdown_connections 2024-02-19T16:31:20.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.436+0000 7fe382dbd640 1 -- 172.21.15.8:0/774725386 wait complete. 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 Processor -- start 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 -- start start 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37c100770 0x7fe37c19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe37c1076f0 0x7fe37c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c197f10 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c198080 con 0x7fe37c1076f0 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe382dbd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c198240 con 0x7fe37c100770 2024-02-19T16:31:20.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36868/0 (socket says 172.21.15.8:36868) 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 -- 172.21.15.8:0/693050911 learned_addr learned my addr 172.21.15.8:0/693050911 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe37bfff640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37c100770 0x7fe37c19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe38135a640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe37c1076f0 0x7fe37c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 -- 172.21.15.8:0/693050911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37c100770 msgr2=0x7fe37c19dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37c100770 0x7fe37c19dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 -- 172.21.15.8:0/693050911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe37c1076f0 msgr2=0x7fe37c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe37c1076f0 0x7fe37c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 -- 172.21.15.8:0/693050911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374009900 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.437+0000 7fe380b59640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c19d710 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe37000e9f0 tx=0x7fe37000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.438+0000 7fe379ffb640 1 -- 172.21.15.8:0/693050911 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009700 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.438+0000 7fe379ffb640 1 -- 172.21.15.8:0/693050911 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37000cd80 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.438+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c198520 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.438+0000 7fe379ffb640 1 -- 172.21.15.8:0/693050911 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370018920 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.438+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c1a47b0 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe379ffb640 1 -- 172.21.15.8:0/693050911 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe370018a80 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 Processor -- start 2024-02-19T16:31:20.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 -- start start 2024-02-19T16:31:20.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900250 0x55d067900630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067909210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d06790bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0678bb960 con 0x55d067900b70 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06786f310 con 0x55d067909750 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe38598f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06786c840 con 0x55d067900250 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.440+0000 7fe379ffb640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe36c0a2230 0x7fe36c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067909210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe38135a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900250 0x55d067900630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067909210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36884/0 (socket says 172.21.15.8:36884) 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 -- 172.21.15.8:0/1119093708 learned_addr learned my addr 172.21.15.8:0/1119093708 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe38135a640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe36c0a2230 0x7fe36c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe37bfff640 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d06790bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe379ffb640 1 -- 172.21.15.8:0/693050911 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe370014070 con 0x7fe37c0ffdb0 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 -- 172.21.15.8:0/1119093708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900250 msgr2=0x55d067900630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900250 0x55d067900630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 -- 172.21.15.8:0/1119093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 msgr2=0x55d06790bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d06790bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 -- 172.21.15.8:0/1119093708 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d06782a060 con 0x55d067900b70 2024-02-19T16:31:20.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe37bfff640 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d06790bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe38135a640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe36c0a2230 0x7fe36c0a46f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe374005e30 tx=0x7fe374009360 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe380b59640 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067909210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe3700cbbc0 tx=0x7fe37001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e1366629f1a88a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe366ffd640 1 -- 172.21.15.8:0/1119093708 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37009d040 con 0x55d067900b70 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe366ffd640 1 -- 172.21.15.8:0/1119093708 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37009ef50 con 0x55d067900b70 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.441+0000 7fe366ffd640 1 -- 172.21.15.8:0/1119093708 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700ce3a0 con 0x55d067900b70 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 -- 172.21.15.8:0/1119093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 msgr2=0x55d067909210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067909210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe3700cbbc0 tx=0x7fe37001f000 comp rx=0 tx=0).stop 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 -- 172.21.15.8:0/1119093708 shutdown_connections 2024-02-19T16:31:20.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d06790bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067909210 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 --2- 172.21.15.8:0/1119093708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900250 0x55d067900630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 -- 172.21.15.8:0/1119093708 >> 172.21.15.8:0/1119093708 conn(0x55d0678fa980 msgr2=0x55d0678fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 -- 172.21.15.8:0/1119093708 shutdown_connections 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 -- 172.21.15.8:0/1119093708 wait complete. 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 -- start start 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.442+0000 7fe38598f740 1 Processor -- start 2024-02-19T16:31:20.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900250 0x55d067906770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067906cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0679071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d067868020 con 0x55d067900b70 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06753b430 con 0x55d067900250 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0674f7e00 con 0x55d067909750 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 10 client.0 _ll_register_callbacks cb 0x55d06780fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067906cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067906cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36900/0 (socket says 172.21.15.8:36900) 2024-02-19T16:31:20.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 -- 172.21.15.8:0/324197679 learned_addr learned my addr 172.21.15.8:0/324197679 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe380b59640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900250 0x55d067906770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38135a640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0679071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 -- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 msgr2=0x55d0679071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0679071f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 -- 172.21.15.8:0/324197679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900250 msgr2=0x55d067906770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900250 0x55d067906770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 -- 172.21.15.8:0/324197679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c1a47b0 con 0x55d067900b70 2024-02-19T16:31:20.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe37bfff640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067906cb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe3680126e0 tx=0x7fe368012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a69f5f11b85654 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe35bfff640 1 -- 172.21.15.8:0/324197679 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368017070 con 0x55d067900b70 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe35bfff640 1 -- 172.21.15.8:0/324197679 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe368012190 con 0x55d067900b70 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.443+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0675677e0 con 0x55d067900b70 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe35bfff640 1 -- 172.21.15.8:0/324197679 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe36800f920 con 0x55d067900b70 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 10 client.17466 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d0678666a0 con 0x55d067900b70 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe35bfff640 1 -- 172.21.15.8:0/324197679 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d0675677e0 con 0x55d067900b70 2024-02-19T16:31:20.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe35bfff640 1 -- 172.21.15.8:0/324197679 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe368010b90 con 0x55d067900b70 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 10 client.17466 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 10 client.17466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 20 client.17466 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067919c30 0x55d06791c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 4 client.17466 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0675a4720 con 0x55d067919c30 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe380b59640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067919c30 0x55d06791c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe380b59640 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067919c30 0x55d06791c050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c273e8b785a37d3 server_cookie=15180139e191600e in_seq=0 out_seq=0 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.444+0000 7fe35bfff640 10 client.17466 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe35bfff640 1 -- 172.21.15.8:0/324197679 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d0675a4720 con 0x55d067919c30 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe35bfff640 10 client.17466 handle_command_reply: tid=0 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 client.17466 shutdown 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 msgr2=0x55d067906cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067906cb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe3680126e0 tx=0x7fe368012ba0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 shutdown_connections 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067919c30 0x55d06791c050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0679071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe35bfff640 0 client.17466 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900b70 0x55d067906cb0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 --2- 172.21.15.8:0/324197679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900250 0x55d067906770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 >> 172.21.15.8:0/324197679 conn(0x55d0678fa980 msgr2=0x55d0678fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 shutdown_connections 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.445+0000 7fe38598f740 1 -- 172.21.15.8:0/324197679 wait complete. 2024-02-19T16:31:20.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.446+0000 7fe38598f740 20 client.17466 trim_cache size 0 max 16384 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 Processor -- start 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 -- start start 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067907010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900b70 0x55d0678fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0678fc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0677ed130 con 0x55d067900250 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0678fb980 con 0x55d067900b70 2024-02-19T16:31:20.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38598f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d067902380 con 0x55d067909750 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067907010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067907010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36914/0 (socket says 172.21.15.8:36914) 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 -- 172.21.15.8:0/1874340413 learned_addr learned my addr 172.21.15.8:0/1874340413 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe38135a640 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0678fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe37bfff640 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900b70 0x55d0678fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 -- 172.21.15.8:0/1874340413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 msgr2=0x55d0678fc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0678fc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 -- 172.21.15.8:0/1874340413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900b70 msgr2=0x55d0678fc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900b70 0x55d0678fc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.458+0000 7fe380b59640 1 -- 172.21.15.8:0/1874340413 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d067802c20 con 0x55d067900250 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe380b59640 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067907010 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe37009abb0 tx=0x7fe370000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c635119e7360de0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe3667fc640 1 -- 172.21.15.8:0/1874340413 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37009d040 con 0x55d067900250 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe3667fc640 1 -- 172.21.15.8:0/1874340413 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37009ac80 con 0x55d067900250 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe3667fc640 1 -- 172.21.15.8:0/1874340413 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700ce600 con 0x55d067900250 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 -- 172.21.15.8:0/1874340413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 msgr2=0x55d067907010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067907010 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe37009abb0 tx=0x7fe370000bc0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 -- 172.21.15.8:0/1874340413 shutdown_connections 2024-02-19T16:31:20.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067909750 0x55d0678fc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067900b70 0x55d0678fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 --2- 172.21.15.8:0/1874340413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067907010 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 -- 172.21.15.8:0/1874340413 >> 172.21.15.8:0/1874340413 conn(0x55d0678fa980 msgr2=0x55d0678fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 -- 172.21.15.8:0/1874340413 shutdown_connections 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.459+0000 7fe38598f740 1 -- 172.21.15.8:0/1874340413 wait complete. 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 -- start start 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 Processor -- start 2024-02-19T16:31:20.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067903170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900b70 0x55d0679036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d067974520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d067903e70 con 0x55d067900250 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d067903fe0 con 0x55d067909750 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d067974a90 con 0x55d067900b70 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38598f740 10 client.0 _ll_register_callbacks cb 0x7fe368009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067903170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067903170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36922/0 (socket says 172.21.15.8:36922) 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 -- 172.21.15.8:0/2484264927 learned_addr learned my addr 172.21.15.8:0/2484264927 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe38135a640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d067974520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe37bfff640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900b70 0x55d0679036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 -- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900b70 msgr2=0x55d0679036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900b70 0x55d0679036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 -- 172.21.15.8:0/2484264927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 msgr2=0x55d067974520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d067974520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe380b59640 1 -- 172.21.15.8:0/2484264927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d06782a060 con 0x55d067900250 2024-02-19T16:31:20.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.460+0000 7fe37bfff640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900b70 0x55d0679036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe380b59640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067903170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe3700a3070 tx=0x7fe3700cbde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffd5141bfd060ac server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe35bfff640 1 -- 172.21.15.8:0/2484264927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37009d050 con 0x55d067900250 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe35bfff640 1 -- 172.21.15.8:0/2484264927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe3700991b0 con 0x55d067900250 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe37009ef50 con 0x55d067900250 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe35bfff640 1 -- 172.21.15.8:0/2484264927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3700b92c0 con 0x55d067900250 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 10 client.17472 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:20.472 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d0678288c0 con 0x55d067900250 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe35bfff640 1 -- 172.21.15.8:0/2484264927 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37009ef50 con 0x55d067900250 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe35bfff640 1 -- 172.21.15.8:0/2484264927 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe3700cd070 con 0x55d067900250 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 10 client.17472 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 10 client.17472 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 20 client.17472 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067975e40 0x55d067978230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 4 client.17472 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.461+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0675f3890 con 0x55d067975e40 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.462+0000 7fe37bfff640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067975e40 0x55d067978230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.462+0000 7fe37bfff640 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067975e40 0x55d067978230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f29e9bb74baf19 server_cookie=831206e09cbd3062 in_seq=0 out_seq=0 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.462+0000 7fe35bfff640 10 client.17472 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.462+0000 7fe35bfff640 1 -- 172.21.15.8:0/2484264927 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0675f3890 con 0x55d067975e40 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.462+0000 7fe35bfff640 10 client.17472 handle_command_reply: tid=0 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.462+0000 7fe38598f740 1 client.17472 shutdown 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 msgr2=0x55d067903170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067903170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe3700a3070 tx=0x7fe3700cbde0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 shutdown_connections 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55d067975e40 0x55d067978230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d067909750 0x55d067974520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d067900b70 0x55d0679036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe35bfff640 0 client.17472 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 --2- 172.21.15.8:0/2484264927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d067900250 0x55d067903170 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 >> 172.21.15.8:0/2484264927 conn(0x55d0678fa980 msgr2=0x55d0678fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 shutdown_connections 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 1 -- 172.21.15.8:0/2484264927 wait complete. 2024-02-19T16:31:20.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe38598f740 20 client.17472 trim_cache size 0 max 16384 2024-02-19T16:31:20.476 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:31:20.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe36c0a2230 msgr2=0x7fe36c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe36c0a2230 0x7fe36c0a46f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe374005e30 tx=0x7fe374009360 comp rx=0 tx=0).stop 2024-02-19T16:31:20.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 msgr2=0x7fe37c19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c19d710 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe37000e9f0 tx=0x7fe37000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 shutdown_connections 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe37c1076f0 0x7fe37c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe36c0a2230 0x7fe36c0a46f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37c100770 0x7fe37c19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 --2- 172.21.15.8:0/693050911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe37c0ffdb0 0x7fe37c19d710 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 >> 172.21.15.8:0/693050911 conn(0x7fe37c075730 msgr2=0x7fe37c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 shutdown_connections 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.463+0000 7fe382dbd640 1 -- 172.21.15.8:0/693050911 wait complete. 2024-02-19T16:31:20.478 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:31:20.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:31:20.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 -- 172.21.15.8:0/3196165158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 msgr2=0x7f2950107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 --2- 172.21.15.8:0/3196165158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950107940 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2940009900 tx=0x7f294002bc30 comp rx=0 tx=0).stop 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 -- 172.21.15.8:0/3196165158 shutdown_connections 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 --2- 172.21.15.8:0/3196165158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 0x7f295010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 --2- 172.21.15.8:0/3196165158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29501014a0 0x7f2950101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 --2- 172.21.15.8:0/3196165158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950107940 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 -- 172.21.15.8:0/3196165158 >> 172.21.15.8:0/3196165158 conn(0x7f29500788b0 msgr2=0x7f2950078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 -- 172.21.15.8:0/3196165158 shutdown_connections 2024-02-19T16:31:20.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 -- 172.21.15.8:0/3196165158 wait complete. 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 Processor -- start 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.582+0000 7f295771e640 1 -- start start 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f295771e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29501014a0 0x7f29500737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f295771e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 0x7f2950073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f295771e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f295771e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29501a5e20 con 0x7f2950107560 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f295771e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29501a5f90 con 0x7f29501014a0 2024-02-19T16:31:20.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f295771e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29501a6150 con 0x7f2950101e60 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2955cbb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 0x7f2950073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2955cbb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36944/0 (socket says 172.21.15.8:36944) 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2955cbb640 1 -- 172.21.15.8:0/3623815820 learned_addr learned my addr 172.21.15.8:0/3623815820 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f29554ba640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29501014a0 0x7f29500737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 -- 172.21.15.8:0/3623815820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29501014a0 msgr2=0x7f29500737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29501014a0 0x7f29500737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 -- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 msgr2=0x7f2950074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 -- 172.21.15.8:0/3623815820 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2940009560 con 0x7f2950101e60 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2955cbb640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f2954cb9640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 0x7f2950073d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f294c005fd0 tx=0x7f294c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.583+0000 7f29467fc640 1 -- 172.21.15.8:0/3623815820 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009c30 con 0x7f2950101e60 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.584+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29501a6430 con 0x7f2950101e60 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.584+0000 7f29467fc640 1 -- 172.21.15.8:0/3623815820 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c004510 con 0x7f2950101e60 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.584+0000 7f29467fc640 1 -- 172.21.15.8:0/3623815820 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c003e70 con 0x7f2950101e60 2024-02-19T16:31:20.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.584+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29501a6990 con 0x7f2950101e60 2024-02-19T16:31:20.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.585+0000 7f29467fc640 1 -- 172.21.15.8:0/3623815820 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f294c004000 con 0x7f2950101e60 2024-02-19T16:31:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 Processor -- start 2024-02-19T16:31:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 -- start start 2024-02-19T16:31:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f29467fc640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f293c0a2260 0x7f293c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cc0c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cbbe960 con 0x55568cc03250 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cb72310 con 0x55568cc03b70 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f295a3d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cb6f840 con 0x55568cc0c750 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f2955cbb640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f293c0a2260 0x7f293c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f29467fc640 1 -- 172.21.15.8:0/3623815820 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f294c0cdbf0 con 0x7f2950101e60 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f29554ba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f29554ba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc03630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36954/0 (socket says 172.21.15.8:36954) 2024-02-19T16:31:20.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f29554ba640 1 -- 172.21.15.8:0/1821730297 learned_addr learned my addr 172.21.15.8:0/1821730297 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cc0c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.586+0000 7f2955cbb640 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2954cb9640 1 -- 172.21.15.8:0/1821730297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 msgr2=0x55568cc0eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2954cb9640 1 -- 172.21.15.8:0/1821730297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 msgr2=0x55568cc03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2954cb9640 1 -- 172.21.15.8:0/1821730297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55568cb2d060 con 0x55568cc03b70 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f29554ba640 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2955cbb640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f293c0a2260 0x7f293c0a4720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29480049d0 tx=0x7f2948011690 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cc0c210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f294c0caee0 tx=0x7f294c09a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9befcd81497d176e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f293b7fe640 1 -- 172.21.15.8:0/1821730297 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009d90 con 0x55568cc03b70 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f293b7fe640 1 -- 172.21.15.8:0/1821730297 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c00e540 con 0x55568cc03b70 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.587+0000 7f293b7fe640 1 -- 172.21.15.8:0/1821730297 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0035f0 con 0x55568cc03b70 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 -- 172.21.15.8:0/1821730297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 msgr2=0x55568cc0c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cc0c210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f294c0caee0 tx=0x7f294c09a130 comp rx=0 tx=0).stop 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 -- 172.21.15.8:0/1821730297 shutdown_connections 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cc0c210 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1821730297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 -- 172.21.15.8:0/1821730297 >> 172.21.15.8:0/1821730297 conn(0x55568cbfd980 msgr2=0x55568cbfdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 -- 172.21.15.8:0/1821730297 shutdown_connections 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 -- 172.21.15.8:0/1821730297 wait complete. 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 -- start start 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.588+0000 7f295a3d3740 1 Processor -- start 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc09cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cb6b020 con 0x55568cc03b70 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568c83e430 con 0x55568cc03250 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568c7fae00 con 0x55568cc0c750 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 10 client.0 _ll_register_callbacks cb 0x55568cb12cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc09cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc09cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36960/0 (socket says 172.21.15.8:36960) 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 -- 172.21.15.8:0/1466857480 learned_addr learned my addr 172.21.15.8:0/1466857480 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f29554ba640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2955cbb640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 -- 172.21.15.8:0/1466857480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 msgr2=0x55568cc0a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 -- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 msgr2=0x55568cc097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 -- 172.21.15.8:0/1466857480 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29501a6990 con 0x55568cc03b70 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f29554ba640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2954cb9640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc09cf0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f294c003fd0 tx=0x7f294c0a2920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479ea0a0e00ddff4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2938ff9640 1 -- 172.21.15.8:0/1466857480 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009d90 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2938ff9640 1 -- 172.21.15.8:0/1466857480 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f294c0035f0 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55568c86a7e0 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f2938ff9640 1 -- 172.21.15.8:0/1466857480 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294c00e350 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 10 client.17484 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.589+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55568cb696a0 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f2938ff9640 1 -- 172.21.15.8:0/1466857480 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55568c86a7e0 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f2938ff9640 1 -- 172.21.15.8:0/1466857480 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f294c00ec80 con 0x55568cc03b70 2024-02-19T16:31:20.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f295a3d3740 10 client.17484 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f295a3d3740 10 client.17484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f295a3d3740 20 client.17484 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc1c6f0 0x55568cc1eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f295a3d3740 4 client.17484 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55568c8a7720 con 0x55568cc1c6f0 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f29554ba640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc1c6f0 0x55568cc1eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f29554ba640 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc1c6f0 0x55568cc1eae0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51a7887455426988 server_cookie=12cdcb443e81243e in_seq=0 out_seq=0 2024-02-19T16:31:20.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.590+0000 7f2938ff9640 10 client.17484 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.591+0000 7f2938ff9640 1 -- 172.21.15.8:0/1466857480 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55568c8a7720 con 0x55568cc1c6f0 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.591+0000 7f2938ff9640 10 client.17484 handle_command_reply: tid=0 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.591+0000 7f295a3d3740 1 client.17484 shutdown 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 msgr2=0x55568cc09cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc09cf0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f294c003fd0 tx=0x7f294c0a2920 comp rx=0 tx=0).stop 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 shutdown_connections 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc1c6f0 0x55568cc1eae0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc09cf0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f2938ff9640 0 client.17484 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 --2- 172.21.15.8:0/1466857480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 >> 172.21.15.8:0/1466857480 conn(0x55568cbfd980 msgr2=0x55568cbff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 shutdown_connections 2024-02-19T16:31:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 1 -- 172.21.15.8:0/1466857480 wait complete. 2024-02-19T16:31:20.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.592+0000 7f295a3d3740 20 client.17484 trim_cache size 0 max 16384 2024-02-19T16:31:20.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 Processor -- start 2024-02-19T16:31:20.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 -- start start 2024-02-19T16:31:20.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc0bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cbf8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568caf0130 con 0x55568cc03250 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cc02c90 con 0x55568cc03b70 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f295a3d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cc045f0 con 0x55568cc0c750 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f29554ba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc0bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f29554ba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc0bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36966/0 (socket says 172.21.15.8:36966) 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f29554ba640 1 -- 172.21.15.8:0/549675457 learned_addr learned my addr 172.21.15.8:0/549675457 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.604+0000 7f2954cb9640 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cbf8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f2955cbb640 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f29554ba640 1 -- 172.21.15.8:0/549675457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 msgr2=0x55568cc0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f29554ba640 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f29554ba640 1 -- 172.21.15.8:0/549675457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 msgr2=0x55568cbf8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f29554ba640 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cbf8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f29554ba640 1 -- 172.21.15.8:0/549675457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55568cb05c20 con 0x55568cc03250 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f2955cbb640 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f29554ba640 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc0bee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f294002c170 tx=0x7f2940005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bbdcdde0655b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f293affd640 1 -- 172.21.15.8:0/549675457 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940045040 con 0x55568cc03250 2024-02-19T16:31:20.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f293affd640 1 -- 172.21.15.8:0/549675457 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2940047040 con 0x55568cc03250 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f293affd640 1 -- 172.21.15.8:0/549675457 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294002dea0 con 0x55568cc03250 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f295a3d3740 1 -- 172.21.15.8:0/549675457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 msgr2=0x55568cc0bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.605+0000 7f295a3d3740 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc0bee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f294002c170 tx=0x7f2940005a20 comp rx=0 tx=0).stop 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- 172.21.15.8:0/549675457 shutdown_connections 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03b70 0x55568cbf8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 --2- 172.21.15.8:0/549675457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03250 0x55568cc0bee0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- 172.21.15.8:0/549675457 >> 172.21.15.8:0/549675457 conn(0x55568cbfd980 msgr2=0x55568cbfcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- 172.21.15.8:0/549675457 shutdown_connections 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- 172.21.15.8:0/549675457 wait complete. 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- start start 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 Processor -- start 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc10530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc6ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cc704c0 con 0x55568cc03b70 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cc70630 con 0x55568cc03250 2024-02-19T16:31:20.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568cc707f0 con 0x55568cc0c750 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f295a3d3740 10 client.0 _ll_register_callbacks cb 0x7f2940009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f2954cb9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f29554ba640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc10530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f2954cb9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc10a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36974/0 (socket says 172.21.15.8:36974) 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f2954cb9640 1 -- 172.21.15.8:0/2803555495 learned_addr learned my addr 172.21.15.8:0/2803555495 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.606+0000 7f2955cbb640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc6ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2954cb9640 1 -- 172.21.15.8:0/2803555495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 msgr2=0x55568cc6ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2954cb9640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc6ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2954cb9640 1 -- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 msgr2=0x55568cc10530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2954cb9640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc10530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2954cb9640 1 -- 172.21.15.8:0/2803555495 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2940030070 con 0x55568cc03b70 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f29554ba640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc10530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2954cb9640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc10a70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f294c096690 tx=0x7f294c003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe80586f3e81d97 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:20.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2938ff9640 1 -- 172.21.15.8:0/2803555495 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009d90 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2938ff9640 1 -- 172.21.15.8:0/2803555495 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f294c0035f0 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55568cbf90f0 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2938ff9640 1 -- 172.21.15.8:0/2803555495 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294c09a000 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f295a3d3740 10 client.17490 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55568cb2b8c0 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2938ff9640 1 -- 172.21.15.8:0/2803555495 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55568cbf90f0 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f2938ff9640 1 -- 172.21.15.8:0/2803555495 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f294c00ec00 con 0x55568cc03b70 2024-02-19T16:31:20.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f295a3d3740 10 client.17490 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f295a3d3740 10 client.17490 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.607+0000 7f295a3d3740 20 client.17490 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc78d20 0x55568cc7b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 4 client.17490 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f294c00e540 con 0x55568cc78d20 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f29554ba640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc78d20 0x55568cc7b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f29554ba640 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc78d20 0x55568cc7b110 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b0d11b061d6c8b2 server_cookie=718bfd8a7536e640 in_seq=0 out_seq=0 2024-02-19T16:31:20.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f2938ff9640 10 client.17490 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f2938ff9640 1 -- 172.21.15.8:0/2803555495 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f294c00e540 con 0x55568cc78d20 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f2938ff9640 10 client.17490 handle_command_reply: tid=0 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 client.17490 shutdown 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 msgr2=0x55568cc10a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc10a70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f294c096690 tx=0x7f294c003870 comp rx=0 tx=0).stop 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 shutdown_connections 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55568cc78d20 0x55568cc7b110 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55568cc0c750 0x55568cc6ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f2938ff9640 0 client.17490 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55568cc03b70 0x55568cc10a70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 --2- 172.21.15.8:0/2803555495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55568cc03250 0x55568cc10530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 >> 172.21.15.8:0/2803555495 conn(0x55568cbfd980 msgr2=0x55568cbfeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 shutdown_connections 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.608+0000 7f295a3d3740 1 -- 172.21.15.8:0/2803555495 wait complete. 2024-02-19T16:31:20.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295a3d3740 20 client.17490 trim_cache size 0 max 16384 2024-02-19T16:31:20.623 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:31:20.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f293c0a2260 msgr2=0x7f293c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f293c0a2260 0x7f293c0a4720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29480049d0 tx=0x7f2948011690 comp rx=0 tx=0).stop 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 msgr2=0x7f2950073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 0x7f2950073d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f294c005fd0 tx=0x7f294c00dde0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 shutdown_connections 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2950107560 0x7f2950074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f293c0a2260 0x7f293c0a4720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2950101e60 0x7f2950073d30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 --2- 172.21.15.8:0/3623815820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29501014a0 0x7f29500737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 >> 172.21.15.8:0/3623815820 conn(0x7f29500788b0 msgr2=0x7f29500fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 shutdown_connections 2024-02-19T16:31:20.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.609+0000 7f295771e640 1 -- 172.21.15.8:0/3623815820 wait complete. 2024-02-19T16:31:20.625 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:20 smithi008 ceph-mon[32573]: pgmap v262: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 223 B/s wr, 6 op/s 2024-02-19T16:31:20.625 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:20 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/558302395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:31:20.626 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:31:20.626 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:31:20.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 -- 172.21.15.8:0/4197900151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c103080 msgr2=0x7f377c10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 --2- 172.21.15.8:0/4197900151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c103080 0x7f377c10b560 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3770009900 tx=0x7f377002bcd0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 -- 172.21.15.8:0/4197900151 shutdown_connections 2024-02-19T16:31:20.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 --2- 172.21.15.8:0/4197900151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c10bb30 0x7f377c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 --2- 172.21.15.8:0/4197900151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c103080 0x7f377c10b560 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 --2- 172.21.15.8:0/4197900151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c102760 0x7f377c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 -- 172.21.15.8:0/4197900151 >> 172.21.15.8:0/4197900151 conn(0x7f377c077550 msgr2=0x7f377c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 -- 172.21.15.8:0/4197900151 shutdown_connections 2024-02-19T16:31:20.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.729+0000 7f37836eb640 1 -- 172.21.15.8:0/4197900151 wait complete. 2024-02-19T16:31:20.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 Processor -- start 2024-02-19T16:31:20.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 -- start start 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 0x7f377c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c103080 0x7f377c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c10bb30 0x7f377c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c198030 con 0x7f377c102760 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1981a0 con 0x7f377c103080 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f37836eb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c198360 con 0x7f377c10bb30 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f3781487640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 0x7f377c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f3781487640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 0x7f377c19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36980/0 (socket says 172.21.15.8:36980) 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f3781487640 1 -- 172.21.15.8:0/3924043394 learned_addr learned my addr 172.21.15.8:0/3924043394 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f3780c86640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c103080 0x7f377c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.730+0000 7f3781c88640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c10bb30 0x7f377c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f3781487640 1 -- 172.21.15.8:0/3924043394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c10bb30 msgr2=0x7f377c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f3781487640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c10bb30 0x7f377c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f3781487640 1 -- 172.21.15.8:0/3924043394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c103080 msgr2=0x7f377c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f3781487640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c103080 0x7f377c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f3781487640 1 -- 172.21.15.8:0/3924043394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770009560 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f3781487640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 0x7f377c19d7e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f377400d950 tx=0x7f377400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f377a7fc640 1 -- 172.21.15.8:0/3924043394 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774009c30 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c198640 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f377a7fc640 1 -- 172.21.15.8:0/3924043394 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3774009d90 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f377a7fc640 1 -- 172.21.15.8:0/3924043394 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400f7e0 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.731+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c1a4880 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f377a7fc640 1 -- 172.21.15.8:0/3924043394 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f3774010ad0 con 0x7f377c102760 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 Processor -- start 2024-02-19T16:31:20.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 -- start start 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b093210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b095b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7b045960 con 0x561a7b08ab70 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7aff9310 con 0x561a7b093750 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f37863a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7aff6840 con 0x561a7b08a250 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b093210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781c88640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3780c86640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b095b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b093210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36994/0 (socket says 172.21.15.8:36994) 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 -- 172.21.15.8:0/4054280668 learned_addr learned my addr 172.21.15.8:0/4054280668 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f377a7fc640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37640a2230 0x7f37640a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781c88640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37640a2230 0x7f37640a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f377a7fc640 1 -- 172.21.15.8:0/3924043394 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f37740ce0a0 con 0x7f377c102760 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 -- 172.21.15.8:0/4054280668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 msgr2=0x561a7b08a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 -- 172.21.15.8:0/4054280668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 msgr2=0x561a7b095b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b095b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781487640 1 -- 172.21.15.8:0/4054280668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a7afb4060 con 0x561a7b08ab70 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3780c86640 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b095b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.734+0000 7f3781c88640 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.735+0000 7f3781487640 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b093210 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f377409a440 tx=0x7f3774065350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c342691b6d1f5ea server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.735+0000 7f3781c88640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37640a2230 0x7f37640a46f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f376c009fd0 tx=0x7f376c004ab0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.735+0000 7f37637fe640 1 -- 172.21.15.8:0/4054280668 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377409a0a0 con 0x561a7b08ab70 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.735+0000 7f37637fe640 1 -- 172.21.15.8:0/4054280668 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f377409a740 con 0x561a7b08ab70 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.735+0000 7f37637fe640 1 -- 172.21.15.8:0/4054280668 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740cb670 con 0x561a7b08ab70 2024-02-19T16:31:20.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- 172.21.15.8:0/4054280668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 msgr2=0x561a7b093210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b093210 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f377409a440 tx=0x7f3774065350 comp rx=0 tx=0).stop 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- 172.21.15.8:0/4054280668 shutdown_connections 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b095b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b093210 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- 172.21.15.8:0/4054280668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- 172.21.15.8:0/4054280668 >> 172.21.15.8:0/4054280668 conn(0x561a7b084980 msgr2=0x561a7b084d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- 172.21.15.8:0/4054280668 shutdown_connections 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- 172.21.15.8:0/4054280668 wait complete. 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- start start 2024-02-19T16:31:20.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 Processor -- start 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b0907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7aff2020 con 0x561a7b08ab70 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7acc5430 con 0x561a7b093750 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7ac81e00 con 0x561a7b08a250 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.736+0000 7f37863a0740 10 client.0 _ll_register_callbacks cb 0x561a7af99cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b090ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37004/0 (socket says 172.21.15.8:37004) 2024-02-19T16:31:20.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3781487640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b0907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 -- 172.21.15.8:0/169983451 learned_addr learned my addr 172.21.15.8:0/169983451 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3781c88640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 -- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 msgr2=0x561a7b0907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b0907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 -- 172.21.15.8:0/169983451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 msgr2=0x561a7b091220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 -- 172.21.15.8:0/169983451 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c1a4880 con 0x561a7b08ab70 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3781487640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b0907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3780c86640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b090ce0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3770035000 tx=0x7f377002cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51b193bf7c26a6d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3760ff9640 1 -- 172.21.15.8:0/169983451 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770004250 con 0x561a7b08ab70 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3760ff9640 1 -- 172.21.15.8:0/169983451 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f3770046070 con 0x561a7b08ab70 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a7acf17e0 con 0x561a7b08ab70 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f3760ff9640 1 -- 172.21.15.8:0/169983451 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37700043b0 con 0x561a7b08ab70 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f37863a0740 10 client.17502 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.737+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561a7aff06a0 con 0x561a7b08ab70 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f3760ff9640 1 -- 172.21.15.8:0/169983451 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a7acf17e0 con 0x561a7b08ab70 2024-02-19T16:31:20.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f3760ff9640 1 -- 172.21.15.8:0/169983451 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f377003c040 con 0x561a7b08ab70 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f37863a0740 10 client.17502 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f37863a0740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f37863a0740 20 client.17502 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f37863a0740 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b0a3b60 0x561a7b0a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f37863a0740 4 client.17502 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a7ad2e720 con 0x561a7b0a3b60 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f3781487640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b0a3b60 0x561a7b0a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f3760ff9640 10 client.17502 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.738+0000 7f3781487640 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b0a3b60 0x561a7b0a5f80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ca12341bbbe5d9 server_cookie=3c4b0b29d4687949 in_seq=0 out_seq=0 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f3760ff9640 1 -- 172.21.15.8:0/169983451 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561a7ad2e720 con 0x561a7b0a3b60 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f3760ff9640 10 client.17502 handle_command_reply: tid=0 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 client.17502 shutdown 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 msgr2=0x561a7b090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b090ce0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3770035000 tx=0x7f377002cab0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 shutdown_connections 2024-02-19T16:31:20.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08ab70 0x561a7b090ce0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b0a3b60 0x561a7b0a5f80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 --2- 172.21.15.8:0/169983451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b0907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 >> 172.21.15.8:0/169983451 conn(0x561a7b084980 msgr2=0x561a7b0866d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f3760ff9640 0 client.17502 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 shutdown_connections 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 1 -- 172.21.15.8:0/169983451 wait complete. 2024-02-19T16:31:20.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.739+0000 7f37863a0740 20 client.17502 trim_cache size 0 max 16384 2024-02-19T16:31:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 Processor -- start 2024-02-19T16:31:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 -- start start 2024-02-19T16:31:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 0x561a7b092e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08ab70 0x561a7b08af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7af77130 con 0x561a7b08a250 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7b08b4f0 con 0x561a7b093750 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f37863a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7b0961c0 con 0x561a7b08ab70 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 0x561a7b092e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 0x561a7b092e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37016/0 (socket says 172.21.15.8:37016) 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 -- 172.21.15.8:0/1989855802 learned_addr learned my addr 172.21.15.8:0/1989855802 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781c88640 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3780c86640 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08ab70 0x561a7b08af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 -- 172.21.15.8:0/1989855802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08ab70 msgr2=0x561a7b08af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08ab70 0x561a7b08af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 -- 172.21.15.8:0/1989855802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 msgr2=0x561a7b091040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 -- 172.21.15.8:0/1989855802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a7af8cc20 con 0x561a7b08a250 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781c88640 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.752+0000 7f3781487640 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 0x561a7b092e60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f377409a6d0 tx=0x7f3774096b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8555ca92f1781bce server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f3762ffd640 1 -- 172.21.15.8:0/1989855802 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774098b60 con 0x561a7b08a250 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f3762ffd640 1 -- 172.21.15.8:0/1989855802 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37740987e0 con 0x561a7b08a250 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f3762ffd640 1 -- 172.21.15.8:0/1989855802 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740ce360 con 0x561a7b08a250 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 -- 172.21.15.8:0/1989855802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 msgr2=0x561a7b092e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 0x561a7b092e60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f377409a6d0 tx=0x7f3774096b10 comp rx=0 tx=0).stop 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 -- 172.21.15.8:0/1989855802 shutdown_connections 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b093750 0x561a7b091040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08ab70 0x561a7b08af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 --2- 172.21.15.8:0/1989855802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b08a250 0x561a7b092e60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 -- 172.21.15.8:0/1989855802 >> 172.21.15.8:0/1989855802 conn(0x561a7b084980 msgr2=0x561a7b083b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 -- 172.21.15.8:0/1989855802 shutdown_connections 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 -- 172.21.15.8:0/1989855802 wait complete. 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.753+0000 7f37863a0740 1 -- start start 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 Processor -- start 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b08ab70 0x561a7b08ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 0x561a7b08cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7b08d620 con 0x561a7b093750 2024-02-19T16:31:20.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7b08d790 con 0x561a7b08ab70 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a7b103490 con 0x561a7b08a250 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f37863a0740 10 client.0 _ll_register_callbacks cb 0x7f377002c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 0x561a7b08cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 0x561a7b08cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37018/0 (socket says 172.21.15.8:37018) 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781487640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 -- 172.21.15.8:0/3942378456 learned_addr learned my addr 172.21.15.8:0/3942378456 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3780c86640 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b08ab70 0x561a7b08ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 -- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 msgr2=0x561a7b08e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 -- 172.21.15.8:0/3942378456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b08ab70 msgr2=0x561a7b08ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b08ab70 0x561a7b08ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.754+0000 7f3781c88640 1 -- 172.21.15.8:0/3942378456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c013170 con 0x561a7b093750 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3781c88640 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 0x561a7b08cf50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f376c003cc0 tx=0x7f376c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8652aaf12433f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3760ff9640 1 -- 172.21.15.8:0/3942378456 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c027070 con 0x561a7b093750 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3760ff9640 1 -- 172.21.15.8:0/3942378456 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f376c004540 con 0x561a7b093750 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a7b0800f0 con 0x561a7b093750 2024-02-19T16:31:20.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3760ff9640 1 -- 172.21.15.8:0/3942378456 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f376c004700 con 0x561a7b093750 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 10 client.17508 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561a7afb4060 con 0x561a7b093750 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3760ff9640 1 -- 172.21.15.8:0/3942378456 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a7b0800f0 con 0x561a7b093750 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3760ff9640 1 -- 172.21.15.8:0/3942378456 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f376c010430 con 0x561a7b093750 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 10 client.17508 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 20 client.17508 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b104c20 0x561a7b107040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:20.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 4 client.17508 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f377409a740 con 0x561a7b104c20 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.755+0000 7f3781487640 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b104c20 0x561a7b107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f3781487640 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b104c20 0x561a7b107040 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4356ce80e41804 server_cookie=506928860e23a1d0 in_seq=0 out_seq=0 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f3760ff9640 10 client.17508 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f3760ff9640 1 -- 172.21.15.8:0/3942378456 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f377409a740 con 0x561a7b104c20 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f3760ff9640 10 client.17508 handle_command_reply: tid=0 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 client.17508 shutdown 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 msgr2=0x561a7b08cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 0x561a7b08cf50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f376c003cc0 tx=0x7f376c004a40 comp rx=0 tx=0).stop 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 shutdown_connections 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561a7b093750 0x561a7b08cf50 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a7b08ab70 0x561a7b08ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x561a7b104c20 0x561a7b107040 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 --2- 172.21.15.8:0/3942378456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a7b08a250 0x561a7b08e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f3760ff9640 0 client.17508 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 >> 172.21.15.8:0/3942378456 conn(0x561a7b084980 msgr2=0x561a7b09a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 shutdown_connections 2024-02-19T16:31:20.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 1 -- 172.21.15.8:0/3942378456 wait complete. 2024-02-19T16:31:20.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.756+0000 7f37863a0740 20 client.17508 trim_cache size 0 max 16384 2024-02-19T16:31:20.771 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:31:20.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37640a2230 msgr2=0x7f37640a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37640a2230 0x7f37640a46f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f376c009fd0 tx=0x7f376c004ab0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 msgr2=0x7f377c19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 0x7f377c19d7e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f377400d950 tx=0x7f377400de10 comp rx=0 tx=0).stop 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 shutdown_connections 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f37640a2230 0x7f37640a46f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c10bb30 0x7f377c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c103080 0x7f377c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 --2- 172.21.15.8:0/3924043394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f377c102760 0x7f377c19d7e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 >> 172.21.15.8:0/3924043394 conn(0x7f377c077550 msgr2=0x7f377c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 shutdown_connections 2024-02-19T16:31:20.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:20.757+0000 7f37836eb640 1 -- 172.21.15.8:0/3924043394 wait complete. 2024-02-19T16:31:20.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:20 smithi156 ceph-mon[36502]: pgmap v262: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 223 B/s wr, 6 op/s 2024-02-19T16:31:20.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:20 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/558302395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:31:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:20 smithi191 ceph-mon[37203]: pgmap v262: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 223 B/s wr, 6 op/s 2024-02-19T16:31:21.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:20 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/558302395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:31:21.775 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:31:21.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:31:21.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:31:21.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 -- 172.21.15.8:0/2103775408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900690b0 msgr2=0x7f8b9010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 --2- 172.21.15.8:0/2103775408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900690b0 0x7f8b9010e320 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009cd0 tx=0x7f8b8802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 -- 172.21.15.8:0/2103775408 shutdown_connections 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 --2- 172.21.15.8:0/2103775408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900690b0 0x7f8b9010e320 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 --2- 172.21.15.8:0/2103775408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b900686f0 0x7f8b90068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 --2- 172.21.15.8:0/2103775408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b901074f0 0x7f8b901078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 -- 172.21.15.8:0/2103775408 >> 172.21.15.8:0/2103775408 conn(0x7f8b900774f0 msgr2=0x7f8b90077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 -- 172.21.15.8:0/2103775408 shutdown_connections 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.892+0000 7f8b98702640 1 -- 172.21.15.8:0/2103775408 wait complete. 2024-02-19T16:31:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 Processor -- start 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 -- start start 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 0x7f8b9019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900690b0 0x7f8b9019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b901074f0 0x7f8b90197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90197f50 con 0x7f8b900686f0 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b901980c0 con 0x7f8b900690b0 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b98702640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90198280 con 0x7f8b901074f0 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 0x7f8b9019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 0x7f8b9019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37034/0 (socket says 172.21.15.8:37034) 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 -- 172.21.15.8:0/3122861170 learned_addr learned my addr 172.21.15.8:0/3122861170 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900690b0 0x7f8b9019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b901074f0 0x7f8b90197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 -- 172.21.15.8:0/3122861170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b901074f0 msgr2=0x7f8b90197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b901074f0 0x7f8b90197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 -- 172.21.15.8:0/3122861170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900690b0 msgr2=0x7f8b9019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900690b0 0x7f8b9019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b9649e640 1 -- 172.21.15.8:0/3122861170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88009900 con 0x7f8b900686f0 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.893+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900690b0 0x7f8b9019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:31:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.894+0000 7f8b9649e640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 0x7f8b9019d700 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9f0 tx=0x7f8b8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.894+0000 7f8b877fe640 1 -- 172.21.15.8:0/3122861170 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c009700 con 0x7f8b900686f0 2024-02-19T16:31:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.894+0000 7f8b877fe640 1 -- 172.21.15.8:0/3122861170 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c00cd80 con 0x7f8b900686f0 2024-02-19T16:31:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.894+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b90198560 con 0x7f8b900686f0 2024-02-19T16:31:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.894+0000 7f8b877fe640 1 -- 172.21.15.8:0/3122861170 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c018920 con 0x7f8b900686f0 2024-02-19T16:31:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.894+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901a47a0 con 0x7f8b900686f0 2024-02-19T16:31:21.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.896+0000 7f8b877fe640 1 -- 172.21.15.8:0/3122861170 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8b8c018a80 con 0x7f8b900686f0 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 Processor -- start 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 -- start start 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37d30 0x556a29a402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 0x556a29a42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a299a5f90 con 0x556a29a408b0 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a299a34c0 con 0x556a29a37d30 2024-02-19T16:31:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a2999eca0 con 0x556a29a37380 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b877fe640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b7c0a2230 0x7f8b7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b96c9f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 0x556a29a42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b9649e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37d30 0x556a29a402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b96c9f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a37760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:58280/0 (socket says 172.21.15.8:58280) 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 0x556a29a42d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37046/0 (socket says 172.21.15.8:37046) 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b96c9f640 1 -- 172.21.15.8:0/2104607932 learned_addr learned my addr 172.21.15.8:0/2104607932 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.897+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b7c0a2230 0x7f8b7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b877fe640 1 -- 172.21.15.8:0/3122861170 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f8b8c014070 con 0x7f8b900686f0 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 -- 172.21.15.8:0/2104607932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37d30 msgr2=0x556a29a402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37d30 0x556a29a402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 -- 172.21.15.8:0/2104607932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 msgr2=0x556a29a42d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 0x556a29a42d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 -- 172.21.15.8:0/2104607932 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a299610b0 con 0x556a29a37380 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 0x556a29a42d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b7c0a2230 0x7f8b7c0a46f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b8802c700 tx=0x7f8b88048000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a37760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b880098d0 tx=0x7f8b88034830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff834fc00e05f30 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b84ff9640 1 -- 172.21.15.8:0/2104607932 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88032d70 con 0x556a29a37380 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b84ff9640 1 -- 172.21.15.8:0/2104607932 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b88003e50 con 0x556a29a37380 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.898+0000 7f8b84ff9640 1 -- 172.21.15.8:0/2104607932 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88004030 con 0x556a29a37380 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2104607932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 msgr2=0x556a29a37760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a37760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b880098d0 tx=0x7f8b88034830 comp rx=0 tx=0).stop 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2104607932 shutdown_connections 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a408b0 0x556a29a42d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37d30 0x556a29a402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2104607932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a37760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2104607932 >> 172.21.15.8:0/2104607932 conn(0x556a29a31840 msgr2=0x556a29a31c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2104607932 shutdown_connections 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2104607932 wait complete. 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 -- start start 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.899+0000 7f8b9b3b9740 1 Processor -- start 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29943f40 con 0x556a29a37d30 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a296724b0 con 0x556a29a37380 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29924670 con 0x556a29a408b0 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9b3b9740 10 client.0 _ll_register_callbacks cb 0x556a29991f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9649e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b96c9f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a3dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37062/0 (socket says 172.21.15.8:37062) 2024-02-19T16:31:21.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 -- 172.21.15.8:0/2714087180 learned_addr learned my addr 172.21.15.8:0/2714087180 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 -- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 msgr2=0x556a29a3e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 -- 172.21.15.8:0/2714087180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 msgr2=0x556a29a3d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 -- 172.21.15.8:0/2714087180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9010ec20 con 0x556a29a37d30 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b9649e640 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a3dd50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8b80012770 tx=0x7f8b80012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7bbd269793786c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:21.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b79ffb640 1 -- 172.21.15.8:0/2714087180 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017070 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.900+0000 7f8b79ffb640 1 -- 172.21.15.8:0/2714087180 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f8b80011bb0 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a29978f40 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b79ffb640 1 -- 172.21.15.8:0/2714087180 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b80011d70 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 10 client.17520 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556a2999d320 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b79ffb640 1 -- 172.21.15.8:0/2714087180 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a29978f40 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b79ffb640 1 -- 172.21.15.8:0/2714087180 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b8001c5e0 con 0x556a29a37d30 2024-02-19T16:31:21.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 10 client.17520 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 20 client.17520 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29a50e30 0x556a29a53250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 4 client.17520 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a29995310 con 0x556a29a50e30 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.901+0000 7f8b9649e640 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29a50e30 0x556a29a53250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9649e640 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29a50e30 0x556a29a53250 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572a57dbdd35604 server_cookie=c658fc95defcdfa8 in_seq=0 out_seq=0 2024-02-19T16:31:21.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b79ffb640 10 client.17520 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b79ffb640 1 -- 172.21.15.8:0/2714087180 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556a29995310 con 0x556a29a50e30 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b79ffb640 10 client.17520 handle_command_reply: tid=0 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9b3b9740 1 client.17520 shutdown 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 msgr2=0x556a29a3dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a3dd50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8b80012770 tx=0x7f8b80012c30 comp rx=0 tx=0).stop 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 shutdown_connections 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29a50e30 0x556a29a53250 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b79ffb640 0 client.17520 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.902+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a3dd50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/2714087180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 >> 172.21.15.8:0/2714087180 conn(0x556a29a31840 msgr2=0x556a29a337a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 shutdown_connections 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/2714087180 wait complete. 2024-02-19T16:31:21.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.903+0000 7f8b9b3b9740 20 client.17520 trim_cache size 0 max 16384 2024-02-19T16:31:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 Processor -- start 2024-02-19T16:31:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 -- start start 2024-02-19T16:31:21.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29978f40 con 0x556a29a37d30 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29a32640 con 0x556a29a37380 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29a39450 con 0x556a29a408b0 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.919+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a308e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37068/0 (socket says 172.21.15.8:37068) 2024-02-19T16:31:21.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 -- 172.21.15.8:0/3860425276 learned_addr learned my addr 172.21.15.8:0/3860425276 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b9649e640 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b96c9f640 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 -- 172.21.15.8:0/3860425276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 msgr2=0x556a29a3e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 -- 172.21.15.8:0/3860425276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 msgr2=0x556a29a3ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 -- 172.21.15.8:0/3860425276 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a29939c00 con 0x556a29a37d30 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a308e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8b8001ca90 tx=0x7f8b80009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324f78eb35d9ebd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:21.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b7bfff640 1 -- 172.21.15.8:0/3860425276 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017040 con 0x556a29a37d30 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b7bfff640 1 -- 172.21.15.8:0/3860425276 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b80005890 con 0x556a29a37d30 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.920+0000 7f8b7bfff640 1 -- 172.21.15.8:0/3860425276 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800055c0 con 0x556a29a37d30 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/3860425276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 msgr2=0x556a29a308e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a308e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8b8001ca90 tx=0x7f8b80009e60 comp rx=0 tx=0).stop 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/3860425276 shutdown_connections 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a408b0 0x556a29a3e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a308e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/3860425276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a37380 0x556a29a3ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/3860425276 >> 172.21.15.8:0/3860425276 conn(0x556a29a31840 msgr2=0x556a29a31c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/3860425276 shutdown_connections 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/3860425276 wait complete. 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- start start 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 Processor -- start 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a48300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a44680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a408b0 0x556a29a44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29a48bd0 con 0x556a29a37d30 2024-02-19T16:31:21.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29a45100 con 0x556a29a408b0 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a29a452c0 con 0x556a29a37380 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.921+0000 7f8b9b3b9740 10 client.0 _ll_register_callbacks cb 0x7f8b80009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b96c9f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a408b0 0x556a29a44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b9649e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a48300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37078/0 (socket says 172.21.15.8:37078) 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 -- 172.21.15.8:0/317291236 learned_addr learned my addr 172.21.15.8:0/317291236 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b9649e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a48300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:58302/0 (socket says 172.21.15.8:58302) 2024-02-19T16:31:21.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b96c9f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a408b0 0x556a29a44bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:59468/0 (socket says 172.21.15.8:59468) 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 -- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 msgr2=0x556a29a48300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a48300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 -- 172.21.15.8:0/317291236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a408b0 msgr2=0x556a29a44bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a408b0 0x556a29a44bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 -- 172.21.15.8:0/317291236 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a2999d320 con 0x556a29a37d30 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b95c9d640 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a44680 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8b8001ca60 tx=0x7f8b80011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5db17c3aeb22ef server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b79ffb640 1 -- 172.21.15.8:0/317291236 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017040 con 0x556a29a37d30 2024-02-19T16:31:21.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b79ffb640 1 -- 172.21.15.8:0/317291236 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f8b8001c6f0 con 0x556a29a37d30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a299c4900 con 0x556a29a37d30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b79ffb640 1 -- 172.21.15.8:0/317291236 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b80005240 con 0x556a29a37d30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b9b3b9740 10 client.17526 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.922+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556a2995f910 con 0x556a29a37d30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b79ffb640 1 -- 172.21.15.8:0/317291236 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a299c4900 con 0x556a29a37d30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b79ffb640 1 -- 172.21.15.8:0/317291236 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b80031070 con 0x556a29a37d30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9b3b9740 10 client.17526 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9b3b9740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:21.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9b3b9740 20 client.17526 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:21.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29aad890 0x556a29aafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:21.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9b3b9740 4 client.17526 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:31:21.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556a2969e860 con 0x556a29aad890 2024-02-19T16:31:21.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9649e640 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29aad890 0x556a29aafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:21.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b9649e640 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29aad890 0x556a29aafcb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b0d2d074ff5460 server_cookie=4376c97a4970f710 in_seq=0 out_seq=0 2024-02-19T16:31:21.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.923+0000 7f8b79ffb640 10 client.17526 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:21.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.924+0000 7f8b79ffb640 1 -- 172.21.15.8:0/317291236 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556a2969e860 con 0x556a29aad890 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.924+0000 7f8b79ffb640 10 client.17526 handle_command_reply: tid=0 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.924+0000 7f8b9b3b9740 1 client.17526 shutdown 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 msgr2=0x556a29a44680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a44680 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8b8001ca60 tx=0x7f8b80011e40 comp rx=0 tx=0).stop 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 shutdown_connections 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556a29a408b0 0x556a29a44bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556a29a37d30 0x556a29a44680 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556a29aad890 0x556a29aafcb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 --2- 172.21.15.8:0/317291236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556a29a37380 0x556a29a48300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 >> 172.21.15.8:0/317291236 conn(0x556a29a31840 msgr2=0x556a29a308e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:21.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b79ffb640 0 client.17526 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:21.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 shutdown_connections 2024-02-19T16:31:21.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 1 -- 172.21.15.8:0/317291236 wait complete. 2024-02-19T16:31:21.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b9b3b9740 20 client.17526 trim_cache size 0 max 16384 2024-02-19T16:31:21.937 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:31:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:31:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "5226d933-905f-4bf5-b97e-c831890ffb67", 2024-02-19T16:31:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:31:21.937 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:31:21.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b7c0a2230 msgr2=0x7f8b7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b7c0a2230 0x7f8b7c0a46f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b8802c700 tx=0x7f8b88048000 comp rx=0 tx=0).stop 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 msgr2=0x7f8b9019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 0x7f8b9019d700 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9f0 tx=0x7f8b8c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 shutdown_connections 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f8b7c0a2230 0x7f8b7c0a46f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b901074f0 0x7f8b90197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900690b0 0x7f8b9019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 --2- 172.21.15.8:0/3122861170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b900686f0 0x7f8b9019d700 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 >> 172.21.15.8:0/3122861170 conn(0x7f8b900774f0 msgr2=0x7f8b900fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 shutdown_connections 2024-02-19T16:31:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:21.925+0000 7f8b98702640 1 -- 172.21.15.8:0/3122861170 wait complete. 2024-02-19T16:31:21.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:31:21.940 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:31:22.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 -- 172.21.15.8:0/3434646019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82900ff980 msgr2=0x7f82900ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 --2- 172.21.15.8:0/3434646019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82900ff980 0x7f82900ffe00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8280009900 tx=0x7f828002bc30 comp rx=0 tx=0).stop 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 -- 172.21.15.8:0/3434646019 shutdown_connections 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 --2- 172.21.15.8:0/3434646019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8290100340 0x7f829010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 --2- 172.21.15.8:0/3434646019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82900ff980 0x7f82900ffe00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 --2- 172.21.15.8:0/3434646019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8290107570 0x7f8290107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 -- 172.21.15.8:0/3434646019 >> 172.21.15.8:0/3434646019 conn(0x7f8290076900 msgr2=0x7f82900fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:22.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 -- 172.21.15.8:0/3434646019 shutdown_connections 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.045+0000 7f8297efa640 1 -- 172.21.15.8:0/3434646019 wait complete. 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 Processor -- start 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 -- start start 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82900ff980 0x7f829019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8290100340 0x7f829019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 0x7f8290199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019a140 con 0x7f8290107570 2024-02-19T16:31:22.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019a2b0 con 0x7f82900ff980 2024-02-19T16:31:22.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8297efa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019a470 con 0x7f8290100340 2024-02-19T16:31:22.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 0x7f8290199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 0x7f8290199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37086/0 (socket says 172.21.15.8:37086) 2024-02-19T16:31:22.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 -- 172.21.15.8:0/1123447404 learned_addr learned my addr 172.21.15.8:0/1123447404 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:22.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8295495640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8290100340 0x7f829019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8295c96640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82900ff980 0x7f829019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 -- 172.21.15.8:0/1123447404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8290100340 msgr2=0x7f829019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8290100340 0x7f829019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 -- 172.21.15.8:0/1123447404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82900ff980 msgr2=0x7f829019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82900ff980 0x7f829019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 -- 172.21.15.8:0/1123447404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009560 con 0x7f8290107570 2024-02-19T16:31:22.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8296497640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 0x7f8290199af0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f828800eb10 tx=0x7f828800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:22.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8286ffd640 1 -- 172.21.15.8:0/1123447404 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288004b20 con 0x7f8290107570 2024-02-19T16:31:22.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.046+0000 7f8286ffd640 1 -- 172.21.15.8:0/1123447404 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8288004c80 con 0x7f8290107570 2024-02-19T16:31:22.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.047+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829019a750 con 0x7f8290107570 2024-02-19T16:31:22.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.047+0000 7f8286ffd640 1 -- 172.21.15.8:0/1123447404 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288010650 con 0x7f8290107570 2024-02-19T16:31:22.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.047+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82901a69e0 con 0x7f8290107570 2024-02-19T16:31:22.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f8286ffd640 1 -- 172.21.15.8:0/1123447404 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f8288005020 con 0x7f8290107570 2024-02-19T16:31:22.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 Processor -- start 2024-02-19T16:31:22.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 -- start start 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f8286ffd640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f827c0a2230 0x7f827c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410f4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410fd7b0 0x55ca410ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca41063280 con 0x55ca410f42b0 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca410607b0 con 0x55ca410f4bd0 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.049+0000 7f829aba8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4105bf90 con 0x55ca410fd7b0 2024-02-19T16:31:22.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295495640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f827c0a2230 0x7f827c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8286ffd640 1 -- 172.21.15.8:0/1123447404 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f8288014070 con 0x7f8290107570 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410f4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37096/0 (socket says 172.21.15.8:37096) 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 -- 172.21.15.8:0/3588269257 learned_addr learned my addr 172.21.15.8:0/3588269257 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8296497640 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410fd7b0 0x55ca410ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295495640 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 -- 172.21.15.8:0/3588269257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410fd7b0 msgr2=0x55ca410ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410fd7b0 0x55ca410ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 -- 172.21.15.8:0/3588269257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 msgr2=0x55ca410fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 -- 172.21.15.8:0/3588269257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4101a570 con 0x55ca410f42b0 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8296497640 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410fd7b0 0x55ca410ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295495640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f827c0a2230 0x7f827c0a46f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8280006010 tx=0x7f828002c6a0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f8295c96640 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410f4690 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f828c009c70 tx=0x7f828c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acb6294523de028 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f827bfff640 1 -- 172.21.15.8:0/3588269257 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c004230 con 0x55ca410f42b0 2024-02-19T16:31:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f827bfff640 1 -- 172.21.15.8:0/3588269257 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f828c004390 con 0x55ca410f42b0 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.050+0000 7f827bfff640 1 -- 172.21.15.8:0/3588269257 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c019760 con 0x55ca410f42b0 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 -- 172.21.15.8:0/3588269257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 msgr2=0x55ca410f4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410f4690 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f828c009c70 tx=0x7f828c011810 comp rx=0 tx=0).stop 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 -- 172.21.15.8:0/3588269257 shutdown_connections 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410fd7b0 0x55ca410ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 --2- 172.21.15.8:0/3588269257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410f4690 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 -- 172.21.15.8:0/3588269257 >> 172.21.15.8:0/3588269257 conn(0x55ca410ee990 msgr2=0x55ca410eed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 -- 172.21.15.8:0/3588269257 shutdown_connections 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 -- 172.21.15.8:0/3588269257 wait complete. 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 -- start start 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.051+0000 7f829aba8740 1 Processor -- start 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca410fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410fd7b0 0x55ca410faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca40d2f430 con 0x55ca410f42b0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca40cebe00 con 0x55ca410fd7b0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca40fe0fd0 con 0x55ca410f4bd0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f829aba8740 10 client.0 _ll_register_callbacks cb 0x55ca41036fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37110/0 (socket says 172.21.15.8:37110) 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 -- 172.21.15.8:0/3846575984 learned_addr learned my addr 172.21.15.8:0/3846575984 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295495640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca410fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8296497640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410fd7b0 0x55ca410faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 -- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 msgr2=0x55ca410fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca410fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 -- 172.21.15.8:0/3846575984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410fd7b0 msgr2=0x55ca410faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410fd7b0 0x55ca410faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 -- 172.21.15.8:0/3846575984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c0098d0 con 0x55ca410f42b0 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8296497640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410fd7b0 0x55ca410faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f8295c96640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fd130 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f828c004350 tx=0x7f828c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4188dc5ffb244b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.052+0000 7f82797fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f82797fa640 1 -- 172.21.15.8:0/3846575984 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c003800 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f82797fa640 1 -- 172.21.15.8:0/3846575984 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f828c003960 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca40d5b7e0 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f82797fa640 1 -- 172.21.15.8:0/3846575984 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828c003b40 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 10 client.17538 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ca4105a610 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f82797fa640 1 -- 172.21.15.8:0/3846575984 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ca40d5b7e0 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f82797fa640 1 -- 172.21.15.8:0/3846575984 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f828c01a640 con 0x55ca410f42b0 2024-02-19T16:31:22.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 10 client.17538 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 20 client.17538 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca4110d7a0 0x55ca4110fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 4 client.17538 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca40d98720 con 0x55ca4110d7a0 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.053+0000 7f8295495640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca4110d7a0 0x55ca4110fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.054+0000 7f8295495640 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca4110d7a0 0x55ca4110fbe0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaedbcf5906e0294 server_cookie=b9ae0021b6603b7b in_seq=0 out_seq=0 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.054+0000 7f82797fa640 10 client.17538 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:22.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f82797fa640 1 -- 172.21.15.8:0/3846575984 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ca40d98720 con 0x55ca4110d7a0 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f82797fa640 10 client.17538 handle_command_reply: tid=0 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 client.17538 shutdown 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 msgr2=0x55ca410fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fd130 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f828c004350 tx=0x7f828c009ee0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 shutdown_connections 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410fd7b0 0x55ca410faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca4110d7a0 0x55ca4110fbe0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca410fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 --2- 172.21.15.8:0/3846575984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fd130 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f82797fa640 0 client.17538 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:22.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 >> 172.21.15.8:0/3846575984 conn(0x55ca410ee990 msgr2=0x55ca410f0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:22.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 shutdown_connections 2024-02-19T16:31:22.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 1 -- 172.21.15.8:0/3846575984 wait complete. 2024-02-19T16:31:22.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.058+0000 7f829aba8740 20 client.17538 trim_cache size 0 max 16384 2024-02-19T16:31:22.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 Processor -- start 2024-02-19T16:31:22.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 -- start start 2024-02-19T16:31:22.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410ee990 0x55ca410ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca410ee240 con 0x55ca410f42b0 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca41100170 con 0x55ca410f4bd0 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f829aba8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4110d5a0 con 0x55ca410ee990 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.076+0000 7f8295495640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37126/0 (socket says 172.21.15.8:37126) 2024-02-19T16:31:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8296497640 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 -- 172.21.15.8:0/2538204899 learned_addr learned my addr 172.21.15.8:0/2538204899 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295c96640 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410ee990 0x55ca410ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 -- 172.21.15.8:0/2538204899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410ee990 msgr2=0x55ca410ed8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410ee990 0x55ca410ed8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 -- 172.21.15.8:0/2538204899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 msgr2=0x55ca410fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 -- 172.21.15.8:0/2538204899 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca40ff6ac0 con 0x55ca410f42b0 2024-02-19T16:31:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8296497640 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f8295495640 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fcbf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8280009900 tx=0x7f8280046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fbf81da2ce2bb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f827b7fe640 1 -- 172.21.15.8:0/2538204899 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800049e0 con 0x55ca410f42b0 2024-02-19T16:31:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f827b7fe640 1 -- 172.21.15.8:0/2538204899 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8280053070 con 0x55ca410f42b0 2024-02-19T16:31:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.077+0000 7f827b7fe640 1 -- 172.21.15.8:0/2538204899 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280005170 con 0x55ca410f42b0 2024-02-19T16:31:22.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 -- 172.21.15.8:0/2538204899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 msgr2=0x55ca410fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fcbf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8280009900 tx=0x7f8280046000 comp rx=0 tx=0).stop 2024-02-19T16:31:22.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 -- 172.21.15.8:0/2538204899 shutdown_connections 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410f4bd0 0x55ca410fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca410fcbf0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 --2- 172.21.15.8:0/2538204899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410ee990 0x55ca410ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 -- 172.21.15.8:0/2538204899 >> 172.21.15.8:0/2538204899 conn(0x55ca410fa620 msgr2=0x55ca410ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 -- 172.21.15.8:0/2538204899 shutdown_connections 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 -- 172.21.15.8:0/2538204899 wait complete. 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 -- start start 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.078+0000 7f829aba8740 1 Processor -- start 2024-02-19T16:31:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 0x55ca41100630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca41100b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca411010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca410ff840 con 0x55ca410f42b0 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca411690b0 con 0x55ca410ee990 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca411691f0 con 0x55ca410f4bd0 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f829aba8740 10 client.0 _ll_register_callbacks cb 0x7f8288004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca41100b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295c96640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 0x55ca41100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca41100b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37130/0 (socket says 172.21.15.8:37130) 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295c96640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 0x55ca41100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:59516/0 (socket says 172.21.15.8:59516) 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 -- 172.21.15.8:0/3041312763 learned_addr learned my addr 172.21.15.8:0/3041312763 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8296497640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca411010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 -- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 msgr2=0x55ca411010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca411010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 -- 172.21.15.8:0/3041312763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 msgr2=0x55ca41100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 0x55ca41100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295495640 1 -- 172.21.15.8:0/3041312763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829019a750 con 0x55ca410f42b0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.079+0000 7f8295c96640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 0x55ca41100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f8295495640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca41100b70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8280009850 tx=0x7f8280004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d52dd24812adca server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f82797fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f82797fa640 1 -- 172.21.15.8:0/3041312763 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280004ae0 con 0x55ca410f42b0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f82797fa640 1 -- 172.21.15.8:0/3041312763 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f8280053850 con 0x55ca410f42b0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f828c004390 con 0x55ca410f42b0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f82797fa640 1 -- 172.21.15.8:0/3041312763 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8280053a10 con 0x55ca410f42b0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 10 client.17544 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f8280049050 con 0x55ca410f42b0 2024-02-19T16:31:22.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f82797fa640 1 -- 172.21.15.8:0/3041312763 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f828006a020 con 0x55ca410f42b0 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f82797fa640 1 -- 172.21.15.8:0/3041312763 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8280067070 con 0x55ca410f42b0 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 10 client.17544 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 20 client.17544 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca410fd7b0 0x55ca4116c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 4 client.17544 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca40de7890 con 0x55ca410fd7b0 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.080+0000 7f8295c96640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca410fd7b0 0x55ca4116c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:22.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.081+0000 7f8295c96640 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca410fd7b0 0x55ca4116c6d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0d2697f490c7b1 server_cookie=740e99d104f028b4 in_seq=0 out_seq=0 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.081+0000 7f82797fa640 10 client.17544 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.081+0000 7f82797fa640 1 -- 172.21.15.8:0/3041312763 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55ca40de7890 con 0x55ca410fd7b0 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.081+0000 7f82797fa640 10 client.17544 handle_command_reply: tid=0 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.081+0000 7f829aba8740 1 client.17544 shutdown 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 msgr2=0x55ca41100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca41100b70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8280009850 tx=0x7f8280004800 comp rx=0 tx=0).stop 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 shutdown_connections 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ca410fd7b0 0x55ca4116c6d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca410f4bd0 0x55ca411010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca410f42b0 0x55ca41100b70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 --2- 172.21.15.8:0/3041312763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca410ee990 0x55ca41100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 >> 172.21.15.8:0/3041312763 conn(0x55ca410fa620 msgr2=0x55ca410eda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f82797fa640 0 client.17544 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 shutdown_connections 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 1 -- 172.21.15.8:0/3041312763 wait complete. 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f829aba8740 20 client.17544 trim_cache size 0 max 16384 2024-02-19T16:31:22.093 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2932 inodes in the stack)", 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: "5226d933-905f-4bf5-b97e-c831890ffb67": { 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "5226d933-905f-4bf5-b97e-c831890ffb67", 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:31:22.094 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:31:22.095 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:31:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f827c0a2230 msgr2=0x7f827c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f827c0a2230 0x7f827c0a46f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8280006010 tx=0x7f828002c6a0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 msgr2=0x7f8290199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 0x7f8290199af0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f828800eb10 tx=0x7f828800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 shutdown_connections 2024-02-19T16:31:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8290107570 0x7f8290199af0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f827c0a2230 0x7f827c0a46f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8290100340 0x7f829019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 --2- 172.21.15.8:0/1123447404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82900ff980 0x7f829019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 >> 172.21.15.8:0/1123447404 conn(0x7f8290076900 msgr2=0x7f82900fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.082+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 shutdown_connections 2024-02-19T16:31:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:22.083+0000 7f8297efa640 1 -- 172.21.15.8:0/1123447404 wait complete. 2024-02-19T16:31:22.097 INFO:tasks.cephfs.filesystem:scrub status for tag:5226d933-905f-4bf5-b97e-c831890ffb67 - {'path': '/', 'tag': '5226d933-905f-4bf5-b97e-c831890ffb67', 'options': 'recursive,force'} 2024-02-19T16:31:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:22 smithi156 ceph-mon[36502]: pgmap v263: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:22 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:31:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:22 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:22.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:22 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:22.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:22 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:22.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:22 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:22 smithi008 ceph-mon[32573]: pgmap v263: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:22 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:31:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:22 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:22 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:22 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:23.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:22 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:22 smithi191 ceph-mon[37203]: pgmap v263: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:22 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:31:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:22 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:22 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:22 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:23.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:22 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:23.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:23.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:23.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:23.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:23 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:24.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:24.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:24.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:23 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:24.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:24.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:23 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:24.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:24 smithi156 ceph-mon[36502]: pgmap v264: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:25.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:24 smithi008 ceph-mon[32573]: pgmap v264: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:25.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:24 smithi191 ceph-mon[37203]: pgmap v264: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:31:26.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:26 smithi156 ceph-mon[36502]: pgmap v265: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:31:27.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:26 smithi008 ceph-mon[32573]: pgmap v265: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:31:27.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:26 smithi191 ceph-mon[37203]: pgmap v265: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:31:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:28 smithi156 ceph-mon[36502]: pgmap v266: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:31:29.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:28 smithi008 ceph-mon[32573]: pgmap v266: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:31:29.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:28 smithi191 ceph-mon[37203]: pgmap v266: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:31:29.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:29 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:31:29.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:29 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:31:30.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:29 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:31:30.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:29 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:31:30.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:29 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:31:30.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:29 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:31:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:30 smithi156 ceph-mon[36502]: pgmap v267: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:31:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:30 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:31:31.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:30 smithi008 ceph-mon[32573]: pgmap v267: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:31:31.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:30 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:31:31.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:30 smithi191 ceph-mon[37203]: pgmap v267: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:31:31.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:30 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:31:32.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:32 smithi156 ceph-mon[36502]: pgmap v268: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:33.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:32 smithi008 ceph-mon[32573]: pgmap v268: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:33.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:32 smithi191 ceph-mon[37203]: pgmap v268: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:34.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:34 smithi156 ceph-mon[36502]: pgmap v269: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:35.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:34 smithi008 ceph-mon[32573]: pgmap v269: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:35.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:34 smithi191 ceph-mon[37203]: pgmap v269: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:31:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:36 smithi156 ceph-mon[36502]: pgmap v270: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:31:37.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:36 smithi008 ceph-mon[32573]: pgmap v270: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:31:37.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:36 smithi191 ceph-mon[37203]: pgmap v270: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:31:38.829 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:38 smithi191 ceph-mon[37203]: pgmap v271: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:38.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:38 smithi156 ceph-mon[36502]: pgmap v271: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:39.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:38 smithi008 ceph-mon[32573]: pgmap v271: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:40 smithi156 ceph-mon[36502]: pgmap v272: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:41.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:40 smithi008 ceph-mon[32573]: pgmap v272: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:41.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:40 smithi191 ceph-mon[37203]: pgmap v272: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:42 smithi156 ceph-mon[36502]: pgmap v273: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:43.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:42 smithi008 ceph-mon[32573]: pgmap v273: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:43.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:42 smithi191 ceph-mon[37203]: pgmap v273: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:44 smithi156 ceph-mon[36502]: pgmap v274: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:45.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:44 smithi008 ceph-mon[32573]: pgmap v274: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:45.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:44 smithi191 ceph-mon[37203]: pgmap v274: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:46.822 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:46 smithi008 ceph-mon[32573]: pgmap v275: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:46 smithi156 ceph-mon[36502]: pgmap v275: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:47.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:46 smithi191 ceph-mon[37203]: pgmap v275: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:48 smithi156 ceph-mon[36502]: pgmap v276: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:49.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:48 smithi008 ceph-mon[32573]: pgmap v276: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:49.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:48 smithi191 ceph-mon[37203]: pgmap v276: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:50 smithi156 ceph-mon[36502]: pgmap v277: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:51.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:50 smithi008 ceph-mon[32573]: pgmap v277: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:51.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:50 smithi191 ceph-mon[37203]: pgmap v277: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:51.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:51 smithi156 ceph-mon[36502]: pgmap v278: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:52.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:51 smithi008 ceph-mon[32573]: pgmap v278: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:52.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:51 smithi191 ceph-mon[37203]: pgmap v278: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:52.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:31:52.099 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:31:52.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3972269858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4107640 msgr2=0x7f63c4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3972269858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4107640 0x7f63c4107a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009890 tx=0x7f63b402bb70 comp rx=0 tx=0).stop 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3972269858 shutdown_connections 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3972269858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c406a4c0 0x7f63c410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3972269858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4069b20 0x7f63c4069f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3972269858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4107640 0x7f63c4107a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.215+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3972269858 >> 172.21.15.8:0/3972269858 conn(0x7f63c4077190 msgr2=0x7f63c40775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3972269858 shutdown_connections 2024-02-19T16:31:52.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3972269858 wait complete. 2024-02-19T16:31:52.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 Processor -- start 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 -- start start 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4069b20 0x7f63c419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 0x7f63c419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.216+0000 7f63c9d8c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c4107640 0x7f63c4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c9d8c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c4197ef0 con 0x7f63c406a4c0 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c9d8c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c4198060 con 0x7f63c4107640 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c9d8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c4198220 con 0x7f63c4069b20 2024-02-19T16:31:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 0x7f63c419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 0x7f63c419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59662/0 (socket says 172.21.15.8:59662) 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3362162790 learned_addr learned my addr 172.21.15.8:0/3362162790 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c37fe640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4069b20 0x7f63c419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c3fff640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c4107640 0x7f63c4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3362162790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4069b20 msgr2=0x7f63c419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4069b20 0x7f63c419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3362162790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c4107640 msgr2=0x7f63c4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c4107640 0x7f63c4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3362162790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009490 con 0x7f63c406a4c0 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c37fe640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4069b20 0x7f63c419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 0x7f63c419dc20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f63b800e9c0 tx=0x7f63b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c0ff9640 1 -- 172.21.15.8:0/3362162790 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b800ccc0 con 0x7f63c406a4c0 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c4198500 con 0x7f63c406a4c0 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c0ff9640 1 -- 172.21.15.8:0/3362162790 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63b800ce20 con 0x7f63c406a4c0 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c41a4710 con 0x7f63c406a4c0 2024-02-19T16:31:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.217+0000 7f63c0ff9640 1 -- 172.21.15.8:0/3362162790 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8010430 con 0x7f63c406a4c0 2024-02-19T16:31:52.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63c0ff9640 1 -- 172.21.15.8:0/3362162790 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f63b8004f80 con 0x7f63c406a4c0 2024-02-19T16:31:52.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 Processor -- start 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 -- start start 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e970690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e979270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e8df280 con 0x55e29e9702b0 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.220+0000 7f63cc96a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e8dc7b0 con 0x55e29e9797b0 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c0ff9640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f63ac0a2230 0x7f63ac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63cc96a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e8d7f90 con 0x55e29e970bd0 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e970690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e970690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59668/0 (socket says 172.21.15.8:59668) 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c3fff640 1 -- 172.21.15.8:0/743838858 learned_addr learned my addr 172.21.15.8:0/743838858 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e979270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c3fff640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f63ac0a2230 0x7f63ac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c0ff9640 1 -- 172.21.15.8:0/3362162790 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f63b8014070 con 0x7f63c406a4c0 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 -- 172.21.15.8:0/743838858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 msgr2=0x55e29e97bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 -- 172.21.15.8:0/743838858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 msgr2=0x55e29e970690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e970690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 -- 172.21.15.8:0/743838858 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29e896570 con 0x55e29e970bd0 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c3fff640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f63ac0a2230 0x7f63ac0a46f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0099e0 tx=0x7f63bc009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63c37fe640 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e979270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f63b40057c0 tx=0x7f63b40056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5ad50a57485113 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63a9ffb640 1 -- 172.21.15.8:0/743838858 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4004240 con 0x55e29e970bd0 2024-02-19T16:31:52.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63a9ffb640 1 -- 172.21.15.8:0/743838858 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63b4034920 con 0x55e29e970bd0 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.221+0000 7f63a9ffb640 1 -- 172.21.15.8:0/743838858 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b402cb10 con 0x55e29e970bd0 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 -- 172.21.15.8:0/743838858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 msgr2=0x55e29e979270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e979270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f63b40057c0 tx=0x7f63b40056d0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 -- 172.21.15.8:0/743838858 shutdown_connections 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e979270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 --2- 172.21.15.8:0/743838858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e970690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 -- 172.21.15.8:0/743838858 >> 172.21.15.8:0/743838858 conn(0x55e29e96a990 msgr2=0x55e29e96ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 -- 172.21.15.8:0/743838858 shutdown_connections 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.222+0000 7f63cc96a740 1 -- 172.21.15.8:0/743838858 wait complete. 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 -- start start 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 Processor -- start 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e9791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e976560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e976b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e5ab430 con 0x55e29e970bd0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e567e00 con 0x55e29e9797b0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e85cfd0 con 0x55e29e9702b0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63cc96a740 10 client.0 _ll_register_callbacks cb 0x55e29e8b2fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e976560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e976560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59676/0 (socket says 172.21.15.8:59676) 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63c2ffd640 1 -- 172.21.15.8:0/1344745117 learned_addr learned my addr 172.21.15.8:0/1344745117 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63c37fe640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e9791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.223+0000 7f63c3fff640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e976b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c37fe640 1 -- 172.21.15.8:0/1344745117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 msgr2=0x55e29e976b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c37fe640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e976b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c37fe640 1 -- 172.21.15.8:0/1344745117 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 msgr2=0x55e29e976560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c37fe640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e976560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c37fe640 1 -- 172.21.15.8:0/1344745117 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c410ec30 con 0x55e29e9702b0 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e976560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63c37fe640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e9791d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009890 tx=0x7f63b402c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dabcf679ecc9c90 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:31:52.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63a2ffd640 1 -- 172.21.15.8:0/1344745117 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4004450 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63a2ffd640 1 -- 172.21.15.8:0/1344745117 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f63b4032600 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e29e5d77e0 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63a2ffd640 1 -- 172.21.15.8:0/1344745117 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b40327c0 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 10 client.27412 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e29e8d6610 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63a2ffd640 1 -- 172.21.15.8:0/1344745117 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e29e5d77e0 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63a2ffd640 1 -- 172.21.15.8:0/1344745117 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63b40037b0 con 0x55e29e9702b0 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 10 client.27412 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 10 client.27412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 20 client.27412 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e989500 0x55e29e98b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 4 client.27412 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.224+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e29e614720 con 0x55e29e989500 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.225+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e989500 0x55e29e98b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.225+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e989500 0x55e29e98b8f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8fb1a78f7256381 server_cookie=cd9c7b38824ceb3a in_seq=0 out_seq=0 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.225+0000 7f63a2ffd640 10 client.27412 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63a2ffd640 1 -- 172.21.15.8:0/1344745117 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e29e614720 con 0x55e29e989500 2024-02-19T16:31:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63a2ffd640 10 client.27412 handle_command_reply: tid=0 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 client.27412 shutdown 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 msgr2=0x55e29e9791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e9791d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009890 tx=0x7f63b402c690 comp rx=0 tx=0).stop 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 shutdown_connections 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e976b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e976560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e989500 0x55e29e98b8f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1344745117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e9791d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 >> 172.21.15.8:0/1344745117 conn(0x55e29e96a990 msgr2=0x55e29e96c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63a2ffd640 0 client.27412 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 shutdown_connections 2024-02-19T16:31:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 1 -- 172.21.15.8:0/1344745117 wait complete. 2024-02-19T16:31:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.226+0000 7f63cc96a740 20 client.27412 trim_cache size 0 max 16384 2024-02-19T16:31:52.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 Processor -- start 2024-02-19T16:31:52.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 -- start start 2024-02-19T16:31:52.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e976500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e978c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e9791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e9863f0 con 0x55e29e970bd0 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e96ff20 con 0x55e29e9797b0 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.242+0000 7f63cc96a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e98bf20 con 0x55e29e9702b0 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e978c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e978c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59682/0 (socket says 172.21.15.8:59682) 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 -- 172.21.15.8:0/1766322790 learned_addr learned my addr 172.21.15.8:0/1766322790 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c37fe640 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e976500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c3fff640 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e9791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 -- 172.21.15.8:0/1766322790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 msgr2=0x55e29e976500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e976500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 -- 172.21.15.8:0/1766322790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 msgr2=0x55e29e9791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e9791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 -- 172.21.15.8:0/1766322790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29e872ac0 con 0x55e29e970bd0 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e978c90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005740 tx=0x7f63b801bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f336ec3968e1b61 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63a97fa640 1 -- 172.21.15.8:0/1766322790 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b807d070 con 0x55e29e970bd0 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63a97fa640 1 -- 172.21.15.8:0/1766322790 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63b80b7070 con 0x55e29e970bd0 2024-02-19T16:31:52.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.243+0000 7f63a97fa640 1 -- 172.21.15.8:0/1766322790 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b809ef40 con 0x55e29e970bd0 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 -- 172.21.15.8:0/1766322790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 msgr2=0x55e29e978c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e978c90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005740 tx=0x7f63b801bb30 comp rx=0 tx=0).stop 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 -- 172.21.15.8:0/1766322790 shutdown_connections 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e9791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e970bd0 0x55e29e978c90 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 --2- 172.21.15.8:0/1766322790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e9702b0 0x55e29e976500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 -- 172.21.15.8:0/1766322790 >> 172.21.15.8:0/1766322790 conn(0x55e29e96a990 msgr2=0x55e29e975df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 -- 172.21.15.8:0/1766322790 shutdown_connections 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 -- 172.21.15.8:0/1766322790 wait complete. 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 -- start start 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.244+0000 7f63cc96a740 1 Processor -- start 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e981140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e97aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e97b500 con 0x55e29e9702b0 2024-02-19T16:31:52.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e97b670 con 0x55e29e9797b0 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29e97b830 con 0x55e29e970bd0 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 10 client.0 _ll_register_callbacks cb 0x7f63bc011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e981140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e97aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e981140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59692/0 (socket says 172.21.15.8:59692) 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c37fe640 1 -- 172.21.15.8:0/3374908108 learned_addr learned my addr 172.21.15.8:0/3374908108 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c3fff640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3374908108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 msgr2=0x55e29e97afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3374908108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 msgr2=0x55e29e981140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e981140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 -- 172.21.15.8:0/3374908108 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c4198500 con 0x55e29e970bd0 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c37fe640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e981140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63c2ffd640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e97aa80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005740 tx=0x7f63b8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d78dcc4be219bc server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63a2ffd640 1 -- 172.21.15.8:0/3374908108 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b807d030 con 0x55e29e970bd0 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63a2ffd640 1 -- 172.21.15.8:0/3374908108 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f63b807e070 con 0x55e29e970bd0 2024-02-19T16:31:52.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63b4034920 con 0x55e29e970bd0 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.245+0000 7f63a2ffd640 1 -- 172.21.15.8:0/3374908108 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b809b480 con 0x55e29e970bd0 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 10 client.27418 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e29e8986a0 con 0x55e29e970bd0 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63a2ffd640 1 -- 172.21.15.8:0/3374908108 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63b4034920 con 0x55e29e970bd0 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63a2ffd640 1 -- 172.21.15.8:0/3374908108 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63b801f060 con 0x55e29e970bd0 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 10 client.27418 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 10 client.27418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:52.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 20 client.27418 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e9e5ee0 0x55e29e9e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 4 client.27418 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e29e663890 con 0x55e29e9e5ee0 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63c37fe640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e9e5ee0 0x55e29e9e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63a2ffd640 10 client.27418 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63c37fe640 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e9e5ee0 0x55e29e9e82d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99432c37ad0dd8f1 server_cookie=1fe009e15bd1a208 in_seq=0 out_seq=0 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63a2ffd640 1 -- 172.21.15.8:0/3374908108 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e29e663890 con 0x55e29e9e5ee0 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.246+0000 7f63a2ffd640 10 client.27418 handle_command_reply: tid=0 2024-02-19T16:31:52.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 client.27418 shutdown 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 msgr2=0x55e29e97aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e97aa80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005740 tx=0x7f63b8005880 comp rx=0 tx=0).stop 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 shutdown_connections 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e29e9797b0 0x55e29e97afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55e29e9e5ee0 0x55e29e9e82d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e29e970bd0 0x55e29e97aa80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63a2ffd640 0 client.27418 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 --2- 172.21.15.8:0/3374908108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e29e9702b0 0x55e29e981140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 >> 172.21.15.8:0/3374908108 conn(0x55e29e96a990 msgr2=0x55e29e9691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 shutdown_connections 2024-02-19T16:31:52.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 1 -- 172.21.15.8:0/3374908108 wait complete. 2024-02-19T16:31:52.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63cc96a740 20 client.27418 trim_cache size 0 max 16384 2024-02-19T16:31:52.259 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:31:52.259 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:31:52.259 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:31:52.260 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:31:52.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f63ac0a2230 msgr2=0x7f63ac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f63ac0a2230 0x7f63ac0a46f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0099e0 tx=0x7f63bc009500 comp rx=0 tx=0).stop 2024-02-19T16:31:52.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 msgr2=0x7f63c419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.261 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 0x7f63c419dc20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f63b800e9c0 tx=0x7f63b800ee80 comp rx=0 tx=0).stop 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 shutdown_connections 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c4107640 0x7f63c4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c406a4c0 0x7f63c419dc20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f63ac0a2230 0x7f63ac0a46f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 --2- 172.21.15.8:0/3362162790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63c4069b20 0x7f63c419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.247+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 >> 172.21.15.8:0/3362162790 conn(0x7f63c4077190 msgr2=0x7f63c40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.248+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 shutdown_connections 2024-02-19T16:31:52.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.248+0000 7f63c9d8c640 1 -- 172.21.15.8:0/3362162790 wait complete. 2024-02-19T16:31:52.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:31:52.263 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:31:52.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.363+0000 7f12574da640 1 -- 172.21.15.8:0/1422414012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 msgr2=0x7f125010d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.363+0000 7f12574da640 1 --2- 172.21.15.8:0/1422414012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f125010d2a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1248009cd0 tx=0x7f124802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 -- 172.21.15.8:0/1422414012 shutdown_connections 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 --2- 172.21.15.8:0/1422414012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f125010d2a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 --2- 172.21.15.8:0/1422414012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100fb0 0x7f1250104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 --2- 172.21.15.8:0/1422414012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250100600 0x7f12501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 -- 172.21.15.8:0/1422414012 >> 172.21.15.8:0/1422414012 conn(0x7f1250076770 msgr2=0x7f1250076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 -- 172.21.15.8:0/1422414012 shutdown_connections 2024-02-19T16:31:52.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 -- 172.21.15.8:0/1422414012 wait complete. 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 Processor -- start 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 -- start start 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100600 0x7f1250102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f12501025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250102b10 0x7f12501a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.364+0000 7f12574da640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a6250 con 0x7f12501056d0 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12574da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a63c0 con 0x7f1250100600 2024-02-19T16:31:52.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12574da640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a6580 con 0x7f1250102b10 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f12501025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f12501025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59700/0 (socket says 172.21.15.8:59700) 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 -- 172.21.15.8:0/3264886648 learned_addr learned my addr 172.21.15.8:0/3264886648 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1255276640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100600 0x7f1250102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1255a77640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250102b10 0x7f12501a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 -- 172.21.15.8:0/3264886648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250102b10 msgr2=0x7f12501a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250102b10 0x7f12501a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 -- 172.21.15.8:0/3264886648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100600 msgr2=0x7f1250102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100600 0x7f1250102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 -- 172.21.15.8:0/3264886648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c009490 con 0x7f12501056d0 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1255276640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100600 0x7f1250102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f1254a75640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f12501025d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9a0 tx=0x7f124c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00cd80 con 0x7f12501056d0 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124c0035f0 con 0x7f12501056d0 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1248009900 con 0x7f12501056d0 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c010690 con 0x7f12501056d0 2024-02-19T16:31:52.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.365+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12501a6be0 con 0x7f12501056d0 2024-02-19T16:31:52.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.366+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1250101ad0 con 0x7f12501056d0 2024-02-19T16:31:52.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.367+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f124c003ad0 con 0x7f12501056d0 2024-02-19T16:31:52.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.368+0000 7f12427fc640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f123c0a2260 0x7f123c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.368+0000 7f1255276640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f123c0a2260 0x7f123c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.368+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f124c014070 con 0x7f12501056d0 2024-02-19T16:31:52.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.369+0000 7f1255276640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f123c0a2260 0x7f123c0a4720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fc0 tx=0x7f1244005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.372+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f124c097260 con 0x7f12501056d0 2024-02-19T16:31:52.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.593+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1250068730 con 0x7f12501056d0 2024-02-19T16:31:52.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.593+0000 7f12427fc640 1 -- 172.21.15.8:0/3264886648 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f124c003760 con 0x7f12501056d0 2024-02-19T16:31:52.595 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:31:52.595 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:31:52.596 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:31:52.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f123c0a2260 msgr2=0x7f123c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f123c0a2260 0x7f123c0a4720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fc0 tx=0x7f1244005f90 comp rx=0 tx=0).stop 2024-02-19T16:31:52.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 msgr2=0x7f12501025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f12501025d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9a0 tx=0x7f124c00ee60 comp rx=0 tx=0).stop 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 shutdown_connections 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f123c0a2260 0x7f123c0a4720 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1250102b10 0x7f12501a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12501056d0 0x7f12501025d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 --2- 172.21.15.8:0/3264886648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1250100600 0x7f1250102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 >> 172.21.15.8:0/3264886648 conn(0x7f1250076770 msgr2=0x7f1250069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 shutdown_connections 2024-02-19T16:31:52.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.595+0000 7f12574da640 1 -- 172.21.15.8:0/3264886648 wait complete. 2024-02-19T16:31:52.608 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:31:52.608 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3480409858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f381056e0 msgr2=0x7f4f3810d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3480409858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f381056e0 0x7f4f3810d2b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009cd0 tx=0x7f4f3002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3480409858 shutdown_connections 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3480409858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f381056e0 0x7f4f3810d2b0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3480409858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100fc0 0x7f4f38105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3480409858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f38100610 0x7f4f381009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3480409858 >> 172.21.15.8:0/3480409858 conn(0x7f4f38076770 msgr2=0x7f4f38076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3480409858 shutdown_connections 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3480409858 wait complete. 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.711+0000 7f4f3d5f4640 1 Processor -- start 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 -- start start 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100610 0x7f4f38198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f381056e0 0x7f4f38198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 0x7f4f381a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f381a40a0 con 0x7f4f38199450 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f381a4210 con 0x7f4f381056e0 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f3d5f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f381a43d0 con 0x7f4f38100610 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 0x7f4f381a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100610 0x7f4f38198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 0x7f4f381a39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59722/0 (socket says 172.21.15.8:59722) 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f377fe640 1 -- 172.21.15.8:0/3893820145 learned_addr learned my addr 172.21.15.8:0/3893820145 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f367fc640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f381056e0 0x7f4f38198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 -- 172.21.15.8:0/3893820145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f381056e0 msgr2=0x7f4f38198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f381056e0 0x7f4f38198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 -- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 msgr2=0x7f4f381a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 0x7f4f381a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 -- 172.21.15.8:0/3893820145 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009490 con 0x7f4f38100610 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f377fe640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 0x7f4f381a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.712+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100610 0x7f4f38198930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9f0 tx=0x7f4f2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.713+0000 7f4f1ffff640 1 -- 172.21.15.8:0/3893820145 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800cd80 con 0x7f4f38100610 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.713+0000 7f4f1ffff640 1 -- 172.21.15.8:0/3893820145 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f280035f0 con 0x7f4f38100610 2024-02-19T16:31:52.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.713+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f30009900 con 0x7f4f38100610 2024-02-19T16:31:52.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.713+0000 7f4f1ffff640 1 -- 172.21.15.8:0/3893820145 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28010690 con 0x7f4f38100610 2024-02-19T16:31:52.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.713+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f381a4a30 con 0x7f4f38100610 2024-02-19T16:31:52.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.714+0000 7f4f1ffff640 1 -- 172.21.15.8:0/3893820145 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4f28003af0 con 0x7f4f38100610 2024-02-19T16:31:52.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.715+0000 7f4f1ffff640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4f200a2230 0x7f4f200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.715+0000 7f4f401cd740 1 Processor -- start 2024-02-19T16:31:52.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.715+0000 7f4f401cd740 1 -- start start 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.715+0000 7f4f367fc640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4f200a2230 0x7f4f200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f1ffff640 1 -- 172.21.15.8:0/3893820145 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f4f28014070 con 0x7f4f38100610 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfee9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9b70 0x562cdfef2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfef2750 0x562cdfef4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f401cd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfea4960 con 0x562cdfee9b70 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f401cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfe58310 con 0x562cdfef2750 2024-02-19T16:31:52.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f401cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfe55840 con 0x562cdfee9250 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfee9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9b70 0x562cdfef2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfee9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:56462/0 (socket says 172.21.15.8:56462) 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f36ffd640 1 -- 172.21.15.8:0/421594534 learned_addr learned my addr 172.21.15.8:0/421594534 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f377fe640 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfef2750 0x562cdfef4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4f200a2230 0x7f4f200a46f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f3806a670 tx=0x7f4f2c005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 -- 172.21.15.8:0/421594534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 msgr2=0x562cdfee9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfee9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 -- 172.21.15.8:0/421594534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfef2750 msgr2=0x562cdfef4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfef2750 0x562cdfef4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 -- 172.21.15.8:0/421594534 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cdfe13060 con 0x562cdfee9b70 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfee9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f377fe640 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfef2750 0x562cdfef4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f367fc640 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9b70 0x562cdfef2210 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4f380698d0 tx=0x7f4f2c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a72caf9c30eb22d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.716+0000 7f4f1d7fa640 1 -- 172.21.15.8:0/421594534 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00eb30 con 0x562cdfee9b70 2024-02-19T16:31:52.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f1d7fa640 1 -- 172.21.15.8:0/421594534 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c021070 con 0x562cdfee9b70 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f1d7fa640 1 -- 172.21.15.8:0/421594534 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c004c20 con 0x562cdfee9b70 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 -- 172.21.15.8:0/421594534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9b70 msgr2=0x562cdfef2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9b70 0x562cdfef2210 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4f380698d0 tx=0x7f4f2c014000 comp rx=0 tx=0).stop 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 -- 172.21.15.8:0/421594534 shutdown_connections 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfef2750 0x562cdfef4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9b70 0x562cdfef2210 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 --2- 172.21.15.8:0/421594534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfee9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 -- 172.21.15.8:0/421594534 >> 172.21.15.8:0/421594534 conn(0x562cdfee3980 msgr2=0x562cdfee3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 -- 172.21.15.8:0/421594534 shutdown_connections 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 -- 172.21.15.8:0/421594534 wait complete. 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.717+0000 7f4f401cd740 1 -- start start 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 Processor -- start 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 0x562cdfeef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfeefcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfef2750 0x562cdfef01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfe51020 con 0x562cdfee9250 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfb24430 con 0x562cdfee9b70 2024-02-19T16:31:52.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfae0e00 con 0x562cdfef2750 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f401cd740 10 client.0 _ll_register_callbacks cb 0x562cdfdf8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 0x562cdfeef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 0x562cdfeef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59744/0 (socket says 172.21.15.8:59744) 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 -- 172.21.15.8:0/402705254 learned_addr learned my addr 172.21.15.8:0/402705254 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f377fe640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfef2750 0x562cdfef01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f367fc640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfeefcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 -- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfef2750 msgr2=0x562cdfef01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfef2750 0x562cdfef01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 -- 172.21.15.8:0/402705254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 msgr2=0x562cdfeefcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfeefcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 -- 172.21.15.8:0/402705254 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c017070 con 0x562cdfee9250 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 0x562cdfeef770 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009460 tx=0x7f4f2809af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4655bbda33ec370 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f1aff5640 1 -- 172.21.15.8:0/402705254 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f280a3600 con 0x562cdfee9250 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.718+0000 7f4f1aff5640 1 -- 172.21.15.8:0/402705254 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f4f2809d040 con 0x562cdfee9250 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cdfb507e0 con 0x562cdfee9250 2024-02-19T16:31:52.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f1aff5640 1 -- 172.21.15.8:0/402705254 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f280974d0 con 0x562cdfee9250 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 10 client.17580 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562cdfe4f6a0 con 0x562cdfee9250 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f1aff5640 1 -- 172.21.15.8:0/402705254 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562cdfb507e0 con 0x562cdfee9250 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f1aff5640 1 -- 172.21.15.8:0/402705254 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f280a3170 con 0x562cdfee9250 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 10 client.17580 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 10 client.17580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 20 client.17580 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff029d0 0x562cdff04dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 4 client.17580 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cdfb8d720 con 0x562cdff029d0 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.719+0000 7f4f367fc640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff029d0 0x562cdff04dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f367fc640 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff029d0 0x562cdff04dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df27cfbfe0d4c2e4 server_cookie=46eea1319d783c2a in_seq=0 out_seq=0 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f1aff5640 10 client.17580 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f1aff5640 1 -- 172.21.15.8:0/402705254 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562cdfb8d720 con 0x562cdff029d0 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f1aff5640 10 client.17580 handle_command_reply: tid=0 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 client.17580 shutdown 2024-02-19T16:31:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 msgr2=0x562cdfeef770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 0x562cdfeef770 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009460 tx=0x7f4f2809af20 comp rx=0 tx=0).stop 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 shutdown_connections 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff029d0 0x562cdff04dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfef2750 0x562cdfef01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfeefcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f1aff5640 0 client.17580 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 --2- 172.21.15.8:0/402705254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfee9250 0x562cdfeef770 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 >> 172.21.15.8:0/402705254 conn(0x562cdfee3980 msgr2=0x562cdfee56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 shutdown_connections 2024-02-19T16:31:52.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.720+0000 7f4f401cd740 1 -- 172.21.15.8:0/402705254 wait complete. 2024-02-19T16:31:52.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.721+0000 7f4f401cd740 20 client.17580 trim_cache size 0 max 16384 2024-02-19T16:31:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 Processor -- start 2024-02-19T16:31:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 -- start start 2024-02-19T16:31:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfef0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfee5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfee5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfdd6130 con 0x562cdfef2750 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfeff640 con 0x562cdfee9b70 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f401cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdff027d0 con 0x562cdfee9250 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfee5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfef0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfee5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59748/0 (socket says 172.21.15.8:59748) 2024-02-19T16:31:52.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 -- 172.21.15.8:0/1475518941 learned_addr learned my addr 172.21.15.8:0/1475518941 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f367fc640 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfee5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 -- 172.21.15.8:0/1475518941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 msgr2=0x562cdfef0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfef0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 -- 172.21.15.8:0/1475518941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 msgr2=0x562cdfee5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfee5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f377fe640 1 -- 172.21.15.8:0/1475518941 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cdfdebc20 con 0x562cdfef2750 2024-02-19T16:31:52.744 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.733+0000 7f4f367fc640 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfee5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f377fe640 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfee5790 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009bf0 tx=0x7f4f300348a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce8f4f710b2dbf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f1cff9640 1 -- 172.21.15.8:0/1475518941 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30032340 con 0x562cdfef2750 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f1cff9640 1 -- 172.21.15.8:0/1475518941 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f300324a0 con 0x562cdfef2750 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f1cff9640 1 -- 172.21.15.8:0/1475518941 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30003d60 con 0x562cdfef2750 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 -- 172.21.15.8:0/1475518941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 msgr2=0x562cdfee5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfee5790 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009bf0 tx=0x7f4f300348a0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 -- 172.21.15.8:0/1475518941 shutdown_connections 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfee5790 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9b70 0x562cdfee5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 --2- 172.21.15.8:0/1475518941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9250 0x562cdfef0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 -- 172.21.15.8:0/1475518941 >> 172.21.15.8:0/1475518941 conn(0x562cdfee3980 msgr2=0x562cdfee4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 -- 172.21.15.8:0/1475518941 shutdown_connections 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.734+0000 7f4f401cd740 1 -- 172.21.15.8:0/1475518941 wait complete. 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 -- start start 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 Processor -- start 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9250 0x562cdfefa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9b70 0x562cdfeec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfeece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfeed3a0 con 0x562cdfef2750 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfeed510 con 0x562cdfee9250 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdfeed6d0 con 0x562cdfee9b70 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f401cd740 10 client.0 _ll_register_callbacks cb 0x7f4f2809b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfeece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfeece60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59752/0 (socket says 172.21.15.8:59752) 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 -- 172.21.15.8:0/661357501 learned_addr learned my addr 172.21.15.8:0/661357501 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9250 0x562cdfefa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f367fc640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9b70 0x562cdfeec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 -- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9b70 msgr2=0x562cdfeec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9b70 0x562cdfeec920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 -- 172.21.15.8:0/661357501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9250 msgr2=0x562cdfefa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9250 0x562cdfefa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 -- 172.21.15.8:0/661357501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cdfe13060 con 0x562cdfef2750 2024-02-19T16:31:52.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9250 0x562cdfefa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f377fe640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfeece60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4f30005e30 tx=0x7f4f300328e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ccc9393a2bf2a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.735+0000 7f4f1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f1aff5640 1 -- 172.21.15.8:0/661357501 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30032340 con 0x562cdfef2750 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f1aff5640 1 -- 172.21.15.8:0/661357501 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f4f30032a50 con 0x562cdfef2750 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f2c021070 con 0x562cdfef2750 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f1aff5640 1 -- 172.21.15.8:0/661357501 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f30048620 con 0x562cdfef2750 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 10 client.17586 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:52.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f4f30030070 con 0x562cdfef2750 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f1aff5640 1 -- 172.21.15.8:0/661357501 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c021070 con 0x562cdfef2750 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f1aff5640 1 -- 172.21.15.8:0/661357501 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f3004f070 con 0x562cdfef2750 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 10 client.17586 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 10 client.17586 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 20 client.17586 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff5f440 0x562cdff61830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 4 client.17586 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562cdfbdc890 con 0x562cdff5f440 2024-02-19T16:31:52.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.736+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff5f440 0x562cdff61830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f36ffd640 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff5f440 0x562cdff61830 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=caadc97984598f0 server_cookie=75fe807e5753a158 in_seq=0 out_seq=0 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f1aff5640 10 client.17586 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f1aff5640 1 -- 172.21.15.8:0/661357501 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562cdfbdc890 con 0x562cdff5f440 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f1aff5640 10 client.17586 handle_command_reply: tid=0 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 client.17586 shutdown 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 msgr2=0x562cdfeece60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfeece60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4f30005e30 tx=0x7f4f300328e0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 shutdown_connections 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x562cdff5f440 0x562cdff61830 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562cdfef2750 0x562cdfeece60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f1aff5640 0 client.17586 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cdfee9b70 0x562cdfeec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 --2- 172.21.15.8:0/661357501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562cdfee9250 0x562cdfefa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 >> 172.21.15.8:0/661357501 conn(0x562cdfee3980 msgr2=0x562cdfee5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 shutdown_connections 2024-02-19T16:31:52.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 1 -- 172.21.15.8:0/661357501 wait complete. 2024-02-19T16:31:52.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.737+0000 7f4f401cd740 20 client.17586 trim_cache size 0 max 16384 2024-02-19T16:31:52.753 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:31:52.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4f200a2230 msgr2=0x7f4f200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4f200a2230 0x7f4f200a46f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f3806a670 tx=0x7f4f2c005f90 comp rx=0 tx=0).stop 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100610 msgr2=0x7f4f38198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100610 0x7f4f38198930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9f0 tx=0x7f4f2800eeb0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 shutdown_connections 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f38199450 0x7f4f381a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f381056e0 0x7f4f38198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f4f200a2230 0x7f4f200a46f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 --2- 172.21.15.8:0/3893820145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f38100610 0x7f4f38198930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 >> 172.21.15.8:0/3893820145 conn(0x7f4f38076770 msgr2=0x7f4f380fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 shutdown_connections 2024-02-19T16:31:52.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.738+0000 7f4f3d5f4640 1 -- 172.21.15.8:0/3893820145 wait complete. 2024-02-19T16:31:52.756 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:31:52.756 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:31:52.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 -- 172.21.15.8:0/2065509056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6581005d0 msgr2=0x7fd6581009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/2065509056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6581005d0 0x7fd6581009b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd650009be0 tx=0x7fd65002bef0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 -- 172.21.15.8:0/2065509056 shutdown_connections 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/2065509056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581056a0 0x7fd65810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/2065509056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd658100f80 0x7fd658104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/2065509056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6581005d0 0x7fd6581009b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.861+0000 7fd65ea6b640 1 -- 172.21.15.8:0/2065509056 >> 172.21.15.8:0/2065509056 conn(0x7fd6580774f0 msgr2=0x7fd658077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 -- 172.21.15.8:0/2065509056 shutdown_connections 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 -- 172.21.15.8:0/2065509056 wait complete. 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 Processor -- start 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 -- start start 2024-02-19T16:31:52.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581005d0 0x7fd658102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 0x7fd6581025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd658102ae0 0x7fd6581a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a6340 con 0x7fd6581056a0 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a64b0 con 0x7fd6581005d0 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65ea6b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a6670 con 0x7fd658102ae0 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd657fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 0x7fd6581025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd657fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 0x7fd6581025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59766/0 (socket says 172.21.15.8:59766) 2024-02-19T16:31:52.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd657fff640 1 -- 172.21.15.8:0/476756403 learned_addr learned my addr 172.21.15.8:0/476756403 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.862+0000 7fd65c807640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581005d0 0x7fd658102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd65d008640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd658102ae0 0x7fd6581a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd657fff640 1 -- 172.21.15.8:0/476756403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd658102ae0 msgr2=0x7fd6581a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd657fff640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd658102ae0 0x7fd6581a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd657fff640 1 -- 172.21.15.8:0/476756403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581005d0 msgr2=0x7fd658102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd657fff640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581005d0 0x7fd658102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd657fff640 1 -- 172.21.15.8:0/476756403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650009840 con 0x7fd6581056a0 2024-02-19T16:31:52.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd65c807640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581005d0 0x7fd658102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd657fff640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 0x7fd6581025a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00e9f0 tx=0x7fd64c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd655ffb640 1 -- 172.21.15.8:0/476756403 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c009600 con 0x7fd6581056a0 2024-02-19T16:31:52.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6581a6950 con 0x7fd6581056a0 2024-02-19T16:31:52.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd655ffb640 1 -- 172.21.15.8:0/476756403 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd64c00ccc0 con 0x7fd6581056a0 2024-02-19T16:31:52.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd655ffb640 1 -- 172.21.15.8:0/476756403 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c018880 con 0x7fd6581056a0 2024-02-19T16:31:52.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.863+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6581a6eb0 con 0x7fd6581056a0 2024-02-19T16:31:52.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.865+0000 7fd655ffb640 1 -- 172.21.15.8:0/476756403 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fd64c010a80 con 0x7fd6581056a0 2024-02-19T16:31:52.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 Processor -- start 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 -- start start 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd655ffb640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6400a2230 0x7fd6400a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf782b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7834210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7836b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf77e6960 con 0x55fcf782b250 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf779a310 con 0x55fcf7834750 2024-02-19T16:31:52.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd661642740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf7797840 con 0x55fcf782bb70 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf782b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf782b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59780/0 (socket says 172.21.15.8:59780) 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65d008640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6400a2230 0x7fd6400a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd657fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7834210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 -- 172.21.15.8:0/1507544583 learned_addr learned my addr 172.21.15.8:0/1507544583 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd655ffb640 1 -- 172.21.15.8:0/476756403 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fd64c014070 con 0x7fd6581056a0 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65d008640 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7836b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 -- 172.21.15.8:0/1507544583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 msgr2=0x55fcf7834210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7834210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 -- 172.21.15.8:0/1507544583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 msgr2=0x55fcf7836b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7836b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.866+0000 7fd65c807640 1 -- 172.21.15.8:0/1507544583 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcf7755060 con 0x55fcf782b250 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd65d008640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6400a2230 0x7fd6400a46f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55fcf7826b10 tx=0x7fd6480098c0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd65c807640 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf782b630 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd65002c430 tx=0x7fd650033e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10d90cd1f1fa1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd63effd640 1 -- 172.21.15.8:0/1507544583 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65002ca90 con 0x55fcf782b250 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd63effd640 1 -- 172.21.15.8:0/1507544583 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd650003970 con 0x55fcf782b250 2024-02-19T16:31:52.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd63effd640 1 -- 172.21.15.8:0/1507544583 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650004530 con 0x55fcf782b250 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 -- 172.21.15.8:0/1507544583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 msgr2=0x55fcf782b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf782b630 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd65002c430 tx=0x7fd650033e80 comp rx=0 tx=0).stop 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 -- 172.21.15.8:0/1507544583 shutdown_connections 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7836b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7834210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 --2- 172.21.15.8:0/1507544583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf782b630 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 -- 172.21.15.8:0/1507544583 >> 172.21.15.8:0/1507544583 conn(0x55fcf7825980 msgr2=0x55fcf7825d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 -- 172.21.15.8:0/1507544583 shutdown_connections 2024-02-19T16:31:52.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.867+0000 7fd661642740 1 -- 172.21.15.8:0/1507544583 wait complete. 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 -- start start 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 Processor -- start 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782b250 0x55fcf7831740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7831c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf78321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf7793020 con 0x55fcf782bb70 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf7466430 con 0x55fcf7834750 2024-02-19T16:31:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf7422e00 con 0x55fcf782b250 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd661642740 10 client.0 _ll_register_callbacks cb 0x55fcf773acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd657fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7831c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd657fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7831c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59782/0 (socket says 172.21.15.8:59782) 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd657fff640 1 -- 172.21.15.8:0/413022690 learned_addr learned my addr 172.21.15.8:0/413022690 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.868+0000 7fd65d008640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf78321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd657fff640 1 -- 172.21.15.8:0/413022690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782b250 msgr2=0x55fcf7831740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd657fff640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782b250 0x55fcf7831740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd657fff640 1 -- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 msgr2=0x55fcf78321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd657fff640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf78321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd657fff640 1 -- 172.21.15.8:0/413022690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65810da80 con 0x55fcf782bb70 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd65d008640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf78321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd657fff640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7831c80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09aaf0 tx=0x7fd64c09aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c934aad51ca7bf72 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd637fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd637fff640 1 -- 172.21.15.8:0/413022690 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0a34e0 con 0x55fcf782bb70 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd637fff640 1 -- 172.21.15.8:0/413022690 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fd64c0cbd20 con 0x55fcf782bb70 2024-02-19T16:31:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcf74927e0 con 0x55fcf782bb70 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd637fff640 1 -- 172.21.15.8:0/413022690 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64c0cbee0 con 0x55fcf782bb70 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd661642740 10 client.17598 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fcf77916a0 con 0x55fcf782bb70 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd637fff640 1 -- 172.21.15.8:0/413022690 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd64c0bf080 con 0x55fcf782bb70 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.869+0000 7fd637fff640 1 -- 172.21.15.8:0/413022690 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd64c0ce2f0 con 0x55fcf782bb70 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd661642740 10 client.17598 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd661642740 10 client.17598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd661642740 20 client.17598 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd661642740 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf7844b40 0x55fcf7846f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd661642740 4 client.17598 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcf74cf720 con 0x55fcf7844b40 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd65c807640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf7844b40 0x55fcf7846f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd637fff640 10 client.17598 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.870+0000 7fd65c807640 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf7844b40 0x55fcf7846f60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de5eb41f3d8fa431 server_cookie=c39b962228d024f6 in_seq=0 out_seq=0 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd637fff640 1 -- 172.21.15.8:0/413022690 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fcf74cf720 con 0x55fcf7844b40 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd637fff640 10 client.17598 handle_command_reply: tid=0 2024-02-19T16:31:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 client.17598 shutdown 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 msgr2=0x55fcf7831c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7831c80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09aaf0 tx=0x7fd64c09aeb0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 shutdown_connections 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf7844b40 0x55fcf7846f60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf78321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7831c80 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 --2- 172.21.15.8:0/413022690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782b250 0x55fcf7831740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd637fff640 0 client.17598 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:52.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 >> 172.21.15.8:0/413022690 conn(0x55fcf7825980 msgr2=0x55fcf78276d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 shutdown_connections 2024-02-19T16:31:52.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 1 -- 172.21.15.8:0/413022690 wait complete. 2024-02-19T16:31:52.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.871+0000 7fd661642740 20 client.17598 trim_cache size 0 max 16384 2024-02-19T16:31:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 Processor -- start 2024-02-19T16:31:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 -- start start 2024-02-19T16:31:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf7831fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7827250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7827790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf7718130 con 0x55fcf782b250 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf782ac90 con 0x55fcf7834750 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd661642740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf78215c0 con 0x55fcf782bb70 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd65c807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf7831fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd65c807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf7831fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59794/0 (socket says 172.21.15.8:59794) 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd65c807640 1 -- 172.21.15.8:0/2663091655 learned_addr learned my addr 172.21.15.8:0/2663091655 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd657fff640 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7827250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.884+0000 7fd65d008640 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7827790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65c807640 1 -- 172.21.15.8:0/2663091655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 msgr2=0x55fcf7827250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65c807640 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7827250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65c807640 1 -- 172.21.15.8:0/2663091655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 msgr2=0x55fcf7827790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65c807640 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7827790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65c807640 1 -- 172.21.15.8:0/2663091655 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcf772dc20 con 0x55fcf782b250 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65d008640 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7827790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:31:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd65c807640 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf7831fe0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd650033ff0 tx=0x7fd650004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef71710843e66e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd63e7fc640 1 -- 172.21.15.8:0/2663091655 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650050270 con 0x55fcf782b250 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd63e7fc640 1 -- 172.21.15.8:0/2663091655 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd65002cde0 con 0x55fcf782b250 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd63e7fc640 1 -- 172.21.15.8:0/2663091655 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6500044c0 con 0x55fcf782b250 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd661642740 1 -- 172.21.15.8:0/2663091655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 msgr2=0x55fcf7831fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.885+0000 7fd661642740 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf7831fe0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd650033ff0 tx=0x7fd650004150 comp rx=0 tx=0).stop 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- 172.21.15.8:0/2663091655 shutdown_connections 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf7834750 0x55fcf7827790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf782bb70 0x55fcf7827250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 --2- 172.21.15.8:0/2663091655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782b250 0x55fcf7831fe0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- 172.21.15.8:0/2663091655 >> 172.21.15.8:0/2663091655 conn(0x55fcf7825980 msgr2=0x55fcf7826b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- 172.21.15.8:0/2663091655 shutdown_connections 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- 172.21.15.8:0/2663091655 wait complete. 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- start start 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 Processor -- start 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf782b250 0x55fcf783c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7838660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf7834750 0x55fcf7838ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf7839270 con 0x55fcf782bb70 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf78393e0 con 0x55fcf782b250 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf78a0440 con 0x55fcf7834750 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd661642740 10 client.0 _ll_register_callbacks cb 0x7fd64c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd657fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7838660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd657fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7838660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59800/0 (socket says 172.21.15.8:59800) 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd657fff640 1 -- 172.21.15.8:0/1502209321 learned_addr learned my addr 172.21.15.8:0/1502209321 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd65c807640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf782b250 0x55fcf783c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.886+0000 7fd65d008640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf7834750 0x55fcf7838ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd657fff640 1 -- 172.21.15.8:0/1502209321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf7834750 msgr2=0x55fcf7838ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd657fff640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf7834750 0x55fcf7838ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd657fff640 1 -- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf782b250 msgr2=0x55fcf783c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd657fff640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf782b250 0x55fcf783c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd657fff640 1 -- 172.21.15.8:0/1502209321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcf7755060 con 0x55fcf782bb70 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd657fff640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7838660 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0977c0 tx=0x7fd64c09a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c2641e236d98d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd637fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd637fff640 1 -- 172.21.15.8:0/1502209321 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0cbf20 con 0x55fcf782bb70 2024-02-19T16:31:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd637fff640 1 -- 172.21.15.8:0/1502209321 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fd64c0a34e0 con 0x55fcf782bb70 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd650003970 con 0x55fcf782bb70 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd637fff640 1 -- 172.21.15.8:0/1502209321 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64c0a3050 con 0x55fcf782bb70 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 10 client.17604 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fcf77538c0 con 0x55fcf782bb70 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd637fff640 1 -- 172.21.15.8:0/1502209321 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd650003970 con 0x55fcf782bb70 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd637fff640 1 -- 172.21.15.8:0/1502209321 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd64c099d40 con 0x55fcf782bb70 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 10 client.17604 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 10 client.17604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:31:52.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 20 client.17604 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf78a17a0 0x55fcf78a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 4 client.17604 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.887+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fcf751e890 con 0x55fcf78a17a0 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd65c807640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf78a17a0 0x55fcf78a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd65c807640 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf78a17a0 0x55fcf78a3bc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ccc553f82b0d41b server_cookie=3a871c4f0e8b5b2b in_seq=0 out_seq=0 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd637fff640 10 client.17604 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd637fff640 1 -- 172.21.15.8:0/1502209321 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fcf751e890 con 0x55fcf78a17a0 2024-02-19T16:31:52.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd637fff640 10 client.17604 handle_command_reply: tid=0 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 client.17604 shutdown 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 msgr2=0x55fcf7838660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7838660 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0977c0 tx=0x7fd64c09a520 comp rx=0 tx=0).stop 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 shutdown_connections 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fcf78a17a0 0x55fcf78a3bc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fcf7834750 0x55fcf7838ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fcf782bb70 0x55fcf7838660 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 --2- 172.21.15.8:0/1502209321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcf782b250 0x55fcf783c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd637fff640 0 client.17604 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:31:52.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 >> 172.21.15.8:0/1502209321 conn(0x55fcf7825980 msgr2=0x55fcf78273d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 shutdown_connections 2024-02-19T16:31:52.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.888+0000 7fd661642740 1 -- 172.21.15.8:0/1502209321 wait complete. 2024-02-19T16:31:52.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd661642740 20 client.17604 trim_cache size 0 max 16384 2024-02-19T16:31:52.902 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:31:52.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6400a2230 msgr2=0x7fd6400a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6400a2230 0x7fd6400a46f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55fcf7826b10 tx=0x7fd6480098c0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 msgr2=0x7fd6581025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 0x7fd6581025a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00e9f0 tx=0x7fd64c00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 shutdown_connections 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6400a2230 0x7fd6400a46f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd658102ae0 0x7fd6581a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6581056a0 0x7fd6581025a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 --2- 172.21.15.8:0/476756403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6581005d0 0x7fd658102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:52.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 >> 172.21.15.8:0/476756403 conn(0x7fd6580774f0 msgr2=0x7fd658068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:52.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 shutdown_connections 2024-02-19T16:31:52.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:52.889+0000 7fd65ea6b640 1 -- 172.21.15.8:0/476756403 wait complete. 2024-02-19T16:31:52.905 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:52 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3264886648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:31:52.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:31:52.906 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:31:53.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.008+0000 7fb52db5e640 1 -- 172.21.15.8:0/3130465327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281005d0 msgr2=0x7fb5281009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.008+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3130465327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281005d0 0x7fb5281009b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb518009930 tx=0x7fb51802bc30 comp rx=0 tx=0).stop 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.008+0000 7fb52db5e640 1 -- 172.21.15.8:0/3130465327 shutdown_connections 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.008+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3130465327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5281056a0 0x7fb52810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.008+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3130465327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528100f80 0x7fb528104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3130465327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281005d0 0x7fb5281009b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- 172.21.15.8:0/3130465327 >> 172.21.15.8:0/3130465327 conn(0x7fb5280774f0 msgr2=0x7fb528077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- 172.21.15.8:0/3130465327 shutdown_connections 2024-02-19T16:31:53.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- 172.21.15.8:0/3130465327 wait complete. 2024-02-19T16:31:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 Processor -- start 2024-02-19T16:31:53.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- start start 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5281005d0 0x7fb52819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 0x7fb52819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528197a80 0x7fb528197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528198470 con 0x7fb5281056a0 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281985e0 con 0x7fb5281005d0 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.009+0000 7fb52db5e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281987a0 con 0x7fb528197a80 2024-02-19T16:31:53.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 0x7fb52819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 0x7fb52819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59814/0 (socket says 172.21.15.8:59814) 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 -- 172.21.15.8:0/3638127680 learned_addr learned my addr 172.21.15.8:0/3638127680 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5281005d0 0x7fb52819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb527fff640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528197a80 0x7fb528197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 -- 172.21.15.8:0/3638127680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528197a80 msgr2=0x7fb528197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528197a80 0x7fb528197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 -- 172.21.15.8:0/3638127680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5281005d0 msgr2=0x7fb52819d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5281005d0 0x7fb52819d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 -- 172.21.15.8:0/3638127680 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009840 con 0x7fb5281056a0 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb526ffd640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 0x7fb52819de10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e9f0 tx=0x7fb51c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb524ff9640 1 -- 172.21.15.8:0/3638127680 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0096c0 con 0x7fb5281056a0 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb524ff9640 1 -- 172.21.15.8:0/3638127680 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51c005500 con 0x7fb5281056a0 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb518009560 con 0x7fb5281056a0 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb524ff9640 1 -- 172.21.15.8:0/3638127680 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c010430 con 0x7fb5281056a0 2024-02-19T16:31:53.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.010+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5281a4ae0 con 0x7fb5281056a0 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.012+0000 7fb524ff9640 1 -- 172.21.15.8:0/3638127680 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb51c005020 con 0x7fb5281056a0 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.012+0000 7fb530811740 1 Processor -- start 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 -- start start 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebd5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebd5b70 0x55c95ebde210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 0x55c95ebe0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95eb90960 con 0x55c95ebde750 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95eb44310 con 0x55c95ebd5250 2024-02-19T16:31:53.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb530811740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95eb41840 con 0x55c95ebd5b70 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 0x55c95ebe0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb524ff9640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5100a22b0 0x7fb5100a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebd5b70 0x55c95ebde210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 0x55c95ebe0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59816/0 (socket says 172.21.15.8:59816) 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 -- 172.21.15.8:0/3473048463 learned_addr learned my addr 172.21.15.8:0/3473048463 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb527fff640 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebd5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb527fff640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5100a22b0 0x7fb5100a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb524ff9640 1 -- 172.21.15.8:0/3638127680 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb51c014070 con 0x7fb5281056a0 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 -- 172.21.15.8:0/3473048463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebd5b70 msgr2=0x55c95ebde210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebd5b70 0x55c95ebde210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 -- 172.21.15.8:0/3473048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 msgr2=0x55c95ebd5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebd5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 -- 172.21.15.8:0/3473048463 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c95eaff060 con 0x55c95ebde750 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb5277fe640 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebd5b70 0x55c95ebde210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:53.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb527fff640 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebd5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.013+0000 7fb526ffd640 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 0x55c95ebe0b40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb51c09aa60 tx=0x7fb51c004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f3991a6bc32747 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb50dffb640 1 -- 172.21.15.8:0/3473048463 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c09b460 con 0x55c95ebde750 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb527fff640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5100a22b0 0x7fb5100a4770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb520005fd0 tx=0x7fb520004ab0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb50dffb640 1 -- 172.21.15.8:0/3473048463 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51c005670 con 0x55c95ebde750 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb50dffb640 1 -- 172.21.15.8:0/3473048463 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0046c0 con 0x55c95ebde750 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 -- 172.21.15.8:0/3473048463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 msgr2=0x55c95ebe0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 0x55c95ebe0b40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb51c09aa60 tx=0x7fb51c004a90 comp rx=0 tx=0).stop 2024-02-19T16:31:53.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 -- 172.21.15.8:0/3473048463 shutdown_connections 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebde750 0x55c95ebe0b40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebd5b70 0x55c95ebde210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 --2- 172.21.15.8:0/3473048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebd5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 -- 172.21.15.8:0/3473048463 >> 172.21.15.8:0/3473048463 conn(0x55c95ebcf980 msgr2=0x55c95ebcfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 -- 172.21.15.8:0/3473048463 shutdown_connections 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.014+0000 7fb530811740 1 -- 172.21.15.8:0/3473048463 wait complete. 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 -- start start 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 Processor -- start 2024-02-19T16:31:53.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ebdb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ebdbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebdc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95eb3d020 con 0x55c95ebd5250 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95e810430 con 0x55c95ebd5b70 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95e7cce00 con 0x55c95ebde750 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb530811740 10 client.0 _ll_register_callbacks cb 0x55c95eae4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ebdb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ebdb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59820/0 (socket says 172.21.15.8:59820) 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb5277fe640 1 -- 172.21.15.8:0/1705949803 learned_addr learned my addr 172.21.15.8:0/1705949803 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb527fff640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebdc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.015+0000 7fb526ffd640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ebdbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb5277fe640 1 -- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 msgr2=0x55c95ebdc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb5277fe640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebdc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb5277fe640 1 -- 172.21.15.8:0/1705949803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 msgr2=0x55c95ebdbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb5277fe640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ebdbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb5277fe640 1 -- 172.21.15.8:0/1705949803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5281a4ae0 con 0x55c95ebd5250 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb526ffd640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ebdbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb5277fe640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ebdb730 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb518009930 tx=0x7fb51802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a9efd778db5523 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb506ffd640 1 -- 172.21.15.8:0/1705949803 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518004050 con 0x55c95ebd5250 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb506ffd640 1 -- 172.21.15.8:0/1705949803 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb5180041b0 con 0x55c95ebd5250 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c95e83c7e0 con 0x55c95ebd5250 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb506ffd640 1 -- 172.21.15.8:0/1705949803 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb518004370 con 0x55c95ebd5250 2024-02-19T16:31:53.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb530811740 10 client.17616 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c95eb3b6a0 con 0x55c95ebd5250 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.016+0000 7fb506ffd640 1 -- 172.21.15.8:0/1705949803 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c95e83c7e0 con 0x55c95ebd5250 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb506ffd640 1 -- 172.21.15.8:0/1705949803 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb51803c040 con 0x55c95ebd5250 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb530811740 10 client.17616 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb530811740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb530811740 20 client.17616 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb530811740 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ebeea40 0x55c95ebf0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb530811740 4 client.17616 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:53.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c95e879720 con 0x55c95ebeea40 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb526ffd640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ebeea40 0x55c95ebf0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb526ffd640 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ebeea40 0x55c95ebf0e50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b0f61a53170b10 server_cookie=f42ffd4da481ad83 in_seq=0 out_seq=0 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.017+0000 7fb506ffd640 10 client.17616 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb506ffd640 1 -- 172.21.15.8:0/1705949803 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c95e879720 con 0x55c95ebeea40 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb506ffd640 10 client.17616 handle_command_reply: tid=0 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 client.17616 shutdown 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 msgr2=0x55c95ebdb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ebdb730 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb518009930 tx=0x7fb51802ca10 comp rx=0 tx=0).stop 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 shutdown_connections 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ebeea40 0x55c95ebf0e50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebdc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ebdbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 --2- 172.21.15.8:0/1705949803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ebdb730 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 >> 172.21.15.8:0/1705949803 conn(0x55c95ebcf980 msgr2=0x55c95ebd16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb506ffd640 0 client.17616 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:53.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 shutdown_connections 2024-02-19T16:31:53.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 1 -- 172.21.15.8:0/1705949803 wait complete. 2024-02-19T16:31:53.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.018+0000 7fb530811740 20 client.17616 trim_cache size 0 max 16384 2024-02-19T16:31:53.035 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:52 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3264886648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:31:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.030+0000 7fb530811740 1 Processor -- start 2024-02-19T16:31:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.030+0000 7fb530811740 1 -- start start 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb530811740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebdde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb530811740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 0x55c95ebda000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb530811740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebe9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb530811740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95eac2130 con 0x55c95ebd5b70 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb530811740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95ebd4f10 con 0x55c95ebd5250 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb530811740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95ebe11c0 con 0x55c95ebde750 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 0x55c95ebda000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 0x55c95ebda000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59836/0 (socket says 172.21.15.8:59836) 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 -- 172.21.15.8:0/2218689528 learned_addr learned my addr 172.21.15.8:0/2218689528 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb5277fe640 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebdde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb527fff640 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebe9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 -- 172.21.15.8:0/2218689528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 msgr2=0x55c95ebe9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebe9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 -- 172.21.15.8:0/2218689528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 msgr2=0x55c95ebdde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebdde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 -- 172.21.15.8:0/2218689528 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c95ead7c20 con 0x55c95ebd5b70 2024-02-19T16:31:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb5277fe640 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebdde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb526ffd640 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 0x55c95ebda000 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb51c09aea0 tx=0x7fb51c09abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61352249c9e931b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb50d7fa640 1 -- 172.21.15.8:0/2218689528 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0a3500 con 0x55c95ebd5b70 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb50d7fa640 1 -- 172.21.15.8:0/2218689528 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51c09b460 con 0x55c95ebd5b70 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.031+0000 7fb50d7fa640 1 -- 172.21.15.8:0/2218689528 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c004790 con 0x55c95ebd5b70 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 -- 172.21.15.8:0/2218689528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 msgr2=0x55c95ebda000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 0x55c95ebda000 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb51c09aea0 tx=0x7fb51c09abe0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 -- 172.21.15.8:0/2218689528 shutdown_connections 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebe9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5b70 0x55c95ebda000 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 --2- 172.21.15.8:0/2218689528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5250 0x55c95ebdde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 -- 172.21.15.8:0/2218689528 >> 172.21.15.8:0/2218689528 conn(0x55c95ebcf980 msgr2=0x55c95ebd1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 -- 172.21.15.8:0/2218689528 shutdown_connections 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 -- 172.21.15.8:0/2218689528 wait complete. 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 -- start start 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.032+0000 7fb530811740 1 Processor -- start 2024-02-19T16:31:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ec42650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ec42b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebd86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95ebd8c70 con 0x55c95ebd5250 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95ebd8de0 con 0x55c95ebd5b70 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95ebd8fa0 con 0x55c95ebde750 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 10 client.0 _ll_register_callbacks cb 0x7fb518009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ec42650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ec42650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59852/0 (socket says 172.21.15.8:59852) 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 -- 172.21.15.8:0/737720027 learned_addr learned my addr 172.21.15.8:0/737720027 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb527fff640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebd86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb526ffd640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ec42b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 -- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 msgr2=0x55c95ebd86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebd86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 -- 172.21.15.8:0/737720027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 msgr2=0x55c95ec42b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ec42b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 -- 172.21.15.8:0/737720027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c95eb3b6a0 con 0x55c95ebd5250 2024-02-19T16:31:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb5277fe640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ec42650 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb5180045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887826b86e8df25c server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb506ffd640 1 -- 172.21.15.8:0/737720027 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180035f0 con 0x55c95ebd5250 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb506ffd640 1 -- 172.21.15.8:0/737720027 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb518000e40 con 0x55c95ebd5250 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.033+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb51c005670 con 0x55c95ebd5250 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb506ffd640 1 -- 172.21.15.8:0/737720027 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb518004df0 con 0x55c95ebd5250 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 10 client.17622 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c95eaff060 con 0x55c95ebd5250 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb506ffd640 1 -- 172.21.15.8:0/737720027 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51c005670 con 0x55c95ebd5250 2024-02-19T16:31:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb506ffd640 1 -- 172.21.15.8:0/737720027 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb51803c040 con 0x55c95ebd5250 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 10 client.17622 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 20 client.17622 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ec4b340 0x55c95ec4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 4 client.17622 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c95e8c8890 con 0x55c95ec4b340 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb526ffd640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ec4b340 0x55c95ec4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:53.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb506ffd640 10 client.17622 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.034+0000 7fb526ffd640 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ec4b340 0x55c95ec4d760 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c85781dcb23e87 server_cookie=69b8e3b22bf79c24 in_seq=0 out_seq=0 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb506ffd640 1 -- 172.21.15.8:0/737720027 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c95e8c8890 con 0x55c95ec4b340 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb506ffd640 10 client.17622 handle_command_reply: tid=0 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 client.17622 shutdown 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 msgr2=0x55c95ec42650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ec42650 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb5180045d0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 shutdown_connections 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55c95ec4b340 0x55c95ec4d760 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c95ebde750 0x55c95ebd86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c95ebd5b70 0x55c95ec42b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb506ffd640 0 client.17622 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 --2- 172.21.15.8:0/737720027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c95ebd5250 0x55c95ec42650 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 >> 172.21.15.8:0/737720027 conn(0x55c95ebcf980 msgr2=0x55c95ebce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 shutdown_connections 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 1 -- 172.21.15.8:0/737720027 wait complete. 2024-02-19T16:31:53.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb530811740 20 client.17622 trim_cache size 0 max 16384 2024-02-19T16:31:53.051 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:31:53.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5100a22b0 msgr2=0x7fb5100a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5100a22b0 0x7fb5100a4770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb520005fd0 tx=0x7fb520004ab0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 msgr2=0x7fb52819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 0x7fb52819de10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e9f0 tx=0x7fb51c00edf0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 shutdown_connections 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5100a22b0 0x7fb5100a4770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528197a80 0x7fb528197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5281056a0 0x7fb52819de10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 --2- 172.21.15.8:0/3638127680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5281005d0 0x7fb52819d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.035+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 >> 172.21.15.8:0/3638127680 conn(0x7fb5280774f0 msgr2=0x7fb528068b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:53.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.036+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 shutdown_connections 2024-02-19T16:31:53.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:53.036+0000 7fb52db5e640 1 -- 172.21.15.8:0/3638127680 wait complete. 2024-02-19T16:31:53.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:52 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3264886648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:31:54.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:53 smithi008 ceph-mon[32573]: pgmap v279: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:54.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:53 smithi191 ceph-mon[37203]: pgmap v279: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:54.054 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:31:54.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:31:54.055 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:31:54.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:53 smithi156 ceph-mon[36502]: pgmap v279: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:54.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.172+0000 7f2cee593640 1 -- 172.21.15.8:0/1130748696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce8101e70 msgr2=0x7f2ce810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.172+0000 7f2cee593640 1 --2- 172.21.15.8:0/1130748696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce8101e70 0x7f2ce810e330 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009ca0 tx=0x7f2ce002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 -- 172.21.15.8:0/1130748696 shutdown_connections 2024-02-19T16:31:54.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 --2- 172.21.15.8:0/1130748696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce8101e70 0x7f2ce810e330 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 --2- 172.21.15.8:0/1130748696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce81014b0 0x7f2ce8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 --2- 172.21.15.8:0/1130748696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce8107570 0x7f2ce8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 -- 172.21.15.8:0/1130748696 >> 172.21.15.8:0/1130748696 conn(0x7f2ce80fcf20 msgr2=0x7f2ce80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 -- 172.21.15.8:0/1130748696 shutdown_connections 2024-02-19T16:31:54.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 -- 172.21.15.8:0/1130748696 wait complete. 2024-02-19T16:31:54.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 Processor -- start 2024-02-19T16:31:54.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.173+0000 7f2cee593640 1 -- start start 2024-02-19T16:31:54.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 0x7f2ce80737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce8101e70 0x7f2ce8073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce8107570 0x7f2ce8074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81a5e00 con 0x7f2ce81014b0 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81a5f70 con 0x7f2ce8101e70 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81a6130 con 0x7f2ce8107570 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 0x7f2ce80737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 0x7f2ce80737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59884/0 (socket says 172.21.15.8:59884) 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 -- 172.21.15.8:0/1109371665 learned_addr learned my addr 172.21.15.8:0/1109371665 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cecb30640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce8107570 0x7f2ce8074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce8101e70 0x7f2ce8073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 -- 172.21.15.8:0/1109371665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce8107570 msgr2=0x7f2ce8074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce8107570 0x7f2ce8074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 -- 172.21.15.8:0/1109371665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce8101e70 msgr2=0x7f2ce8073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce8101e70 0x7f2ce8073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 -- 172.21.15.8:0/1109371665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0009900 con 0x7f2ce81014b0 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 0x7f2ce80737f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800e9f0 tx=0x7f2cd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce57fa640 1 -- 172.21.15.8:0/1109371665 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8009700 con 0x7f2ce81014b0 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce57fa640 1 -- 172.21.15.8:0/1109371665 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cd800cd80 con 0x7f2ce81014b0 2024-02-19T16:31:54.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce81a6410 con 0x7f2ce81014b0 2024-02-19T16:31:54.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2ce57fa640 1 -- 172.21.15.8:0/1109371665 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8018920 con 0x7f2ce81014b0 2024-02-19T16:31:54.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.174+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81a6920 con 0x7f2ce81014b0 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2ce57fa640 1 -- 172.21.15.8:0/1109371665 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f2cd8018a80 con 0x7f2ce81014b0 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 Processor -- start 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 -- start start 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bb9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bc28b0 0x5576a7bc4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7b27f90 con 0x5576a7bb9d30 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7b254c0 con 0x5576a7bc28b0 2024-02-19T16:31:54.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cf1172740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7b20ca0 con 0x5576a7bb9380 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2ce57fa640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2cd00a2260 0x7f2cd00a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2ce7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2ce7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59890/0 (socket says 172.21.15.8:59890) 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2ce7fff640 1 -- 172.21.15.8:0/451031169 learned_addr learned my addr 172.21.15.8:0/451031169 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bc28b0 0x5576a7bc4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cecb30640 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bb9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.177+0000 7f2cecb30640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2cd00a2260 0x7f2cd00a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce57fa640 1 -- 172.21.15.8:0/1109371665 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f2cd8014070 con 0x7f2ce81014b0 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce77fe640 1 -- 172.21.15.8:0/451031169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 msgr2=0x5576a7bb9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bb9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce77fe640 1 -- 172.21.15.8:0/451031169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 msgr2=0x5576a7bc22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce77fe640 1 -- 172.21.15.8:0/451031169 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576a7ae30b0 con 0x5576a7bc28b0 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bc28b0 0x5576a7bc4d00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009bf0 tx=0x7f2cdc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a100aa91e443a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2cecb30640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2cd00a2260 0x7f2cd00a4720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004af0 tx=0x7f2ce0005c50 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2cce7fc640 1 -- 172.21.15.8:0/451031169 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc004050 con 0x5576a7bc28b0 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2cce7fc640 1 -- 172.21.15.8:0/451031169 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cdc005500 con 0x5576a7bc28b0 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.178+0000 7f2cce7fc640 1 -- 172.21.15.8:0/451031169 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc004c60 con 0x5576a7bc28b0 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 -- 172.21.15.8:0/451031169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bc28b0 msgr2=0x5576a7bc4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bc28b0 0x5576a7bc4d00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009bf0 tx=0x7f2cdc011810 comp rx=0 tx=0).stop 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 -- 172.21.15.8:0/451031169 shutdown_connections 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bc28b0 0x5576a7bc4d00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 --2- 172.21.15.8:0/451031169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bb9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 -- 172.21.15.8:0/451031169 >> 172.21.15.8:0/451031169 conn(0x5576a7bb3840 msgr2=0x5576a7bb3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 -- 172.21.15.8:0/451031169 shutdown_connections 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 -- 172.21.15.8:0/451031169 wait complete. 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 -- start start 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.179+0000 7f2cf1172740 1 Processor -- start 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bbf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9d30 0x5576a7bbfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bc0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7ac5f40 con 0x5576a7bc28b0 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a77f44b0 con 0x5576a7bb9380 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7aa6670 con 0x5576a7bb9d30 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cf1172740 10 client.0 _ll_register_callbacks cb 0x5576a7b13f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bc0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bc0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59896/0 (socket says 172.21.15.8:59896) 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9d30 0x5576a7bbfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 -- 172.21.15.8:0/4282556725 learned_addr learned my addr 172.21.15.8:0/4282556725 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bbf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 -- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9d30 msgr2=0x5576a7bbfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9d30 0x5576a7bbfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 -- 172.21.15.8:0/4282556725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 msgr2=0x5576a7bbf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bbf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 -- 172.21.15.8:0/4282556725 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc009850 con 0x5576a7bc28b0 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bbf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cecb30640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bc0290 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0036000 tx=0x7f2ce00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd46525db27cb883 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.180+0000 7f2cc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cc77fe640 1 -- 172.21.15.8:0/4282556725 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0003d80 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cc77fe640 1 -- 172.21.15.8:0/4282556725 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f2ce0032d80 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576a7afaf40 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cc77fe640 1 -- 172.21.15.8:0/4282556725 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce00347b0 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 10 client.17634 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5576a7b1f320 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cc77fe640 1 -- 172.21.15.8:0/4282556725 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5576a7afaf40 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cc77fe640 1 -- 172.21.15.8:0/4282556725 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2ce00424b0 con 0x5576a7bc28b0 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 10 client.17634 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 20 client.17634 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7bd2e80 0x5576a7bd52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 4 client.17634 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576a7b17310 con 0x5576a7bd2e80 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.181+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7bd2e80 0x5576a7bd52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.182+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7bd2e80 0x5576a7bd52a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d53768e8fcd8d33 server_cookie=5ca029debbba9d12 in_seq=0 out_seq=0 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.182+0000 7f2cc77fe640 10 client.17634 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cc77fe640 1 -- 172.21.15.8:0/4282556725 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5576a7b17310 con 0x5576a7bd2e80 2024-02-19T16:31:54.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cc77fe640 10 client.17634 handle_command_reply: tid=0 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 client.17634 shutdown 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 msgr2=0x5576a7bc0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bc0290 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0036000 tx=0x7f2ce00095c0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 shutdown_connections 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bc0290 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7bd2e80 0x5576a7bd52a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9d30 0x5576a7bbfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 --2- 172.21.15.8:0/4282556725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bbf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cc77fe640 0 client.17634 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 >> 172.21.15.8:0/4282556725 conn(0x5576a7bb3840 msgr2=0x5576a7bb57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 shutdown_connections 2024-02-19T16:31:54.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 1 -- 172.21.15.8:0/4282556725 wait complete. 2024-02-19T16:31:54.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.183+0000 7f2cf1172740 20 client.17634 trim_cache size 0 max 16384 2024-02-19T16:31:54.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.199+0000 7f2cf1172740 1 Processor -- start 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.199+0000 7f2cf1172740 1 -- start start 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bc1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9d30 0x5576a7bc00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bcddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cf1172740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7bba670 con 0x5576a7bc28b0 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cf1172740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7bb3370 con 0x5576a7bb9d30 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cf1172740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7bb4410 con 0x5576a7bb9380 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bcddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bcddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59900/0 (socket says 172.21.15.8:59900) 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 -- 172.21.15.8:0/1456104466 learned_addr learned my addr 172.21.15.8:0/1456104466 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bc1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9d30 0x5576a7bc00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 -- 172.21.15.8:0/1456104466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 msgr2=0x5576a7bc1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bc1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 -- 172.21.15.8:0/1456104466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9d30 msgr2=0x5576a7bc00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9d30 0x5576a7bc00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 -- 172.21.15.8:0/1456104466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576a7abbc00 con 0x5576a7bc28b0 2024-02-19T16:31:54.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bc1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9d30 0x5576a7bc00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2cecb30640 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bcddb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2ce002c630 tx=0x7f2ce0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d31d2798f45820 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ccdffb640 1 -- 172.21.15.8:0/1456104466 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00329e0 con 0x5576a7bc28b0 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ccdffb640 1 -- 172.21.15.8:0/1456104466 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ce0032b40 con 0x5576a7bc28b0 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.200+0000 7f2ccdffb640 1 -- 172.21.15.8:0/1456104466 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0060690 con 0x5576a7bc28b0 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 -- 172.21.15.8:0/1456104466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 msgr2=0x5576a7bcddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bcddb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2ce002c630 tx=0x7f2ce0009800 comp rx=0 tx=0).stop 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 -- 172.21.15.8:0/1456104466 shutdown_connections 2024-02-19T16:31:54.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bc28b0 0x5576a7bcddb0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9d30 0x5576a7bc00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 --2- 172.21.15.8:0/1456104466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bb9380 0x5576a7bc1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 -- 172.21.15.8:0/1456104466 >> 172.21.15.8:0/1456104466 conn(0x5576a7bb3840 msgr2=0x5576a7bb5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 -- 172.21.15.8:0/1456104466 shutdown_connections 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 -- 172.21.15.8:0/1456104466 wait complete. 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 -- start start 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.201+0000 7f2cf1172740 1 Processor -- start 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bc28b0 0x5576a7bc6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7bc7340 con 0x5576a7bb9d30 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7bc74b0 con 0x5576a7bb9380 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a7bc7670 con 0x5576a7bc28b0 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cf1172740 10 client.0 _ll_register_callbacks cb 0x7f2cdc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59906/0 (socket says 172.21.15.8:59906) 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 -- 172.21.15.8:0/3490742379 learned_addr learned my addr 172.21.15.8:0/3490742379 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cecb30640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bc28b0 0x5576a7bc6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 -- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bc28b0 msgr2=0x5576a7bc6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bc28b0 0x5576a7bc6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 -- 172.21.15.8:0/3490742379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 msgr2=0x5576a7bca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 -- 172.21.15.8:0/3490742379 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576a7b1f320 con 0x5576a7bb9d30 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2ce77fe640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc68c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc014040 tx=0x7f2cdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b75ea6f870e999d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:54.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.202+0000 7f2cc77fe640 1 -- 172.21.15.8:0/3490742379 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0037f0 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cc77fe640 1 -- 172.21.15.8:0/3490742379 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f2cdc003950 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576a7820860 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cc77fe640 1 -- 172.21.15.8:0/3490742379 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cdc0046f0 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 10 client.17640 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5576a7ae1910 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cc77fe640 1 -- 172.21.15.8:0/3490742379 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5576a7820860 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cc77fe640 1 -- 172.21.15.8:0/3490742379 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2cdc021040 con 0x5576a7bb9d30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 10 client.17640 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 20 client.17640 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7c2fdc0 0x5576a7c321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 4 client.17640 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5576a7871410 con 0x5576a7c2fdc0 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.203+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7c2fdc0 0x5576a7c321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.204+0000 7f2cc77fe640 10 client.17640 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.204+0000 7f2ce7fff640 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7c2fdc0 0x5576a7c321e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6485bcddfe4fe187 server_cookie=b5bab52d12b12d9b in_seq=0 out_seq=0 2024-02-19T16:31:54.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cc77fe640 1 -- 172.21.15.8:0/3490742379 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5576a7871410 con 0x5576a7c2fdc0 2024-02-19T16:31:54.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cc77fe640 10 client.17640 handle_command_reply: tid=0 2024-02-19T16:31:54.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 client.17640 shutdown 2024-02-19T16:31:54.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 msgr2=0x5576a7bc68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc68c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc014040 tx=0x7f2cdc0042f0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 shutdown_connections 2024-02-19T16:31:54.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5576a7c2fdc0 0x5576a7c321e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5576a7bc28b0 0x5576a7bc6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5576a7bb9d30 0x5576a7bc68c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 --2- 172.21.15.8:0/3490742379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576a7bb9380 0x5576a7bca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 >> 172.21.15.8:0/3490742379 conn(0x5576a7bb3840 msgr2=0x5576a7bb4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cc77fe640 0 client.17640 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 shutdown_connections 2024-02-19T16:31:54.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 1 -- 172.21.15.8:0/3490742379 wait complete. 2024-02-19T16:31:54.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.205+0000 7f2cf1172740 20 client.17640 trim_cache size 0 max 16384 2024-02-19T16:31:54.218 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:31:54.218 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:31:54.218 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "93edca86-b7cd-441f-8e71-d643f4cbeaad", 2024-02-19T16:31:54.219 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:31:54.219 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:31:54.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2cd00a2260 msgr2=0x7f2cd00a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2cd00a2260 0x7f2cd00a4720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004af0 tx=0x7f2ce0005c50 comp rx=0 tx=0).stop 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 msgr2=0x7f2ce80737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 0x7f2ce80737f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800e9f0 tx=0x7f2cd800eeb0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 shutdown_connections 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2cd00a2260 0x7f2cd00a4720 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce8107570 0x7f2ce8074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ce8101e70 0x7f2ce8073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 --2- 172.21.15.8:0/1109371665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ce81014b0 0x7f2ce80737f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 >> 172.21.15.8:0/1109371665 conn(0x7f2ce80fcf20 msgr2=0x7f2ce80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 shutdown_connections 2024-02-19T16:31:54.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.206+0000 7f2cee593640 1 -- 172.21.15.8:0/1109371665 wait complete. 2024-02-19T16:31:54.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:31:54.222 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:31:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 -- 172.21.15.8:0/1875480965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101490 msgr2=0x7f7110101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 --2- 172.21.15.8:0/1875480965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101490 0x7f7110101910 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009900 tx=0x7f70fc02bc30 comp rx=0 tx=0).stop 2024-02-19T16:31:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 -- 172.21.15.8:0/1875480965 shutdown_connections 2024-02-19T16:31:54.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 --2- 172.21.15.8:0/1875480965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7110101e50 0x7f711010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 --2- 172.21.15.8:0/1875480965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101490 0x7f7110101910 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 --2- 172.21.15.8:0/1875480965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 0x7f7110107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 -- 172.21.15.8:0/1875480965 >> 172.21.15.8:0/1875480965 conn(0x7f71100fcfa0 msgr2=0x7f71100ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 -- 172.21.15.8:0/1875480965 shutdown_connections 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.328+0000 7f7116cec640 1 -- 172.21.15.8:0/1875480965 wait complete. 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 Processor -- start 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 -- start start 2024-02-19T16:31:54.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7110101490 0x7f711019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 0x7f711019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 0x7f7110197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110197ec0 con 0x7f7110101e50 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110198030 con 0x7f7110107550 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7116cec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101981f0 con 0x7f7110101490 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f7115289640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 0x7f7110197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.329+0000 7f710ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 0x7f711019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 0x7f711019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59914/0 (socket says 172.21.15.8:59914) 2024-02-19T16:31:54.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f7115289640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 0x7f7110197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:44136/0 (socket says 172.21.15.8:44136) 2024-02-19T16:31:54.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 -- 172.21.15.8:0/587172751 learned_addr learned my addr 172.21.15.8:0/587172751 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f7114a88640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7110101490 0x7f711019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 -- 172.21.15.8:0/587172751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7110101490 msgr2=0x7f711019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7110101490 0x7f711019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 -- 172.21.15.8:0/587172751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 msgr2=0x7f7110197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 0x7f7110197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 -- 172.21.15.8:0/587172751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009490 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710ffff640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 0x7f711019dc00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005940 tx=0x7f70fc02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710dffb640 1 -- 172.21.15.8:0/587172751 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0042d0 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc009560 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710dffb640 1 -- 172.21.15.8:0/587172751 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70fc004430 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.330+0000 7f710dffb640 1 -- 172.21.15.8:0/587172751 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc003c50 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.331+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7110073720 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.333+0000 7f710dffb640 1 -- 172.21.15.8:0/587172751 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f70fc0054f0 con 0x7f7110101e50 2024-02-19T16:31:54.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.333+0000 7f711999b740 1 Processor -- start 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.333+0000 7f711999b740 1 -- start start 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f711999b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba60133690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f711999b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 0x55ba6013c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f711999b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f711999b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba600a2280 con 0x55ba6013c7b0 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f711999b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba6009f7b0 con 0x55ba60133bd0 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f711999b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba6009af90 con 0x55ba601332b0 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f710ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 0x55ba6013c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7115289640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba60133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f710ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59916/0 (socket says 172.21.15.8:59916) 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 0x55ba6013c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:44138/0 (socket says 172.21.15.8:44138) 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f710ffff640 1 -- 172.21.15.8:0/880500640 learned_addr learned my addr 172.21.15.8:0/880500640 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f710dffb640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 -- 172.21.15.8:0/880500640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 msgr2=0x55ba60133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba60133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 -- 172.21.15.8:0/880500640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 msgr2=0x55ba6013eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 -- 172.21.15.8:0/880500640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba60059570 con 0x55ba60133bd0 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f710ffff640 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7115289640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f710dffb640 1 -- 172.21.15.8:0/587172751 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f70fc0f36d0 con 0x7f7110101e50 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.334+0000 7f7114a88640 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 0x55ba6013c270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7104009d80 tx=0x7f7104011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=784b611e587e71f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f70faffd640 1 -- 172.21.15.8:0/880500640 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104004030 con 0x55ba60133bd0 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f70faffd640 1 -- 172.21.15.8:0/880500640 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71040035f0 con 0x55ba60133bd0 2024-02-19T16:31:54.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f70faffd640 1 -- 172.21.15.8:0/880500640 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104019700 con 0x55ba60133bd0 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 -- 172.21.15.8:0/880500640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 msgr2=0x55ba6013c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f7115289640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71000a2230 0x7f71000a46f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7108005fd0 tx=0x7f7108004ab0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 0x55ba6013c270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7104009d80 tx=0x7f7104011810 comp rx=0 tx=0).stop 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 -- 172.21.15.8:0/880500640 shutdown_connections 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba60133bd0 0x55ba6013c270 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 --2- 172.21.15.8:0/880500640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba60133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 -- 172.21.15.8:0/880500640 >> 172.21.15.8:0/880500640 conn(0x55ba6012d990 msgr2=0x55ba6012dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 -- 172.21.15.8:0/880500640 shutdown_connections 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.335+0000 7f711999b740 1 -- 172.21.15.8:0/880500640 wait complete. 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 -- start start 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 Processor -- start 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba6013c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 0x55ba601395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba60139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5fd6e430 con 0x55ba60133bd0 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5fd2ae00 con 0x55ba6013c7b0 2024-02-19T16:31:54.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba6001ffd0 con 0x55ba601332b0 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f711999b740 10 client.0 _ll_register_callbacks cb 0x55ba60075fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f710ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 0x55ba601395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.336+0000 7f7114a88640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba6013c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f710ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 0x55ba601395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59926/0 (socket says 172.21.15.8:59926) 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f710ffff640 1 -- 172.21.15.8:0/2628691229 learned_addr learned my addr 172.21.15.8:0/2628691229 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba60139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 -- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 msgr2=0x55ba6013c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba6013c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 -- 172.21.15.8:0/2628691229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 msgr2=0x55ba601395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 0x55ba601395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 -- 172.21.15.8:0/2628691229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71040099e0 con 0x55ba6013c7b0 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f710ffff640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 0x55ba601395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f7115289640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba60139cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f710800fb70 tx=0x7f7108004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df760847fe5eb284 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f70effff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f70effff640 1 -- 172.21.15.8:0/2628691229 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108010d40 con 0x55ba6013c7b0 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba5fd9a7e0 con 0x55ba6013c7b0 2024-02-19T16:31:54.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f70effff640 1 -- 172.21.15.8:0/2628691229 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7108004b20 con 0x55ba6013c7b0 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.337+0000 7f70effff640 1 -- 172.21.15.8:0/2628691229 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7108010ea0 con 0x55ba6013c7b0 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 10 client.27407 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba60099610 con 0x55ba6013c7b0 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f70effff640 1 -- 172.21.15.8:0/2628691229 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ba5fd9a7e0 con 0x55ba6013c7b0 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f70effff640 1 -- 172.21.15.8:0/2628691229 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7108028420 con 0x55ba6013c7b0 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 10 client.27407 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 20 client.27407 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:54.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba6014ca80 0x55ba6014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 4 client.27407 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:31:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba5fdd7720 con 0x55ba6014ca80 2024-02-19T16:31:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.338+0000 7f7114a88640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba6014ca80 0x55ba6014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.339+0000 7f7114a88640 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba6014ca80 0x55ba6014eea0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b6ea28deb45e82 server_cookie=2c5850c382dc13cc in_seq=0 out_seq=0 2024-02-19T16:31:54.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.339+0000 7f70effff640 10 client.27407 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f70effff640 1 -- 172.21.15.8:0/2628691229 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ba5fdd7720 con 0x55ba6014ca80 2024-02-19T16:31:54.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f70effff640 10 client.27407 handle_command_reply: tid=0 2024-02-19T16:31:54.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 client.27407 shutdown 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 msgr2=0x55ba60139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba60139cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f710800fb70 tx=0x7f7108004a40 comp rx=0 tx=0).stop 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 shutdown_connections 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba60139cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba60133bd0 0x55ba601395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba6014ca80 0x55ba6014eea0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 --2- 172.21.15.8:0/2628691229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba601332b0 0x55ba6013c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 >> 172.21.15.8:0/2628691229 conn(0x55ba6012d990 msgr2=0x55ba6012f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f70effff640 0 client.27407 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 shutdown_connections 2024-02-19T16:31:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 1 -- 172.21.15.8:0/2628691229 wait complete. 2024-02-19T16:31:54.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.340+0000 7f711999b740 20 client.27407 trim_cache size 0 max 16384 2024-02-19T16:31:54.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 Processor -- start 2024-02-19T16:31:54.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 -- start start 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 0x55ba60139540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba6013c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba6012e9e0 con 0x55ba601332b0 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba6013a590 con 0x55ba6013c7b0 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f711999b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba60137af0 con 0x55ba60133bd0 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 0x55ba60139540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 0x55ba60139540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59938/0 (socket says 172.21.15.8:59938) 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 -- 172.21.15.8:0/4053002728 learned_addr learned my addr 172.21.15.8:0/4053002728 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7115289640 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba6013c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f710ffff640 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 -- 172.21.15.8:0/4053002728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 msgr2=0x55ba6013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 -- 172.21.15.8:0/4053002728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 msgr2=0x55ba6013c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba6013c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.359+0000 7f7114a88640 1 -- 172.21.15.8:0/4053002728 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba60035ac0 con 0x55ba601332b0 2024-02-19T16:31:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.360+0000 7f7114a88640 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 0x55ba60139540 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7104005e00 tx=0x7f7104011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28429bc2b8f63a51 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.360+0000 7f70fa7fc640 1 -- 172.21.15.8:0/4053002728 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104009e70 con 0x55ba601332b0 2024-02-19T16:31:54.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.360+0000 7f70fa7fc640 1 -- 172.21.15.8:0/4053002728 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7104005520 con 0x55ba601332b0 2024-02-19T16:31:54.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.360+0000 7f70fa7fc640 1 -- 172.21.15.8:0/4053002728 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104024af0 con 0x55ba601332b0 2024-02-19T16:31:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.360+0000 7f711999b740 1 -- 172.21.15.8:0/4053002728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 msgr2=0x55ba60139540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.360+0000 7f711999b740 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 0x55ba60139540 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7104005e00 tx=0x7f7104011dc0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 -- 172.21.15.8:0/4053002728 shutdown_connections 2024-02-19T16:31:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba6013c7b0 0x55ba6013c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 --2- 172.21.15.8:0/4053002728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba601332b0 0x55ba60139540 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 -- 172.21.15.8:0/4053002728 >> 172.21.15.8:0/4053002728 conn(0x55ba6012d990 msgr2=0x55ba60138e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 -- 172.21.15.8:0/4053002728 shutdown_connections 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 -- 172.21.15.8:0/4053002728 wait complete. 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 -- start start 2024-02-19T16:31:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 Processor -- start 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba601332b0 0x55ba6013cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.361+0000 7f711999b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f711999b740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f711999b740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba601437b0 con 0x55ba6013c7b0 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f711999b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba60143920 con 0x55ba601332b0 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f711999b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba60143ae0 con 0x55ba60133bd0 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f711999b740 10 client.0 _ll_register_callbacks cb 0x7f710800fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f710ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59944/0 (socket says 172.21.15.8:59944) 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 -- 172.21.15.8:0/1795575666 learned_addr learned my addr 172.21.15.8:0/1795575666 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7114a88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba601332b0 0x55ba6013cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f710ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:56706/0 (socket says 172.21.15.8:56706) 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 -- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 msgr2=0x55ba6013d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 -- 172.21.15.8:0/1795575666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba601332b0 msgr2=0x55ba6013cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:31:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba601332b0 0x55ba6013cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 -- 172.21.15.8:0/1795575666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba60059570 con 0x55ba6013c7b0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f7115289640 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013d7b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7108012040 tx=0x7f7108028ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cb2f1973d920f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f70effff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f70effff640 1 -- 172.21.15.8:0/1795575666 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108004b20 con 0x55ba6013c7b0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f70effff640 1 -- 172.21.15.8:0/1795575666 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7108039070 con 0x55ba6013c7b0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.362+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71040035f0 con 0x55ba6013c7b0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f70effff640 1 -- 172.21.15.8:0/1795575666 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7108004c80 con 0x55ba6013c7b0 2024-02-19T16:31:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 10 client.17658 fetch_fsmap learned FSMap version 30 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba6005b6a0 con 0x55ba6013c7b0 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f70effff640 1 -- 172.21.15.8:0/1795575666 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71040035f0 con 0x55ba6013c7b0 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f70effff640 1 -- 172.21.15.8:0/1795575666 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f710803d430 con 0x55ba6013c7b0 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 10 client.17658 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 20 client.17658 populate_metadata read hostname 'smithi008' 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba601a8b80 0x55ba601aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 4 client.17658 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:31:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba5fe26890 con 0x55ba601a8b80 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.363+0000 7f7114a88640 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba601a8b80 0x55ba601aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f70effff640 10 client.17658 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f7114a88640 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba601a8b80 0x55ba601aafa0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be13e5b8a050e2a4 server_cookie=7b19db45d0affac0 in_seq=0 out_seq=0 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f70effff640 1 -- 172.21.15.8:0/1795575666 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55ba5fe26890 con 0x55ba601a8b80 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f70effff640 10 client.17658 handle_command_reply: tid=0 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f711999b740 1 client.17658 shutdown 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 msgr2=0x55ba6013d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.364+0000 7f711999b740 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013d7b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7108012040 tx=0x7f7108028ca0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 shutdown_connections 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ba6013c7b0 0x55ba6013d7b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55ba601a8b80 0x55ba601aafa0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ba60133bd0 0x55ba6013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 --2- 172.21.15.8:0/1795575666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba601332b0 0x55ba6013cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f70effff640 0 client.17658 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 >> 172.21.15.8:0/1795575666 conn(0x55ba6012d990 msgr2=0x55ba6012ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 shutdown_connections 2024-02-19T16:31:54.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 1 -- 172.21.15.8:0/1795575666 wait complete. 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f711999b740 20 client.17658 trim_cache size 0 max 16384 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2662 inodes in the stack)", 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stdout: "93edca86-b7cd-441f-8e71-d643f4cbeaad": { 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:31:54.376 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "93edca86-b7cd-441f-8e71-d643f4cbeaad", 2024-02-19T16:31:54.377 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:31:54.377 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:31:54.377 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:31:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:31:54.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71000a2230 msgr2=0x7f71000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71000a2230 0x7f71000a46f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7108005fd0 tx=0x7f7108004ab0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 msgr2=0x7f711019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 0x7f711019dc00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005940 tx=0x7f70fc02c670 comp rx=0 tx=0).stop 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 shutdown_connections 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7110107550 0x7f7110197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7110101e50 0x7f711019dc00 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f71000a2230 0x7f71000a46f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 --2- 172.21.15.8:0/587172751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7110101490 0x7f711019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 >> 172.21.15.8:0/587172751 conn(0x7f71100fcfa0 msgr2=0x7f71100fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:31:54.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.365+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 shutdown_connections 2024-02-19T16:31:54.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:31:54.366+0000 7f7116cec640 1 -- 172.21.15.8:0/587172751 wait complete. 2024-02-19T16:31:54.380 INFO:tasks.cephfs.filesystem:scrub status for tag:93edca86-b7cd-441f-8e71-d643f4cbeaad - {'path': '/', 'tag': '93edca86-b7cd-441f-8e71-d643f4cbeaad', 'options': 'recursive,force'} 2024-02-19T16:31:55.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:54 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:55.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:54 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:54 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:55.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:54 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:54 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:55.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:54 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:54 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:55.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:54 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:54 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:55.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:54 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:54 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:55.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:54 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:54 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:55.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:54 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:54 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:55.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:54 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:55.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:54 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:55.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:54 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:31:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:56.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:31:56.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:55 smithi008 ceph-mon[32573]: pgmap v280: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:31:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:56.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:31:56.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:55 smithi191 ceph-mon[37203]: pgmap v280: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:31:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:31:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:31:56.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:55 smithi156 ceph-mon[36502]: pgmap v280: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:31:58.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:57 smithi008 ceph-mon[32573]: pgmap v281: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:57 smithi191 ceph-mon[37203]: pgmap v281: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:31:58.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:57 smithi156 ceph-mon[36502]: pgmap v281: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:00.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:31:59 smithi008 ceph-mon[32573]: pgmap v282: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:31:59 smithi191 ceph-mon[37203]: pgmap v282: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:00.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:31:59 smithi156 ceph-mon[36502]: pgmap v282: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:02.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:01 smithi191 ceph-mon[37203]: pgmap v283: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:02.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:01 smithi156 ceph-mon[36502]: pgmap v283: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:02.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:01 smithi008 ceph-mon[32573]: pgmap v283: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:04.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:03 smithi191 ceph-mon[37203]: pgmap v284: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:04.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:03 smithi156 ceph-mon[36502]: pgmap v284: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:04.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:03 smithi008 ceph-mon[32573]: pgmap v284: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:04 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:32:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:04 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:32:05.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:04 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:32:05.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:04 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:32:05.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:04 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:32:05.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:04 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:32:05.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:04 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:32:05.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:04 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:32:05.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:04 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:32:05.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:04 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:32:05.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:04 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:32:05.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:04 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:32:06.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:05 smithi156 ceph-mon[36502]: pgmap v285: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:32:06.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:05 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:32:06.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:05 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:32:06.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:05 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:32:06.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:05 smithi008 ceph-mon[32573]: pgmap v285: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:32:06.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:05 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:32:06.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:05 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:32:06.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:05 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:32:06.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:05 smithi191 ceph-mon[37203]: pgmap v285: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:32:06.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:05 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:32:06.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:05 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:32:06.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:05 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:32:08.079 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:07 smithi191 ceph-mon[37203]: pgmap v286: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:32:08.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:07 smithi156 ceph-mon[36502]: pgmap v286: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:32:08.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:07 smithi008 ceph-mon[32573]: pgmap v286: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-02-19T16:32:10.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:09 smithi156 ceph-mon[36502]: pgmap v287: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:32:10.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:09 smithi008 ceph-mon[32573]: pgmap v287: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:32:10.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:09 smithi191 ceph-mon[37203]: pgmap v287: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:32:11.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:10 smithi156 ceph-mon[36502]: osdmap e103: 12 total, 12 up, 12 in 2024-02-19T16:32:11.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:10 smithi008 ceph-mon[32573]: osdmap e103: 12 total, 12 up, 12 in 2024-02-19T16:32:11.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:10 smithi191 ceph-mon[37203]: osdmap e103: 12 total, 12 up, 12 in 2024-02-19T16:32:12.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:11 smithi156 ceph-mon[36502]: pgmap v289: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:32:12.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:11 smithi008 ceph-mon[32573]: pgmap v289: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:32:12.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:11 smithi191 ceph-mon[37203]: pgmap v289: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:32:14.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:13 smithi156 ceph-mon[36502]: pgmap v290: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:32:14.270 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:13 smithi008 ceph-mon[32573]: pgmap v290: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:32:14.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:13 smithi191 ceph-mon[37203]: pgmap v290: 129 pgs: 129 active+clean; 79 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:32:16.072 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:15 smithi008 ceph-mon[32573]: pgmap v291: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:16.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:15 smithi156 ceph-mon[36502]: pgmap v291: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:16.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:15 smithi191 ceph-mon[37203]: pgmap v291: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:17 smithi008 ceph-mon[32573]: osdmap e104: 12 total, 12 up, 12 in 2024-02-19T16:32:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:17 smithi191 ceph-mon[37203]: osdmap e104: 12 total, 12 up, 12 in 2024-02-19T16:32:17.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:17 smithi156 ceph-mon[36502]: osdmap e104: 12 total, 12 up, 12 in 2024-02-19T16:32:18.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:18 smithi008 ceph-mon[32573]: pgmap v293: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:32:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:18 smithi191 ceph-mon[37203]: pgmap v293: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:32:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:18 smithi156 ceph-mon[36502]: pgmap v293: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:32:20.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:20 smithi008 ceph-mon[32573]: pgmap v294: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 229 B/s wr, 7 op/s 2024-02-19T16:32:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:20 smithi191 ceph-mon[37203]: pgmap v294: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 229 B/s wr, 7 op/s 2024-02-19T16:32:20.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:20 smithi156 ceph-mon[36502]: pgmap v294: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 229 B/s wr, 7 op/s 2024-02-19T16:32:22.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:22 smithi008 ceph-mon[32573]: pgmap v295: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:22 smithi191 ceph-mon[37203]: pgmap v295: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:22 smithi156 ceph-mon[36502]: pgmap v295: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:24.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:32:24.381 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:32:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 -- 172.21.15.8:0/2680619053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa140102470 msgr2=0x7fa140069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 --2- 172.21.15.8:0/2680619053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa140102470 0x7fa140069b60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009900 tx=0x7fa13c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 -- 172.21.15.8:0/2680619053 shutdown_connections 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 --2- 172.21.15.8:0/2680619053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa14010dde0 0x7fa1401101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 --2- 172.21.15.8:0/2680619053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14006a130 0x7fa14006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 --2- 172.21.15.8:0/2680619053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa140102470 0x7fa140069b60 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 -- 172.21.15.8:0/2680619053 >> 172.21.15.8:0/2680619053 conn(0x7fa140077190 msgr2=0x7fa1400775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 -- 172.21.15.8:0/2680619053 shutdown_connections 2024-02-19T16:32:24.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.497+0000 7fa14764e640 1 -- 172.21.15.8:0/2680619053 wait complete. 2024-02-19T16:32:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 Processor -- start 2024-02-19T16:32:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 -- start start 2024-02-19T16:32:24.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 0x7fa14019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa140102470 0x7fa14019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa14010dde0 0x7fa140199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14019a0a0 con 0x7fa14006a130 2024-02-19T16:32:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14019a210 con 0x7fa140102470 2024-02-19T16:32:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa14764e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14019a3d0 con 0x7fa14010dde0 2024-02-19T16:32:24.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa1453ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 0x7fa14019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa1453ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 0x7fa14019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58748/0 (socket says 172.21.15.8:58748) 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa1453ea640 1 -- 172.21.15.8:0/4203471364 learned_addr learned my addr 172.21.15.8:0/4203471364 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa145beb640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa14010dde0 0x7fa140199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.498+0000 7fa144be9640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa140102470 0x7fa14019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1453ea640 1 -- 172.21.15.8:0/4203471364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa14010dde0 msgr2=0x7fa140199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1453ea640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa14010dde0 0x7fa140199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1453ea640 1 -- 172.21.15.8:0/4203471364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa140102470 msgr2=0x7fa14019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1453ea640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa140102470 0x7fa14019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1453ea640 1 -- 172.21.15.8:0/4203471364 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c009560 con 0x7fa14006a130 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1453ea640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 0x7fa14019f8a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005940 tx=0x7fa13c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1367fc640 1 -- 172.21.15.8:0/4203471364 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004350 con 0x7fa14006a130 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1367fc640 1 -- 172.21.15.8:0/4203471364 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa13c0044b0 con 0x7fa14006a130 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14019a650 con 0x7fa14006a130 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa1367fc640 1 -- 172.21.15.8:0/4203471364 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c003d30 con 0x7fa14006a130 2024-02-19T16:32:24.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.499+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a6940 con 0x7fa14006a130 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.501+0000 7fa1367fc640 1 -- 172.21.15.8:0/4203471364 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fa13c004df0 con 0x7fa14006a130 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 Processor -- start 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa1367fc640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2230 0x7fa12c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 -- start start 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa1367fc640 1 -- 172.21.15.8:0/4203471364 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fa13c0f0400 con 0x7fa14006a130 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa144be9640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2230 0x7fa12c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bcb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 0x55a759bd6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759b3a280 con 0x55a759bd47b0 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759b377b0 con 0x55a759bcb2b0 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa14a221740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759b32f90 con 0x55a759bcbbd0 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa145beb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 0x55a759bd6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa1453ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bcb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa145beb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 0x55a759bd6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58750/0 (socket says 172.21.15.8:58750) 2024-02-19T16:32:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa145beb640 1 -- 172.21.15.8:0/2858209983 learned_addr learned my addr 172.21.15.8:0/2858209983 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.502+0000 7fa144be9640 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa145beb640 1 -- 172.21.15.8:0/2858209983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 msgr2=0x55a759bd4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa145beb640 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa144be9640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2230 0x7fa12c0a46f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa14010f940 tx=0x7fa130011440 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa145beb640 1 -- 172.21.15.8:0/2858209983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 msgr2=0x55a759bcb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa145beb640 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bcb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa145beb640 1 -- 172.21.15.8:0/2858209983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a759af1570 con 0x55a759bd47b0 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa1453ea640 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bcb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa145beb640 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 0x55a759bd6ba0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa138009f90 tx=0x7fa1380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bc3480cb48ae7b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa12b7fe640 1 -- 172.21.15.8:0/2858209983 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138004d20 con 0x55a759bd47b0 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa12b7fe640 1 -- 172.21.15.8:0/2858209983 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa138004e80 con 0x55a759bd47b0 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.503+0000 7fa12b7fe640 1 -- 172.21.15.8:0/2858209983 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138005280 con 0x55a759bd47b0 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 -- 172.21.15.8:0/2858209983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 msgr2=0x55a759bd6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 0x55a759bd6ba0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa138009f90 tx=0x7fa1380118d0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 -- 172.21.15.8:0/2858209983 shutdown_connections 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bd47b0 0x55a759bd6ba0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 --2- 172.21.15.8:0/2858209983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bcb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 -- 172.21.15.8:0/2858209983 >> 172.21.15.8:0/2858209983 conn(0x55a759bc5990 msgr2=0x55a759bc5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 -- 172.21.15.8:0/2858209983 shutdown_connections 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 -- 172.21.15.8:0/2858209983 wait complete. 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 -- start start 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.504+0000 7fa14a221740 1 Processor -- start 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bd4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bd1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bd47b0 0x55a759bd1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759806430 con 0x55a759bcbbd0 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7597c2e00 con 0x55a759bcb2b0 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759ab7fd0 con 0x55a759bd47b0 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 10 client.0 _ll_register_callbacks cb 0x55a759b0dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bd1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bd1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58756/0 (socket says 172.21.15.8:58756) 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 -- 172.21.15.8:0/3949135410 learned_addr learned my addr 172.21.15.8:0/3949135410 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:24.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 -- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bd47b0 msgr2=0x55a759bd1b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bd47b0 0x55a759bd1b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 -- 172.21.15.8:0/3949135410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 msgr2=0x55a759bd4190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bd4190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 -- 172.21.15.8:0/3949135410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa138009c40 con 0x55a759bcbbd0 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa144be9640 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bd1520 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa13000fa50 tx=0x7fa130004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465b9ddfb8464fdf server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa128ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa128ff9640 1 -- 172.21.15.8:0/3949135410 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13001e9e0 con 0x55a759bcbbd0 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa128ff9640 1 -- 172.21.15.8:0/3949135410 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fa13001eb40 con 0x55a759bcbbd0 2024-02-19T16:32:24.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.505+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a75986f720 con 0x55a759bcbbd0 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa128ff9640 1 -- 172.21.15.8:0/3949135410 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa13001ed00 con 0x55a759bcbbd0 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 10 client.17670 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a759b31610 con 0x55a759bcbbd0 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa128ff9640 1 -- 172.21.15.8:0/3949135410 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a75986f720 con 0x55a759bcbbd0 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa128ff9640 1 -- 172.21.15.8:0/3949135410 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa13001c430 con 0x55a759bcbbd0 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 10 client.17670 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 20 client.17670 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:24.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759be4a00 0x55a759be6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 4 client.17670 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7598be890 con 0x55a759be4a00 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.506+0000 7fa1453ea640 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759be4a00 0x55a759be6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa1453ea640 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759be4a00 0x55a759be6e60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40aa57165bceb749 server_cookie=8617201a181eea9a in_seq=0 out_seq=0 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa128ff9640 10 client.17670 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa128ff9640 1 -- 172.21.15.8:0/3949135410 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a7598be890 con 0x55a759be4a00 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa128ff9640 10 client.17670 handle_command_reply: tid=0 2024-02-19T16:32:24.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa14a221740 1 client.17670 shutdown 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 msgr2=0x55a759bd1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.507+0000 7fa14a221740 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bd1520 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa13000fa50 tx=0x7fa130004730 comp rx=0 tx=0).stop 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 shutdown_connections 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759be4a00 0x55a759be6e60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bd47b0 0x55a759bd1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bd1520 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 --2- 172.21.15.8:0/3949135410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bcb2b0 0x55a759bd4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa128ff9640 0 client.17670 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:24.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 >> 172.21.15.8:0/3949135410 conn(0x55a759bc5990 msgr2=0x55a759bc7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 shutdown_connections 2024-02-19T16:32:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 1 -- 172.21.15.8:0/3949135410 wait complete. 2024-02-19T16:32:24.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.508+0000 7fa14a221740 20 client.17670 trim_cache size 0 max 16384 2024-02-19T16:32:24.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 Processor -- start 2024-02-19T16:32:24.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 -- start start 2024-02-19T16:32:24.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 0x55a759bd0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759bc69e0 con 0x55a759bcb2b0 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759bcac80 con 0x55a759bd47b0 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa14a221740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759bcfa70 con 0x55a759bcbbd0 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa1453ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 0x55a759bd0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa1453ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 0x55a759bd0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58762/0 (socket says 172.21.15.8:58762) 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa1453ea640 1 -- 172.21.15.8:0/1236150404 learned_addr learned my addr 172.21.15.8:0/1236150404 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:24.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa144be9640 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.524+0000 7fa145beb640 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa1453ea640 1 -- 172.21.15.8:0/1236150404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 msgr2=0x55a759bd3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa1453ea640 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa1453ea640 1 -- 172.21.15.8:0/1236150404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 msgr2=0x55a759bd4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa1453ea640 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa1453ea640 1 -- 172.21.15.8:0/1236150404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a759acdac0 con 0x55a759bcb2b0 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa1453ea640 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 0x55a759bd0500 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0b8f60 tx=0x7fa13c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7344ff53a5d0d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa12affd640 1 -- 172.21.15.8:0/1236150404 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0edea0 con 0x55a759bcb2b0 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa12affd640 1 -- 172.21.15.8:0/1236150404 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa13c0c0c50 con 0x55a759bcb2b0 2024-02-19T16:32:24.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.525+0000 7fa12affd640 1 -- 172.21.15.8:0/1236150404 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0f0240 con 0x55a759bcb2b0 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- 172.21.15.8:0/1236150404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 msgr2=0x55a759bd0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 0x55a759bd0500 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0b8f60 tx=0x7fa13c02bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- 172.21.15.8:0/1236150404 shutdown_connections 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcbbd0 0x55a759bd3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- 172.21.15.8:0/1236150404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcb2b0 0x55a759bd0500 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- 172.21.15.8:0/1236150404 >> 172.21.15.8:0/1236150404 conn(0x55a759bc5990 msgr2=0x55a759bcfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- 172.21.15.8:0/1236150404 shutdown_connections 2024-02-19T16:32:24.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- 172.21.15.8:0/1236150404 wait complete. 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- start start 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 Processor -- start 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcb2b0 0x55a759bdc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bdc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759bd60e0 con 0x55a759bcbbd0 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759bd6250 con 0x55a759bd47b0 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a759bd6410 con 0x55a759bcb2b0 2024-02-19T16:32:24.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.526+0000 7fa14a221740 10 client.0 _ll_register_callbacks cb 0x7fa13000f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bdc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bdc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58774/0 (socket says 172.21.15.8:58774) 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa1453ea640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcb2b0 0x55a759bdc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 -- 172.21.15.8:0/1160360949 learned_addr learned my addr 172.21.15.8:0/1160360949 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa145beb640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 -- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcb2b0 msgr2=0x55a759bdc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcb2b0 0x55a759bdc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 -- 172.21.15.8:0/1160360949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 msgr2=0x55a759bd5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 -- 172.21.15.8:0/1160360949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14019a650 con 0x55a759bcbbd0 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa1453ea640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcb2b0 0x55a759bdc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa144be9640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bdc540 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa1300058c0 tx=0x7fa13000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42001ff1a4631a9b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa128ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa128ff9640 1 -- 172.21.15.8:0/1160360949 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13001c3a0 con 0x55a759bcbbd0 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7598327e0 con 0x55a759bcbbd0 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa128ff9640 1 -- 172.21.15.8:0/1160360949 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fa13001c500 con 0x55a759bcbbd0 2024-02-19T16:32:24.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa128ff9640 1 -- 172.21.15.8:0/1160360949 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa13001cc40 con 0x55a759bcbbd0 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa14a221740 10 client.17676 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.527+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a759af36a0 con 0x55a759bcbbd0 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa128ff9640 1 -- 172.21.15.8:0/1160360949 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa13001ecc0 con 0x55a759bcbbd0 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa128ff9640 1 -- 172.21.15.8:0/1160360949 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa13003d060 con 0x55a759bcbbd0 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa14a221740 10 client.17676 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa14a221740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa14a221740 20 client.17676 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:24.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa14a221740 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759c415c0 0x55a759c439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa14a221740 4 client.17676 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7598327e0 con 0x55a759c415c0 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa1453ea640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759c415c0 0x55a759c439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa1453ea640 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759c415c0 0x55a759c439e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3ebcb2c2b4a40b server_cookie=f35ddeff62448a1 in_seq=0 out_seq=0 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.528+0000 7fa128ff9640 10 client.17676 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa128ff9640 1 -- 172.21.15.8:0/1160360949 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7598327e0 con 0x55a759c415c0 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa128ff9640 10 client.17676 handle_command_reply: tid=0 2024-02-19T16:32:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 client.17676 shutdown 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 msgr2=0x55a759bdc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bdc540 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa1300058c0 tx=0x7fa13000f8d0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 shutdown_connections 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a759bd47b0 0x55a759bd5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a759bcbbd0 0x55a759bdc540 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55a759c415c0 0x55a759c439e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 --2- 172.21.15.8:0/1160360949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a759bcb2b0 0x55a759bdc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa128ff9640 0 client.17676 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:24.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 >> 172.21.15.8:0/1160360949 conn(0x55a759bc5990 msgr2=0x55a759bc41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 shutdown_connections 2024-02-19T16:32:24.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 1 -- 172.21.15.8:0/1160360949 wait complete. 2024-02-19T16:32:24.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14a221740 20 client.17676 trim_cache size 0 max 16384 2024-02-19T16:32:24.541 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:32:24.542 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:32:24.542 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:32:24.542 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:32:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2230 msgr2=0x7fa12c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2230 0x7fa12c0a46f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa14010f940 tx=0x7fa130011440 comp rx=0 tx=0).stop 2024-02-19T16:32:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 msgr2=0x7fa14019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 0x7fa14019f8a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005940 tx=0x7fa13c02c670 comp rx=0 tx=0).stop 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 shutdown_connections 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa12c0a2230 0x7fa12c0a46f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa14010dde0 0x7fa140199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa140102470 0x7fa14019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 --2- 172.21.15.8:0/4203471364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa14006a130 0x7fa14019f8a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.529+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 >> 172.21.15.8:0/4203471364 conn(0x7fa140077190 msgr2=0x7fa1400ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.530+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 shutdown_connections 2024-02-19T16:32:24.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.530+0000 7fa14764e640 1 -- 172.21.15.8:0/4203471364 wait complete. 2024-02-19T16:32:24.545 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:32:24.545 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:32:24.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 -- 172.21.15.8:0/1150613240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc600ffa10 msgr2=0x7fbc600ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 --2- 172.21.15.8:0/1150613240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc600ffa10 0x7fbc600ffe70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009930 tx=0x7fbc4802bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 -- 172.21.15.8:0/1150613240 shutdown_connections 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 --2- 172.21.15.8:0/1150613240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc601003b0 0x7fbc6010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 --2- 172.21.15.8:0/1150613240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc600ffa10 0x7fbc600ffe70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 --2- 172.21.15.8:0/1150613240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc60107530 0x7fbc60107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.646+0000 7fbc651f6640 1 -- 172.21.15.8:0/1150613240 >> 172.21.15.8:0/1150613240 conn(0x7fbc60076980 msgr2=0x7fbc600fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.647+0000 7fbc651f6640 1 -- 172.21.15.8:0/1150613240 shutdown_connections 2024-02-19T16:32:24.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.647+0000 7fbc651f6640 1 -- 172.21.15.8:0/1150613240 wait complete. 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.647+0000 7fbc651f6640 1 Processor -- start 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.647+0000 7fbc651f6640 1 -- start start 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc600ffa10 0x7fbc6019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc601003b0 0x7fbc6019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 0x7fbc60197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc601980d0 con 0x7fbc60107530 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60198240 con 0x7fbc600ffa10 2024-02-19T16:32:24.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60198400 con 0x7fbc601003b0 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 0x7fbc60197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 0x7fbc60197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58798/0 (socket says 172.21.15.8:58798) 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5e59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc601003b0 0x7fbc6019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 -- 172.21.15.8:0/282658671 learned_addr learned my addr 172.21.15.8:0/282658671 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5ed9d640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc600ffa10 0x7fbc6019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 -- 172.21.15.8:0/282658671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc601003b0 msgr2=0x7fbc6019de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc601003b0 0x7fbc6019de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 -- 172.21.15.8:0/282658671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc600ffa10 msgr2=0x7fbc6019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc600ffa10 0x7fbc6019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 -- 172.21.15.8:0/282658671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009560 con 0x7fbc60107530 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc5f59e640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 0x7fbc60197a80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000eb10 tx=0x7fbc5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004b20 con 0x7fbc60107530 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc50004c80 con 0x7fbc60107530 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc601986e0 con 0x7fbc60107530 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.648+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50010670 con 0x7fbc60107530 2024-02-19T16:32:24.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.649+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc601a4970 con 0x7fbc60107530 2024-02-19T16:32:24.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.649+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc60068730 con 0x7fbc60107530 2024-02-19T16:32:24.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.651+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fbc50004df0 con 0x7fbc60107530 2024-02-19T16:32:24.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.651+0000 7fbc57fff640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fbc4c0a2230 0x7fbc4c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.651+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fbc50014070 con 0x7fbc60107530 2024-02-19T16:32:24.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.651+0000 7fbc5ed9d640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fbc4c0a2230 0x7fbc4c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:24.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.652+0000 7fbc5ed9d640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fbc4c0a2230 0x7fbc4c0a46f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc60198bc0 tx=0x7fbc58005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:24.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.655+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fbc500974d0 con 0x7fbc60107530 2024-02-19T16:32:24.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:24 smithi008 ceph-mon[32573]: pgmap v296: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:24 smithi191 ceph-mon[37203]: pgmap v296: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:24.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:24 smithi156 ceph-mon[36502]: pgmap v296: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:32:24.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.877+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc60198fb0 con 0x7fbc60107530 2024-02-19T16:32:24.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.877+0000 7fbc57fff640 1 -- 172.21.15.8:0/282658671 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fbc5009ab90 con 0x7fbc60107530 2024-02-19T16:32:24.879 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:32:24.879 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:32:24.879 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:32:24.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fbc4c0a2230 msgr2=0x7fbc4c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fbc4c0a2230 0x7fbc4c0a46f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc60198bc0 tx=0x7fbc58005f90 comp rx=0 tx=0).stop 2024-02-19T16:32:24.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 msgr2=0x7fbc60197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 0x7fbc60197a80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000eb10 tx=0x7fbc5000c2d0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 shutdown_connections 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc60107530 0x7fbc60197a80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fbc4c0a2230 0x7fbc4c0a46f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc601003b0 0x7fbc6019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 --2- 172.21.15.8:0/282658671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc600ffa10 0x7fbc6019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 >> 172.21.15.8:0/282658671 conn(0x7fbc60076980 msgr2=0x7fbc600fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 shutdown_connections 2024-02-19T16:32:24.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.879+0000 7fbc651f6640 1 -- 172.21.15.8:0/282658671 wait complete. 2024-02-19T16:32:24.892 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:32:24.892 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 -- 172.21.15.8:0/4220921796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a81056c0 msgr2=0x7f52a810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 --2- 172.21.15.8:0/4220921796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a81056c0 0x7f52a810d290 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009cd0 tx=0x7f52a002c0f0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 -- 172.21.15.8:0/4220921796 shutdown_connections 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 --2- 172.21.15.8:0/4220921796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a81056c0 0x7f52a810d290 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 --2- 172.21.15.8:0/4220921796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52a8100fa0 0x7f52a8104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 --2- 172.21.15.8:0/4220921796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52a81005f0 0x7f52a81009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 -- 172.21.15.8:0/4220921796 >> 172.21.15.8:0/4220921796 conn(0x7f52a80774c0 msgr2=0x7f52a80778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:24.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 -- 172.21.15.8:0/4220921796 shutdown_connections 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.994+0000 7f52af92c640 1 -- 172.21.15.8:0/4220921796 wait complete. 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 Processor -- start 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 -- start start 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52a81005f0 0x7f52a819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52a81056c0 0x7f52a819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 0x7f52a8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81982f0 con 0x7f52a8197900 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8198460 con 0x7f52a81056c0 2024-02-19T16:32:24.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52af92c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8198620 con 0x7f52a81005f0 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 0x7f52a8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 0x7f52a8197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58828/0 (socket says 172.21.15.8:58828) 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 -- 172.21.15.8:0/2332604988 learned_addr learned my addr 172.21.15.8:0/2332604988 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52acec7640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52a81056c0 0x7f52a819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52a81005f0 0x7f52a819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 -- 172.21.15.8:0/2332604988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52a81005f0 msgr2=0x7f52a819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52a81005f0 0x7f52a819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 -- 172.21.15.8:0/2332604988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52a81056c0 msgr2=0x7f52a819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52a81056c0 0x7f52a819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.995+0000 7f52adec9640 1 -- 172.21.15.8:0/2332604988 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a0009900 con 0x7f52a8197900 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.996+0000 7f52adec9640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 0x7f52a8197db0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005e30 tx=0x7f52a0034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.996+0000 7f529e7fc640 1 -- 172.21.15.8:0/2332604988 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00328b0 con 0x7f52a8197900 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.996+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a81a4610 con 0x7f52a8197900 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.996+0000 7f529e7fc640 1 -- 172.21.15.8:0/2332604988 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52a0003e50 con 0x7f52a8197900 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.996+0000 7f529e7fc640 1 -- 172.21.15.8:0/2332604988 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0004860 con 0x7f52a8197900 2024-02-19T16:32:25.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.996+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a81a4a90 con 0x7f52a8197900 2024-02-19T16:32:25.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.998+0000 7f529e7fc640 1 -- 172.21.15.8:0/2332604988 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f52a000a040 con 0x7f52a8197900 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.998+0000 7f52b25e4740 1 Processor -- start 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.998+0000 7f52b25e4740 1 -- start start 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f501630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f529e7fc640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f52940a2230 0x7f52940a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f50a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f50cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52b25e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f4bc960 con 0x56494f501250 2024-02-19T16:32:25.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52b25e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f470310 con 0x56494f501b70 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52b25e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f46d840 con 0x56494f50a750 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52acec7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f501630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f52940a2230 0x7f52940a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52acec7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f501630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58834/0 (socket says 172.21.15.8:58834) 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52acec7640 1 -- 172.21.15.8:0/3389079665 learned_addr learned my addr 172.21.15.8:0/3389079665 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f50cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52acec7640 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f50a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f529e7fc640 1 -- 172.21.15.8:0/2332604988 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f52a00f0d30 con 0x7f52a8197900 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 -- 172.21.15.8:0/3389079665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 msgr2=0x56494f50a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f50a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 -- 172.21.15.8:0/3389079665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 msgr2=0x56494f501630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f501630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 -- 172.21.15.8:0/3389079665 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56494f42b060 con 0x56494f50a750 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52acec7640 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f501630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f52940a2230 0x7f52940a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52a8198a40 tx=0x7f52a4005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:24.999+0000 7f52adec9640 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f50cb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52a00ee130 tx=0x7f52a0034e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f962ea77b4a9b90 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52937fe640 1 -- 172.21.15.8:0/3389079665 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a004f260 con 0x56494f50a750 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52937fe640 1 -- 172.21.15.8:0/3389079665 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52a00bdad0 con 0x56494f50a750 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52937fe640 1 -- 172.21.15.8:0/3389079665 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00de340 con 0x56494f50a750 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 -- 172.21.15.8:0/3389079665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 msgr2=0x56494f50cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f50cb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52a00ee130 tx=0x7f52a0034e90 comp rx=0 tx=0).stop 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 -- 172.21.15.8:0/3389079665 shutdown_connections 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f50cb40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f50a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3389079665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f501630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 -- 172.21.15.8:0/3389079665 >> 172.21.15.8:0/3389079665 conn(0x56494f4fb980 msgr2=0x56494f4fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 -- 172.21.15.8:0/3389079665 shutdown_connections 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.000+0000 7f52b25e4740 1 -- 172.21.15.8:0/3389079665 wait complete. 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 -- start start 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 Processor -- start 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f507810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f507d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f508290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f469020 con 0x56494f501250 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f13c430 con 0x56494f501b70 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f0f8e00 con 0x56494f50a750 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52b25e4740 10 client.0 _ll_register_callbacks cb 0x56494f410cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52ad6c8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f507810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52ad6c8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f507810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58838/0 (socket says 172.21.15.8:58838) 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52ad6c8640 1 -- 172.21.15.8:0/2565252940 learned_addr learned my addr 172.21.15.8:0/2565252940 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52adec9640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f508290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.001+0000 7f52acec7640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f507d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52ad6c8640 1 -- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 msgr2=0x56494f508290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f508290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52ad6c8640 1 -- 172.21.15.8:0/2565252940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 msgr2=0x56494f507d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f507d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52ad6c8640 1 -- 172.21.15.8:0/2565252940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a81a4a90 con 0x56494f501250 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f507810 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f52a4003aa0 tx=0x7f52a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97acfb615cd8bc4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f5290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f5290ff9640 1 -- 172.21.15.8:0/2565252940 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400ed00 con 0x56494f501250 2024-02-19T16:32:25.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f5290ff9640 1 -- 172.21.15.8:0/2565252940 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f52a401e420 con 0x56494f501250 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56494f1687e0 con 0x56494f501250 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f5290ff9640 1 -- 172.21.15.8:0/2565252940 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a400ee60 con 0x56494f501250 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52b25e4740 10 client.17694 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56494f4676a0 con 0x56494f501250 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.002+0000 7f5290ff9640 1 -- 172.21.15.8:0/2565252940 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56494f1687e0 con 0x56494f501250 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f5290ff9640 1 -- 172.21.15.8:0/2565252940 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52a4024020 con 0x56494f501250 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52b25e4740 10 client.17694 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52b25e4740 10 client.17694 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:32:25.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52b25e4740 20 client.17694 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52b25e4740 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f51cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52b25e4740 4 client.17694 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56494f1a5720 con 0x56494f51aa00 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52acec7640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f51cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f5290ff9640 10 client.17694 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.003+0000 7f52acec7640 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f51cdf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ede3755ba8d18519 server_cookie=1870caeb4871854c in_seq=0 out_seq=0 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f5290ff9640 1 -- 172.21.15.8:0/2565252940 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56494f1a5720 con 0x56494f51aa00 2024-02-19T16:32:25.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f5290ff9640 10 client.17694 handle_command_reply: tid=0 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 client.17694 shutdown 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 msgr2=0x56494f507810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f507810 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f52a4003aa0 tx=0x7f52a4004730 comp rx=0 tx=0).stop 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 shutdown_connections 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f51cdf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f50a750 0x56494f508290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f5290ff9640 0 client.17694 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f507d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 --2- 172.21.15.8:0/2565252940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f501250 0x56494f507810 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 >> 172.21.15.8:0/2565252940 conn(0x56494f4fb980 msgr2=0x56494f4fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 shutdown_connections 2024-02-19T16:32:25.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 1 -- 172.21.15.8:0/2565252940 wait complete. 2024-02-19T16:32:25.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.004+0000 7f52b25e4740 20 client.17694 trim_cache size 0 max 16384 2024-02-19T16:32:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.016+0000 7f52b25e4740 1 Processor -- start 2024-02-19T16:32:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.016+0000 7f52b25e4740 1 -- start start 2024-02-19T16:32:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501250 0x56494f509f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501b70 0x56494f4f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f515890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52b25e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f3ee130 con 0x56494f50a750 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52b25e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f516e90 con 0x56494f501250 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52b25e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f4f6f50 con 0x56494f501b70 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f515890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f515890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58854/0 (socket says 172.21.15.8:58854) 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 -- 172.21.15.8:0/3240431232 learned_addr learned my addr 172.21.15.8:0/3240431232 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501250 0x56494f509f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52acec7640 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501b70 0x56494f4f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 -- 172.21.15.8:0/3240431232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501b70 msgr2=0x56494f4f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501b70 0x56494f4f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 -- 172.21.15.8:0/3240431232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501250 msgr2=0x56494f509f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501250 0x56494f509f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 -- 172.21.15.8:0/3240431232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56494f403c20 con 0x56494f50a750 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f52adec9640 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f515890 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f52a00bd360 tx=0x7f52a0004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b2569bf3e0398b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f5292ffd640 1 -- 172.21.15.8:0/3240431232 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00b9670 con 0x56494f50a750 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f5292ffd640 1 -- 172.21.15.8:0/3240431232 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52a00c5b70 con 0x56494f50a750 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.017+0000 7f5292ffd640 1 -- 172.21.15.8:0/3240431232 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00c1120 con 0x56494f50a750 2024-02-19T16:32:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 -- 172.21.15.8:0/3240431232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 msgr2=0x56494f515890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f515890 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f52a00bd360 tx=0x7f52a0004300 comp rx=0 tx=0).stop 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 -- 172.21.15.8:0/3240431232 shutdown_connections 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f515890 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501b70 0x56494f4f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 --2- 172.21.15.8:0/3240431232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501250 0x56494f509f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 -- 172.21.15.8:0/3240431232 >> 172.21.15.8:0/3240431232 conn(0x56494f4fb980 msgr2=0x56494f4fab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 -- 172.21.15.8:0/3240431232 shutdown_connections 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 -- 172.21.15.8:0/3240431232 wait complete. 2024-02-19T16:32:25.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 -- start start 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.018+0000 7f52b25e4740 1 Processor -- start 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 0x56494f504180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f5046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f504c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f575340 con 0x56494f50a750 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f5754b0 con 0x56494f501b70 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494f575670 con 0x56494f501250 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 10 client.0 _ll_register_callbacks cb 0x7f52a40040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f504c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52acec7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f5046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52ad6c8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 0x56494f504180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f504c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58868/0 (socket says 172.21.15.8:58868) 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52ad6c8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 0x56494f504180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51886/0 (socket says 172.21.15.8:51886) 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52acec7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f5046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:59974/0 (socket says 172.21.15.8:59974) 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 -- 172.21.15.8:0/1658133616 learned_addr learned my addr 172.21.15.8:0/1658133616 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 -- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 msgr2=0x56494f504180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 0x56494f504180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 -- 172.21.15.8:0/1658133616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 msgr2=0x56494f5046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f5046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 -- 172.21.15.8:0/1658133616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56494f42b060 con 0x56494f50a750 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 0x56494f504180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52adec9640 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f504c00 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f52a002c630 tx=0x7f52a00dd0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3719154fc364d6b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f5290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f5290ff9640 1 -- 172.21.15.8:0/1658133616 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00b9670 con 0x56494f50a750 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f5290ff9640 1 -- 172.21.15.8:0/1658133616 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f52a00bd3d0 con 0x56494f50a750 2024-02-19T16:32:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.019+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56494f4f70f0 con 0x56494f50a750 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f5290ff9640 1 -- 172.21.15.8:0/1658133616 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a00c1520 con 0x56494f50a750 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 10 client.17700 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56494f4298c0 con 0x56494f50a750 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f5290ff9640 1 -- 172.21.15.8:0/1658133616 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56494f4f70f0 con 0x56494f50a750 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f5290ff9640 1 -- 172.21.15.8:0/1658133616 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52a00a4070 con 0x56494f50a750 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 10 client.17700 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 10 client.17700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:32:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 20 client.17700 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f578a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 4 client.17700 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f52a00bdad0 con 0x56494f51aa00 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f578a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f52ad6c8640 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f578a80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e98f3d95cff8ce1 server_cookie=1e8729dc7d7e5034 in_seq=0 out_seq=0 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.020+0000 7f5290ff9640 10 client.17700 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f5290ff9640 1 -- 172.21.15.8:0/1658133616 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f52a00bdad0 con 0x56494f51aa00 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f5290ff9640 10 client.17700 handle_command_reply: tid=0 2024-02-19T16:32:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 client.17700 shutdown 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 msgr2=0x56494f504c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f504c00 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f52a002c630 tx=0x7f52a00dd0b0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 shutdown_connections 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x56494f51aa00 0x56494f578a80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56494f50a750 0x56494f504c00 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56494f501b70 0x56494f5046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f5290ff9640 0 client.17700 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 --2- 172.21.15.8:0/1658133616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56494f501250 0x56494f504180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 >> 172.21.15.8:0/1658133616 conn(0x56494f4fb980 msgr2=0x56494f4fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 shutdown_connections 2024-02-19T16:32:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 1 -- 172.21.15.8:0/1658133616 wait complete. 2024-02-19T16:32:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52b25e4740 20 client.17700 trim_cache size 0 max 16384 2024-02-19T16:32:25.034 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:32:25.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f52940a2230 msgr2=0x7f52940a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f52940a2230 0x7f52940a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52a8198a40 tx=0x7f52a4005f90 comp rx=0 tx=0).stop 2024-02-19T16:32:25.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 msgr2=0x7f52a8197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 0x7f52a8197db0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005e30 tx=0x7f52a0034a90 comp rx=0 tx=0).stop 2024-02-19T16:32:25.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 shutdown_connections 2024-02-19T16:32:25.036 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52a8197900 0x7f52a8197db0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52a81056c0 0x7f52a819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f52940a2230 0x7f52940a46f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 --2- 172.21.15.8:0/2332604988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52a81005f0 0x7f52a819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.021+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 >> 172.21.15.8:0/2332604988 conn(0x7f52a80774c0 msgr2=0x7f52a8069dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.022+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 shutdown_connections 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.022+0000 7f52af92c640 1 -- 172.21.15.8:0/2332604988 wait complete. 2024-02-19T16:32:25.037 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:32:25.038 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 -- 172.21.15.8:0/3837115427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 msgr2=0x7fa644101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3837115427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa644101920 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa634009900 tx=0x7fa63402bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 -- 172.21.15.8:0/3837115427 shutdown_connections 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3837115427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa644101e60 0x7fa64410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3837115427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa644101920 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3837115427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644107560 0x7fa644107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.144 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 -- 172.21.15.8:0/3837115427 >> 172.21.15.8:0/3837115427 conn(0x7fa6440fcf90 msgr2=0x7fa6440ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 -- 172.21.15.8:0/3837115427 shutdown_connections 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.140+0000 7fa64bd72640 1 -- 172.21.15.8:0/3837115427 wait complete. 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 Processor -- start 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 -- start start 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa64419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa644101e60 0x7fa64419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644107560 0x7fa6441978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644197ef0 con 0x7fa6441014a0 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644198060 con 0x7fa644107560 2024-02-19T16:32:25.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64bd72640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644198220 con 0x7fa644101e60 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa64419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa64419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58894/0 (socket says 172.21.15.8:58894) 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 -- 172.21.15.8:0/3613994890 learned_addr learned my addr 172.21.15.8:0/3613994890 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64930d640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa644101e60 0x7fa64419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644107560 0x7fa6441978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 -- 172.21.15.8:0/3613994890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa644101e60 msgr2=0x7fa64419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa644101e60 0x7fa64419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 -- 172.21.15.8:0/3613994890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644107560 msgr2=0x7fa6441978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644107560 0x7fa6441978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 -- 172.21.15.8:0/3613994890 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634009560 con 0x7fa6441014a0 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.141+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa64419d6f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa64000d950 tx=0x7fa64000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.142+0000 7fa63affd640 1 -- 172.21.15.8:0/3613994890 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640009a30 con 0x7fa6441014a0 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.142+0000 7fa63affd640 1 -- 172.21.15.8:0/3613994890 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa640009b90 con 0x7fa6441014a0 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.142+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa644198500 con 0x7fa6441014a0 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.142+0000 7fa63affd640 1 -- 172.21.15.8:0/3613994890 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640004a00 con 0x7fa6441014a0 2024-02-19T16:32:25.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.142+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6441a4790 con 0x7fa6441014a0 2024-02-19T16:32:25.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.144+0000 7fa63affd640 1 -- 172.21.15.8:0/3613994890 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fa640009d00 con 0x7fa6441014a0 2024-02-19T16:32:25.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.144+0000 7fa64ea1f740 1 Processor -- start 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 -- start start 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8805a960 con 0x55fd880acb60 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8800e310 con 0x55fd8809f8a0 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8800b840 con 0x55fd8809ef80 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa63affd640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa6300a22b0 0x7fa6300a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64a30f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64930d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64930d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58910/0 (socket says 172.21.15.8:58910) 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64930d640 1 -- 172.21.15.8:0/3583075 learned_addr learned my addr 172.21.15.8:0/3583075 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa6300a22b0 0x7fa6300a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa63affd640 1 -- 172.21.15.8:0/3613994890 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fa6400cddf0 con 0x7fa6441014a0 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 -- 172.21.15.8:0/3583075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 msgr2=0x55fd8809f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 -- 172.21.15.8:0/3583075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 msgr2=0x55fd880aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 -- 172.21.15.8:0/3583075 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd87fc9060 con 0x55fd8809f8a0 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64930d640 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa6300a22b0 0x7fa6300a4770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005c40 tx=0x7fa63c005bd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.145+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880ac620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa6400cb140 tx=0x7fa64006d450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400bd3509091614 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa62ffff640 1 -- 172.21.15.8:0/3583075 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64009aae0 con 0x55fd8809f8a0 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa62ffff640 1 -- 172.21.15.8:0/3583075 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa64009a3e0 con 0x55fd8809f8a0 2024-02-19T16:32:25.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa62ffff640 1 -- 172.21.15.8:0/3583075 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640096a50 con 0x55fd8809f8a0 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3583075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 msgr2=0x55fd880ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880ac620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa6400cb140 tx=0x7fa64006d450 comp rx=0 tx=0).stop 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3583075 shutdown_connections 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880ac620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3583075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.146+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3583075 >> 172.21.15.8:0/3583075 conn(0x55fd88099980 msgr2=0x55fd88099d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3583075 shutdown_connections 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3583075 wait complete. 2024-02-19T16:32:25.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 -- start start 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 Processor -- start 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd880a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd88007020 con 0x55fd880acb60 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd87cda430 con 0x55fd8809f8a0 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd87c96e00 con 0x55fd8809ef80 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64ea1f740 10 client.0 _ll_register_callbacks cb 0x55fd87faecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:25.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64a30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64930d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64930d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:60012/0 (socket says 172.21.15.8:60012) 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64930d640 1 -- 172.21.15.8:0/3735281668 learned_addr learned my addr 172.21.15.8:0/3735281668 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa649b0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd880a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.147+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58922/0 (socket says 172.21.15.8:58922) 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64930d640 1 -- 172.21.15.8:0/3735281668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 msgr2=0x55fd880a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64930d640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd880a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64930d640 1 -- 172.21.15.8:0/3735281668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 msgr2=0x55fd880aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64930d640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64930d640 1 -- 172.21.15.8:0/3735281668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6441a4790 con 0x55fd8809f8a0 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64930d640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa6340048d0 tx=0x7fa634004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a74fbe3e2b7cee1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa62d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3735281668 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634004aa0 con 0x55fd8809f8a0 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd87d067e0 con 0x55fd8809f8a0 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3735281668 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fa634004260 con 0x55fd8809f8a0 2024-02-19T16:32:25.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3735281668 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa634004420 con 0x55fd8809f8a0 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64ea1f740 10 client.27464 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.148+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fd880056a0 con 0x55fd8809f8a0 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3735281668 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd87d067e0 con 0x55fd8809f8a0 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3735281668 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa63403c040 con 0x55fd8809f8a0 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa64ea1f740 10 client.27464 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa64ea1f740 10 client.27464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa64ea1f740 20 client.27464 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd880bcf50 0x55fd880bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa64ea1f740 4 client.27464 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd87d43720 con 0x55fd880bcf50 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd880bcf50 0x55fd880bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd880bcf50 0x55fd880bf370 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb19013660df1e3 server_cookie=ccf6113929f48834 in_seq=0 out_seq=0 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.149+0000 7fa62d7fa640 10 client.27464 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3735281668 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fd87d43720 con 0x55fd880bcf50 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa62d7fa640 10 client.27464 handle_command_reply: tid=0 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 client.27464 shutdown 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 msgr2=0x55fd880aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa6340048d0 tx=0x7fa634004900 comp rx=0 tx=0).stop 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 shutdown_connections 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd880bcf50 0x55fd880bf370 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa62d7fa640 0 client.27464 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3735281668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd880a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 >> 172.21.15.8:0/3735281668 conn(0x55fd88099980 msgr2=0x55fd8809b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 shutdown_connections 2024-02-19T16:32:25.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3735281668 wait complete. 2024-02-19T16:32:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.150+0000 7fa64ea1f740 20 client.27464 trim_cache size 0 max 16384 2024-02-19T16:32:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 Processor -- start 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 -- start start 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd88098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd87f8c130 con 0x55fd880acb60 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8809fda0 con 0x55fd8809f8a0 2024-02-19T16:32:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8809f700 con 0x55fd8809ef80 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd88098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa649b0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd88098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58928/0 (socket says 172.21.15.8:58928) 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 -- 172.21.15.8:0/306231256 learned_addr learned my addr 172.21.15.8:0/306231256 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64930d640 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 -- 172.21.15.8:0/306231256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 msgr2=0x55fd8809b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 -- 172.21.15.8:0/306231256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 msgr2=0x55fd880aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.163+0000 7fa64a30f640 1 -- 172.21.15.8:0/306231256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd87fa1c20 con 0x55fd880acb60 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64a30f640 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd88098a00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0105b0 tx=0x7fa63c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dd1b42f4e0eb5d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa62f7fe640 1 -- 172.21.15.8:0/306231256 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c025040 con 0x55fd880acb60 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa62f7fe640 1 -- 172.21.15.8:0/306231256 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa63c016050 con 0x55fd880acb60 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa62f7fe640 1 -- 172.21.15.8:0/306231256 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0208f0 con 0x55fd880acb60 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 -- 172.21.15.8:0/306231256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 msgr2=0x55fd88098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd88098a00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0105b0 tx=0x7fa63c009500 comp rx=0 tx=0).stop 2024-02-19T16:32:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 -- 172.21.15.8:0/306231256 shutdown_connections 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd88098a00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809f8a0 0x55fd880aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/306231256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809ef80 0x55fd8809b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 -- 172.21.15.8:0/306231256 >> 172.21.15.8:0/306231256 conn(0x55fd88099980 msgr2=0x55fd880982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 -- 172.21.15.8:0/306231256 shutdown_connections 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.164+0000 7fa64ea1f740 1 -- 172.21.15.8:0/306231256 wait complete. 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 -- start start 2024-02-19T16:32:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 Processor -- start 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809ef80 0x55fd880b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809f8a0 0x55fd880b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd880a7a90 con 0x55fd880acb60 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd880a7c00 con 0x55fd8809ef80 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd88118810 con 0x55fd8809f8a0 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64ea1f740 10 client.0 _ll_register_callbacks cb 0x7fa63402c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:25.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64a30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64a30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58932/0 (socket says 172.21.15.8:58932) 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64a30f640 1 -- 172.21.15.8:0/3304994759 learned_addr learned my addr 172.21.15.8:0/3304994759 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa649b0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809ef80 0x55fd880b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.165+0000 7fa64930d640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809f8a0 0x55fd880b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64a30f640 1 -- 172.21.15.8:0/3304994759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809f8a0 msgr2=0x55fd880b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809f8a0 0x55fd880b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64a30f640 1 -- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809ef80 msgr2=0x55fd880b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809ef80 0x55fd880b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64a30f640 1 -- 172.21.15.8:0/3304994759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c00fdc0 con 0x55fd880acb60 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809ef80 0x55fd880b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64a30f640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880b2420 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa63c01fcb0 tx=0x7fa63c004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa8e3fc617cdbf5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa62d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3304994759 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c025040 con 0x55fd880acb60 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3304994759 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fa63c02a680 con 0x55fd880acb60 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa64009a3e0 con 0x55fd880acb60 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3304994759 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa63c02ae30 con 0x55fd880acb60 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 10 client.17718 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:25.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fd87fc78c0 con 0x55fd880acb60 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3304994759 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa64009a3e0 con 0x55fd880acb60 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3304994759 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa63c020cd0 con 0x55fd880acb60 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 10 client.17718 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 10 client.17718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 20 client.17718 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd88119f50 0x55fd8811c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.166+0000 7fa64ea1f740 4 client.17718 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:32:25.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd87d92890 con 0x55fd88119f50 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd88119f50 0x55fd8811c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa62d7fa640 10 client.17718 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa649b0e640 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd88119f50 0x55fd8811c370 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bab62a3c3047f7b3 server_cookie=7e5aef0c8e9e8a9d in_seq=0 out_seq=0 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa62d7fa640 1 -- 172.21.15.8:0/3304994759 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd87d92890 con 0x55fd88119f50 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa62d7fa640 10 client.17718 handle_command_reply: tid=0 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 client.17718 shutdown 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 msgr2=0x55fd880b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880b2420 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa63c01fcb0 tx=0x7fa63c004f00 comp rx=0 tx=0).stop 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 shutdown_connections 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x55fd88119f50 0x55fd8811c370 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fd880acb60 0x55fd880b2420 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8809f8a0 0x55fd880b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 --2- 172.21.15.8:0/3304994759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd8809ef80 0x55fd880b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa62d7fa640 0 client.17718 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 >> 172.21.15.8:0/3304994759 conn(0x55fd88099980 msgr2=0x55fd8809ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 shutdown_connections 2024-02-19T16:32:25.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.167+0000 7fa64ea1f740 1 -- 172.21.15.8:0/3304994759 wait complete. 2024-02-19T16:32:25.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64ea1f740 20 client.17718 trim_cache size 0 max 16384 2024-02-19T16:32:25.180 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:32:25.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa6300a22b0 msgr2=0x7fa6300a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa6300a22b0 0x7fa6300a4770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005c40 tx=0x7fa63c005bd0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 msgr2=0x7fa64419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa64419d6f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa64000d950 tx=0x7fa64000de10 comp rx=0 tx=0).stop 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 shutdown_connections 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644107560 0x7fa6441978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fa6300a22b0 0x7fa6300a4770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa644101e60 0x7fa64419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 --2- 172.21.15.8:0/3613994890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6441014a0 0x7fa64419d6f0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 >> 172.21.15.8:0/3613994890 conn(0x7fa6440fcf90 msgr2=0x7fa6440fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 shutdown_connections 2024-02-19T16:32:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.168+0000 7fa64bd72640 1 -- 172.21.15.8:0/3613994890 wait complete. 2024-02-19T16:32:25.183 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:32:25.183 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:32:25.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.285+0000 7f313c255640 1 -- 172.21.15.8:0/3067309465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134100f80 msgr2=0x7f3134104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.285+0000 7f313c255640 1 --2- 172.21.15.8:0/3067309465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134100f80 0x7f3134104fc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3128009930 tx=0x7f312802bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:25.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 -- 172.21.15.8:0/3067309465 shutdown_connections 2024-02-19T16:32:25.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 --2- 172.21.15.8:0/3067309465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31341056a0 0x7f313410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 --2- 172.21.15.8:0/3067309465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134100f80 0x7f3134104fc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 --2- 172.21.15.8:0/3067309465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 0x7f31341009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 -- 172.21.15.8:0/3067309465 >> 172.21.15.8:0/3067309465 conn(0x7f31340774f0 msgr2=0x7f3134077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 -- 172.21.15.8:0/3067309465 shutdown_connections 2024-02-19T16:32:25.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 -- 172.21.15.8:0/3067309465 wait complete. 2024-02-19T16:32:25.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 Processor -- start 2024-02-19T16:32:25.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 -- start start 2024-02-19T16:32:25.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 0x7f3134198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.286+0000 7f313c255640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31341056a0 0x7f3134198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313c255640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 0x7f31341a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313c255640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341a40a0 con 0x7f3134199450 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313c255640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341a4210 con 0x7f31341056a0 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313c255640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341a43d0 con 0x7f31341005d0 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 0x7f31341a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 0x7f31341a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58938/0 (socket says 172.21.15.8:58938) 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 -- 172.21.15.8:0/1182788263 learned_addr learned my addr 172.21.15.8:0/1182788263 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f3139ff1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 0x7f3134198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f31397f0640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31341056a0 0x7f3134198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 -- 172.21.15.8:0/1182788263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 msgr2=0x7f3134198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 0x7f3134198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 -- 172.21.15.8:0/1182788263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31341056a0 msgr2=0x7f3134198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31341056a0 0x7f3134198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 -- 172.21.15.8:0/1182788263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009560 con 0x7f3134199450 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f3139ff1640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 0x7f3134198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313a7f2640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 0x7f31341a3a00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f312400eaa0 tx=0x7f312400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f3132ffd640 1 -- 172.21.15.8:0/1182788263 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400cdb0 con 0x7f3134199450 2024-02-19T16:32:25.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31341a46b0 con 0x7f3134199450 2024-02-19T16:32:25.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f3132ffd640 1 -- 172.21.15.8:0/1182788263 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3124005020 con 0x7f3134199450 2024-02-19T16:32:25.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f3132ffd640 1 -- 172.21.15.8:0/1182788263 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240116b0 con 0x7f3134199450 2024-02-19T16:32:25.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.287+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31341a4b40 con 0x7f3134199450 2024-02-19T16:32:25.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.289+0000 7f3132ffd640 1 -- 172.21.15.8:0/1182788263 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f3124005520 con 0x7f3134199450 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 Processor -- start 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 -- start start 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b874960 con 0x55f81b8b9250 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b828310 con 0x55f81b8c2750 2024-02-19T16:32:25.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f313ee32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b825840 con 0x55f81b8b9b70 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f3132ffd640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f311c0a2230 0x7f311c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.290+0000 7f31397f0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f31397f0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8b9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58940/0 (socket says 172.21.15.8:58940) 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f31397f0640 1 -- 172.21.15.8:0/4191530858 learned_addr learned my addr 172.21.15.8:0/4191530858 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f313a7f2640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f311c0a2230 0x7f311c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f31397f0640 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3132ffd640 1 -- 172.21.15.8:0/1182788263 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f31240ce6c0 con 0x7f3134199450 2024-02-19T16:32:25.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 -- 172.21.15.8:0/4191530858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 msgr2=0x55f81b8c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 -- 172.21.15.8:0/4191530858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 msgr2=0x55f81b8b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 -- 172.21.15.8:0/4191530858 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f81b7e3060 con 0x55f81b8b9b70 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f31397f0640 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f313a7f2640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f311c0a2230 0x7f311c0a46f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f31240092f0 tx=0x7f3124069ea0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f3139ff1640 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c2210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f312c0103d0 tx=0x7f312c010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ae339716c4e740 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f311bfff640 1 -- 172.21.15.8:0/4191530858 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c004220 con 0x55f81b8b9b70 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f311bfff640 1 -- 172.21.15.8:0/4191530858 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312c015070 con 0x55f81b8b9b70 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.291+0000 7f311bfff640 1 -- 172.21.15.8:0/4191530858 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0118e0 con 0x55f81b8b9b70 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 -- 172.21.15.8:0/4191530858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 msgr2=0x55f81b8c2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c2210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f312c0103d0 tx=0x7f312c010890 comp rx=0 tx=0).stop 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 -- 172.21.15.8:0/4191530858 shutdown_connections 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c2210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 --2- 172.21.15.8:0/4191530858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 -- 172.21.15.8:0/4191530858 >> 172.21.15.8:0/4191530858 conn(0x55f81b8b3980 msgr2=0x55f81b8b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 -- 172.21.15.8:0/4191530858 shutdown_connections 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 -- 172.21.15.8:0/4191530858 wait complete. 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 -- start start 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.292+0000 7f313ee32740 1 Processor -- start 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b821020 con 0x55f81b8b9b70 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b4f4430 con 0x55f81b8c2750 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b4b0e00 con 0x55f81b8b9250 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f313ee32740 10 client.0 _ll_register_callbacks cb 0x55f81b7c8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f31397f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f31397f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8bf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:51994/0 (socket says 172.21.15.8:51994) 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f31397f0640 1 -- 172.21.15.8:0/3016509058 learned_addr learned my addr 172.21.15.8:0/3016509058 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8bfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58952/0 (socket says 172.21.15.8:58952) 2024-02-19T16:32:25.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f31397f0640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 -- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 msgr2=0x55f81b8bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 -- 172.21.15.8:0/3016509058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 msgr2=0x55f81b8c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 -- 172.21.15.8:0/3016509058 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c00bc10 con 0x55f81b8b9b70 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f31397f0640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f3139ff1640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8bfd20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f312c000c00 tx=0x7f312c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e8babfc2d6a303 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.293+0000 7f31197fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f31197fa640 1 -- 172.21.15.8:0/3016509058 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c021070 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f31197fa640 1 -- 172.21.15.8:0/3016509058 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f312c017070 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f81b5207e0 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f31197fa640 1 -- 172.21.15.8:0/3016509058 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312c003fa0 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 10 client.17730 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f81b81f6a0 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f31197fa640 1 -- 172.21.15.8:0/3016509058 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f81b5207e0 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f31197fa640 1 -- 172.21.15.8:0/3016509058 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f312c004750 con 0x55f81b8b9b70 2024-02-19T16:32:25.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 10 client.17730 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 20 client.17730 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b8d2af0 0x55f81b8d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 4 client.17730 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f81b55d720 con 0x55f81b8d2af0 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.294+0000 7f31397f0640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b8d2af0 0x55f81b8d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f31197fa640 10 client.17730 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:25.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f31397f0640 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b8d2af0 0x55f81b8d4f10 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7863e9abed765374 server_cookie=783070db07c273d5 in_seq=0 out_seq=0 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f31197fa640 1 -- 172.21.15.8:0/3016509058 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f81b55d720 con 0x55f81b8d2af0 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f31197fa640 10 client.17730 handle_command_reply: tid=0 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f313ee32740 1 client.17730 shutdown 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 msgr2=0x55f81b8bfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.295+0000 7f313ee32740 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8bfd20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f312c000c00 tx=0x7f312c003820 comp rx=0 tx=0).stop 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 shutdown_connections 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8bfd20 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b8d2af0 0x55f81b8d4f10 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 --2- 172.21.15.8:0/3016509058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f31197fa640 0 client.17730 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:25.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 >> 172.21.15.8:0/3016509058 conn(0x55f81b8b3980 msgr2=0x55f81b8b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 shutdown_connections 2024-02-19T16:32:25.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 1 -- 172.21.15.8:0/3016509058 wait complete. 2024-02-19T16:32:25.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.296+0000 7f313ee32740 20 client.17730 trim_cache size 0 max 16384 2024-02-19T16:32:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.308+0000 7f313ee32740 1 Processor -- start 2024-02-19T16:32:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.308+0000 7f313ee32740 1 -- start start 2024-02-19T16:32:25.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f313ee32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f313ee32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f313ee32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f313ee32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b7a6130 con 0x55f81b8b9250 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f313ee32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b8af2b0 con 0x55f81b8c2750 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f313ee32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b8d5570 con 0x55f81b8b9b70 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8c1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58954/0 (socket says 172.21.15.8:58954) 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 -- 172.21.15.8:0/3259686595 learned_addr learned my addr 172.21.15.8:0/3259686595 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f3139ff1640 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 -- 172.21.15.8:0/3259686595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 msgr2=0x55f81b8c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 -- 172.21.15.8:0/3259686595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 msgr2=0x55f81b8b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 -- 172.21.15.8:0/3259686595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f81b7bbc20 con 0x55f81b8b9250 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f31397f0640 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8c1ea0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3128009230 tx=0x7f3128004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8076e323fae34e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f311b7fe640 1 -- 172.21.15.8:0/3259686595 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312802cbb0 con 0x55f81b8b9250 2024-02-19T16:32:25.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f311b7fe640 1 -- 172.21.15.8:0/3259686595 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3128049040 con 0x55f81b8b9250 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.309+0000 7f311b7fe640 1 -- 172.21.15.8:0/3259686595 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31280506c0 con 0x55f81b8b9250 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 -- 172.21.15.8:0/3259686595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 msgr2=0x55f81b8c1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8c1ea0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3128009230 tx=0x7f3128004970 comp rx=0 tx=0).stop 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 -- 172.21.15.8:0/3259686595 shutdown_connections 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9b70 0x55f81b8c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 --2- 172.21.15.8:0/3259686595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9250 0x55f81b8c1ea0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 -- 172.21.15.8:0/3259686595 >> 172.21.15.8:0/3259686595 conn(0x55f81b8b3980 msgr2=0x55f81b8b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 -- 172.21.15.8:0/3259686595 shutdown_connections 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 -- 172.21.15.8:0/3259686595 wait complete. 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 -- start start 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.310+0000 7f313ee32740 1 Processor -- start 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b8c6880 con 0x55f81b8b9b70 2024-02-19T16:32:25.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b8c69f0 con 0x55f81b8c2750 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81b8c6bb0 con 0x55f81b8b9250 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 10 client.0 _ll_register_callbacks cb 0x7f3128005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58956/0 (socket says 172.21.15.8:58956) 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 -- 172.21.15.8:0/1246761439 learned_addr learned my addr 172.21.15.8:0/1246761439 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f31397f0640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f31397f0640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 -- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 msgr2=0x55f81b8c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 -- 172.21.15.8:0/1246761439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 msgr2=0x55f81b8c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 -- 172.21.15.8:0/1246761439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313410da80 con 0x55f81b8b9b70 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f3139ff1640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8ca2e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c010 tx=0x7f312c003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b71b6395f1fdffd server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f31197fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f31197fa640 1 -- 172.21.15.8:0/1246761439 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c021070 con 0x55f81b8b9b70 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f31197fa640 1 -- 172.21.15.8:0/1246761439 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f312c017030 con 0x55f81b8b9b70 2024-02-19T16:32:25.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.311+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f312c015070 con 0x55f81b8b9b70 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f31197fa640 1 -- 172.21.15.8:0/1246761439 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312c004870 con 0x55f81b8b9b70 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 10 client.17736 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f81b7e18c0 con 0x55f81b8b9b70 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f31197fa640 1 -- 172.21.15.8:0/1246761439 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312c015070 con 0x55f81b8b9b70 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f31197fa640 1 -- 172.21.15.8:0/1246761439 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f312c02d040 con 0x55f81b8b9b70 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 10 client.17736 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 20 client.17736 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:25.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b92f330 0x55f81b931750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 4 client.17736 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f81b5ac890 con 0x55f81b92f330 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.312+0000 7f31397f0640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b92f330 0x55f81b931750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f31197fa640 10 client.17736 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f31397f0640 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b92f330 0x55f81b931750 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e415f0412bd3e2 server_cookie=860bc9f247c18afe in_seq=0 out_seq=0 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f31197fa640 1 -- 172.21.15.8:0/1246761439 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f81b5ac890 con 0x55f81b92f330 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f31197fa640 10 client.17736 handle_command_reply: tid=0 2024-02-19T16:32:25.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 client.17736 shutdown 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 msgr2=0x55f81b8ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8ca2e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c010 tx=0x7f312c003890 comp rx=0 tx=0).stop 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 shutdown_connections 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f81b8c2750 0x55f81b8c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f81b8b9b70 0x55f81b8ca2e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x55f81b92f330 0x55f81b931750 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 --2- 172.21.15.8:0/1246761439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f81b8b9250 0x55f81b8c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f31197fa640 0 client.17736 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:25.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 >> 172.21.15.8:0/1246761439 conn(0x55f81b8b3980 msgr2=0x55f81b8b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 shutdown_connections 2024-02-19T16:32:25.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 1 -- 172.21.15.8:0/1246761439 wait complete. 2024-02-19T16:32:25.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.313+0000 7f313ee32740 20 client.17736 trim_cache size 0 max 16384 2024-02-19T16:32:25.326 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:32:25.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f311c0a2230 msgr2=0x7f311c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f311c0a2230 0x7f311c0a46f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f31240092f0 tx=0x7f3124069ea0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 msgr2=0x7f31341a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 0x7f31341a3a00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f312400eaa0 tx=0x7f312400ef60 comp rx=0 tx=0).stop 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 shutdown_connections 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134199450 0x7f31341a3a00 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31341056a0 0x7f3134198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f311c0a2230 0x7f311c0a46f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 --2- 172.21.15.8:0/1182788263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31341005d0 0x7f3134198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:25.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 >> 172.21.15.8:0/1182788263 conn(0x7f31340774f0 msgr2=0x7f31340fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:25.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 shutdown_connections 2024-02-19T16:32:25.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:25.314+0000 7f313c255640 1 -- 172.21.15.8:0/1182788263 wait complete. 2024-02-19T16:32:25.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:25 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/282658671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:32:25.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:25 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/282658671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:32:25.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:25 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/282658671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:32:26.330 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-19T16:32:26.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:32:26.331 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 -- 172.21.15.8:0/2893729782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4101e70 msgr2=0x7f41f410e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 --2- 172.21.15.8:0/2893729782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4101e70 0x7f41f410e330 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009ca0 tx=0x7f41ec02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 -- 172.21.15.8:0/2893729782 shutdown_connections 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 --2- 172.21.15.8:0/2893729782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4101e70 0x7f41f410e330 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 --2- 172.21.15.8:0/2893729782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f41014b0 0x7f41f4101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 --2- 172.21.15.8:0/2893729782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f4107570 0x7f41f4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 -- 172.21.15.8:0/2893729782 >> 172.21.15.8:0/2893729782 conn(0x7f41f40fcf20 msgr2=0x7f41f40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 -- 172.21.15.8:0/2893729782 shutdown_connections 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.448+0000 7f41fb48f640 1 -- 172.21.15.8:0/2893729782 wait complete. 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 Processor -- start 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 -- start start 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f41014b0 0x7f41f419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f4101e70 0x7f41f419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 0x7f41f4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4197eb0 con 0x7f41f4107570 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4198020 con 0x7f41f41014b0 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41fb48f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41981e0 con 0x7f41f4101e70 2024-02-19T16:32:26.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f9a2c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 0x7f41f4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f922b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f41014b0 0x7f41f419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f4101e70 0x7f41f419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f922b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f41014b0 0x7f41f419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:45734/0 (socket says 172.21.15.8:45734) 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f9a2c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 0x7f41f4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35732/0 (socket says 172.21.15.8:35732) 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f922b640 1 -- 172.21.15.8:0/4164815682 learned_addr learned my addr 172.21.15.8:0/4164815682 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f8a2a640 1 -- 172.21.15.8:0/4164815682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f41014b0 msgr2=0x7f41f419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f41014b0 0x7f41f419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f8a2a640 1 -- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 msgr2=0x7f41f4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 0x7f41f4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f8a2a640 1 -- 172.21.15.8:0/4164815682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009900 con 0x7f41f4101e70 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.449+0000 7f41f9a2c640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 0x7f41f4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.450+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f4101e70 0x7f41f419dbe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.450+0000 7f41ea7fc640 1 -- 172.21.15.8:0/4164815682 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400ccc0 con 0x7f41f4101e70 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.450+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f41984c0 con 0x7f41f4101e70 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.450+0000 7f41ea7fc640 1 -- 172.21.15.8:0/4164815682 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41e400ce20 con 0x7f41f4101e70 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.450+0000 7f41ea7fc640 1 -- 172.21.15.8:0/4164815682 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4018920 con 0x7f41f4101e70 2024-02-19T16:32:26.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.450+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f41a46d0 con 0x7f41f4101e70 2024-02-19T16:32:26.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.452+0000 7f41ea7fc640 1 -- 172.21.15.8:0/4164815682 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f41e4018a80 con 0x7f41f4101e70 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.452+0000 7f41fe066740 1 Processor -- start 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 -- start start 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a465cf90 con 0x5592a46eed30 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a465a4c0 con 0x5592a46f78b0 2024-02-19T16:32:26.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41fe066740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a4655ca0 con 0x5592a46ee380 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41ea7fc640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41e00a2230 0x7f41e00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f9a2c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35748/0 (socket says 172.21.15.8:35748) 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 -- 172.21.15.8:0/2450428710 learned_addr learned my addr 172.21.15.8:0/2450428710 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f9a2c640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41e00a2230 0x7f41e00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41ea7fc640 1 -- 172.21.15.8:0/4164815682 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f41e4014070 con 0x7f41f4101e70 2024-02-19T16:32:26.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 -- 172.21.15.8:0/2450428710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 msgr2=0x5592a46ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 -- 172.21.15.8:0/2450428710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 msgr2=0x5592a46f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 -- 172.21.15.8:0/2450428710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592a46180b0 con 0x5592a46eed30 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f922b640 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f72e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009dc0 tx=0x7f41f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5c309429826832 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.453+0000 7f41f9a2c640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41e00a2230 0x7f41e00a46f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41ec004af0 tx=0x7f41ec009360 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41df7fe640 1 -- 172.21.15.8:0/2450428710 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0005220 con 0x5592a46eed30 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41df7fe640 1 -- 172.21.15.8:0/2450428710 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41f0004c80 con 0x5592a46eed30 2024-02-19T16:32:26.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41df7fe640 1 -- 172.21.15.8:0/2450428710 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00188b0 con 0x5592a46eed30 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 -- 172.21.15.8:0/2450428710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 msgr2=0x5592a46f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f72e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009dc0 tx=0x7f41f0011810 comp rx=0 tx=0).stop 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 -- 172.21.15.8:0/2450428710 shutdown_connections 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f72e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 --2- 172.21.15.8:0/2450428710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 -- 172.21.15.8:0/2450428710 >> 172.21.15.8:0/2450428710 conn(0x5592a46e8840 msgr2=0x5592a46e8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 -- 172.21.15.8:0/2450428710 shutdown_connections 2024-02-19T16:32:26.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.454+0000 7f41fe066740 1 -- 172.21.15.8:0/2450428710 wait complete. 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 -- start start 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 Processor -- start 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a45faf40 con 0x5592a46eed30 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a43294b0 con 0x5592a46f78b0 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a45db670 con 0x5592a46ee380 2024-02-19T16:32:26.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41fe066740 10 client.0 _ll_register_callbacks cb 0x5592a4648f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35758/0 (socket says 172.21.15.8:35758) 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2914210761 learned_addr learned my addr 172.21.15.8:0/2914210761 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41f922b640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.455+0000 7f41f9a2c640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 msgr2=0x5592a46f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2914210761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 msgr2=0x5592a46f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2914210761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f00099f0 con 0x5592a46eed30 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f4cb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f41e409ad70 tx=0x7f41e409a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8993ed32775e2b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41dcff9640 1 -- 172.21.15.8:0/2914210761 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40a2e30 con 0x5592a46eed30 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41dcff9640 1 -- 172.21.15.8:0/2914210761 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f41e409ade0 con 0x5592a46eed30 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592a462ff40 con 0x5592a46eed30 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41dcff9640 1 -- 172.21.15.8:0/2914210761 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e409ee80 con 0x5592a46eed30 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 10 client.17748 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:26.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5592a4654320 con 0x5592a46eed30 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41dcff9640 1 -- 172.21.15.8:0/2914210761 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5592a462ff40 con 0x5592a46eed30 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41dcff9640 1 -- 172.21.15.8:0/2914210761 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f41e40b90e0 con 0x5592a46eed30 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 10 client.17748 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 20 client.17748 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4707dd0 0x5592a470a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.456+0000 7f41fe066740 4 client.17748 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.457+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592a464c310 con 0x5592a4707dd0 2024-02-19T16:32:26.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.457+0000 7f41f922b640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4707dd0 0x5592a470a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.457+0000 7f41f922b640 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4707dd0 0x5592a470a1f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b664eaae5d45ea39 server_cookie=9f76d46b670ed8b0 in_seq=0 out_seq=0 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.457+0000 7f41dcff9640 10 client.17748 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41dcff9640 1 -- 172.21.15.8:0/2914210761 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5592a464c310 con 0x5592a4707dd0 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41dcff9640 10 client.17748 handle_command_reply: tid=0 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 client.17748 shutdown 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 msgr2=0x5592a46f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f4cb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f41e409ad70 tx=0x7f41e409a9e0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 shutdown_connections 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46f4cb0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4707dd0 0x5592a470a1f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 --2- 172.21.15.8:0/2914210761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41dcff9640 0 client.17748 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 >> 172.21.15.8:0/2914210761 conn(0x5592a46e8840 msgr2=0x5592a46ea7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 shutdown_connections 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 1 -- 172.21.15.8:0/2914210761 wait complete. 2024-02-19T16:32:26.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.458+0000 7f41fe066740 20 client.17748 trim_cache size 0 max 16384 2024-02-19T16:32:26.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 Processor -- start 2024-02-19T16:32:26.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 -- start start 2024-02-19T16:32:26.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a46fa700 con 0x5592a46eed30 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a46fa0c0 con 0x5592a46f78b0 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41fe066740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a470a850 con 0x5592a46ee380 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35770/0 (socket says 172.21.15.8:35770) 2024-02-19T16:32:26.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 -- 172.21.15.8:0/3073400139 learned_addr learned my addr 172.21.15.8:0/3073400139 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f9a2c640 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f922b640 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 -- 172.21.15.8:0/3073400139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 msgr2=0x5592a46f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 -- 172.21.15.8:0/3073400139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 msgr2=0x5592a46f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.476+0000 7f41f8a2a640 1 -- 172.21.15.8:0/3073400139 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592a45f0c00 con 0x5592a46eed30 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46efec0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f41e4099090 tx=0x7f41e409aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13e709c6fb009dc server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.481 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41deffd640 1 -- 172.21.15.8:0/3073400139 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40b9290 con 0x5592a46eed30 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41deffd640 1 -- 172.21.15.8:0/3073400139 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41e40a2e30 con 0x5592a46eed30 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41deffd640 1 -- 172.21.15.8:0/3073400139 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40cbd10 con 0x5592a46eed30 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 -- 172.21.15.8:0/3073400139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 msgr2=0x5592a46efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46efec0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f41e4099090 tx=0x7f41e409aa50 comp rx=0 tx=0).stop 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 -- 172.21.15.8:0/3073400139 shutdown_connections 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46f78b0 0x5592a46f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46efec0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 --2- 172.21.15.8:0/3073400139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46ee380 0x5592a46f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.477+0000 7f41fe066740 1 -- 172.21.15.8:0/3073400139 >> 172.21.15.8:0/3073400139 conn(0x5592a46e8840 msgr2=0x5592a46e7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.482 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 -- 172.21.15.8:0/3073400139 shutdown_connections 2024-02-19T16:32:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 -- 172.21.15.8:0/3073400139 wait complete. 2024-02-19T16:32:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 -- start start 2024-02-19T16:32:26.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 Processor -- start 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46ee380 0x5592a46fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46fb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46f78b0 0x5592a46fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a47627a0 con 0x5592a46eed30 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a4762910 con 0x5592a46ee380 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592a4762ad0 con 0x5592a46f78b0 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41fe066740 10 client.0 _ll_register_callbacks cb 0x7f41e400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:26.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41f8a2a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46fb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35772/0 (socket says 172.21.15.8:35772) 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.478+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2043752384 learned_addr learned my addr 172.21.15.8:0/2043752384 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f9a2c640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46f78b0 0x5592a46fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f922b640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46ee380 0x5592a46fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46f78b0 msgr2=0x5592a46fba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46f78b0 0x5592a46fba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2043752384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46ee380 msgr2=0x5592a46fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46ee380 0x5592a46fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f8a2a640 1 -- 172.21.15.8:0/2043752384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592a4654320 con 0x5592a46eed30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41f8a2a640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46fb500 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f41e4097240 tx=0x7f41e40cbb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e714924c52d2ed22 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41dcff9640 1 -- 172.21.15.8:0/2043752384 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40b9290 con 0x5592a46eed30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41dcff9640 1 -- 172.21.15.8:0/2043752384 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f41e40a3280 con 0x5592a46eed30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592a467b900 con 0x5592a46eed30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41dcff9640 1 -- 172.21.15.8:0/2043752384 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e4084120 con 0x5592a46eed30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41fe066740 10 client.17754 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5592a4616910 con 0x5592a46eed30 2024-02-19T16:32:26.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.479+0000 7f41dcff9640 1 -- 172.21.15.8:0/2043752384 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41e40a7020 con 0x5592a46eed30 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41dcff9640 1 -- 172.21.15.8:0/2043752384 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f41e40ce450 con 0x5592a46eed30 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41fe066740 10 client.17754 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41fe066740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41fe066740 20 client.17754 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41fe066740 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4763df0 0x5592a4766210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41fe066740 4 client.17754 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5592a4355860 con 0x5592a4763df0 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41f922b640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4763df0 0x5592a4766210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41dcff9640 10 client.17754 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.480+0000 7f41f922b640 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4763df0 0x5592a4766210 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8dfd00dd82ccd68 server_cookie=b64b989f60f684c9 in_seq=0 out_seq=0 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41dcff9640 1 -- 172.21.15.8:0/2043752384 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5592a4355860 con 0x5592a4763df0 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41dcff9640 10 client.17754 handle_command_reply: tid=0 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 client.17754 shutdown 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 msgr2=0x5592a46fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46fb500 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f41e4097240 tx=0x7f41e40cbb70 comp rx=0 tx=0).stop 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 shutdown_connections 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5592a4763df0 0x5592a4766210 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592a46f78b0 0x5592a46fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41dcff9640 0 client.17754 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592a46eed30 0x5592a46fb500 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 --2- 172.21.15.8:0/2043752384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a46ee380 0x5592a46fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 >> 172.21.15.8:0/2043752384 conn(0x5592a46e8840 msgr2=0x5592a46e9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 shutdown_connections 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 1 -- 172.21.15.8:0/2043752384 wait complete. 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.482+0000 7f41fe066740 20 client.17754 trim_cache size 0 max 16384 2024-02-19T16:32:26.492 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:32:26.493 INFO:teuthology.orchestra.run.smithi008.stdout: "return_code": 0, 2024-02-19T16:32:26.493 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_tag": "90affaab-6bf6-468a-86e3-5fa4870690b2", 2024-02-19T16:32:26.493 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": "asynchronous" 2024-02-19T16:32:26.493 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:32:26.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41e00a2230 msgr2=0x7f41e00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41e00a2230 0x7f41e00a46f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41ec004af0 tx=0x7f41ec009360 comp rx=0 tx=0).stop 2024-02-19T16:32:26.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f4101e70 msgr2=0x7f41f419dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f4101e70 0x7f41f419dbe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 shutdown_connections 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f4107570 0x7f41f4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f41e00a2230 0x7f41e00a46f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41f4101e70 0x7f41f419dbe0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 --2- 172.21.15.8:0/4164815682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f41014b0 0x7f41f419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 >> 172.21.15.8:0/4164815682 conn(0x7f41f40fcf20 msgr2=0x7f41f40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 shutdown_connections 2024-02-19T16:32:26.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.483+0000 7f41fb48f640 1 -- 172.21.15.8:0/4164815682 wait complete. 2024-02-19T16:32:26.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:32:26.496 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:32:26.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 -- 172.21.15.8:0/2089838935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e01003b0 msgr2=0x7f15e010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 --2- 172.21.15.8:0/2089838935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e01003b0 0x7f15e010e320 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009cd0 tx=0x7f15d802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 -- 172.21.15.8:0/2089838935 shutdown_connections 2024-02-19T16:32:26.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 --2- 172.21.15.8:0/2089838935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e01003b0 0x7f15e010e320 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 --2- 172.21.15.8:0/2089838935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e00ff9f0 0x7f15e00ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 --2- 172.21.15.8:0/2089838935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e0107510 0x7f15e01078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 -- 172.21.15.8:0/2089838935 >> 172.21.15.8:0/2089838935 conn(0x7f15e0076920 msgr2=0x7f15e00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 -- 172.21.15.8:0/2089838935 shutdown_connections 2024-02-19T16:32:26.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.601+0000 7f15e4881640 1 -- 172.21.15.8:0/2089838935 wait complete. 2024-02-19T16:32:26.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 Processor -- start 2024-02-19T16:32:26.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 -- start start 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 0x7f15e0100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e01003b0 0x7f15e0101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e0107510 0x7f15e0101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e01a5de0 con 0x7f15e00ff9f0 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e01a5f50 con 0x7f15e01003b0 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15e4881640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e01a6110 con 0x7f15e0107510 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15de59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 0x7f15e0100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15de59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 0x7f15e0100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35782/0 (socket says 172.21.15.8:35782) 2024-02-19T16:32:26.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15de59c640 1 -- 172.21.15.8:0/1132073601 learned_addr learned my addr 172.21.15.8:0/1132073601 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15d7fff640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e01003b0 0x7f15e0101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.602+0000 7f15ded9d640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e0107510 0x7f15e0101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15de59c640 1 -- 172.21.15.8:0/1132073601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e0107510 msgr2=0x7f15e0101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15de59c640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e0107510 0x7f15e0101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15de59c640 1 -- 172.21.15.8:0/1132073601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e01003b0 msgr2=0x7f15e0101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15de59c640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e01003b0 0x7f15e0101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15de59c640 1 -- 172.21.15.8:0/1132073601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8009900 con 0x7f15e00ff9f0 2024-02-19T16:32:26.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15d7fff640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e01003b0 0x7f15e0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:26.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15de59c640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 0x7f15e0100bd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f15d000e9f0 tx=0x7f15d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15d77fe640 1 -- 172.21.15.8:0/1132073601 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0009700 con 0x7f15e00ff9f0 2024-02-19T16:32:26.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15d77fe640 1 -- 172.21.15.8:0/1132073601 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15d000cd80 con 0x7f15e00ff9f0 2024-02-19T16:32:26.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e01a63f0 con 0x7f15e00ff9f0 2024-02-19T16:32:26.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15d77fe640 1 -- 172.21.15.8:0/1132073601 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0018920 con 0x7f15e00ff9f0 2024-02-19T16:32:26.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.603+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e01a6900 con 0x7f15e00ff9f0 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 Processor -- start 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15d77fe640 1 -- 172.21.15.8:0/1132073601 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f15d0018a80 con 0x7f15e00ff9f0 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 -- start start 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dacd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 0x559e9dad8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9da3c280 con 0x559e9dacd2b0 2024-02-19T16:32:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9da397b0 con 0x559e9dad67b0 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.606+0000 7f15e7458740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9da34f90 con 0x559e9dacdbd0 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dacd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 0x559e9dad8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15de59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dacd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35798/0 (socket says 172.21.15.8:35798) 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 0x559e9dad8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:45814/0 (socket says 172.21.15.8:45814) 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 -- 172.21.15.8:0/1232035937 learned_addr learned my addr 172.21.15.8:0/1232035937 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15d77fe640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15cc0a2230 0x7f15cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 -- 172.21.15.8:0/1232035937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 msgr2=0x559e9dad6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 -- 172.21.15.8:0/1232035937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 msgr2=0x559e9dad8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 0x559e9dad8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 -- 172.21.15.8:0/1232035937 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9d9f3570 con 0x559e9dacd2b0 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15d7fff640 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 0x559e9dad8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15d77fe640 1 -- 172.21.15.8:0/1132073601 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f15d0014070 con 0x7f15e00ff9f0 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15cc0a2230 0x7f15cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dacd690 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f15d8004af0 tx=0x7f15d8032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe997e40242660 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15d4ff9640 1 -- 172.21.15.8:0/1232035937 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8034c90 con 0x559e9dacd2b0 2024-02-19T16:32:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.607+0000 7f15ded9d640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15cc0a2230 0x7f15cc0a46f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15d8006010 tx=0x7f15d804a000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15d4ff9640 1 -- 172.21.15.8:0/1232035937 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15d8034780 con 0x559e9dacd2b0 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15d4ff9640 1 -- 172.21.15.8:0/1232035937 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80042a0 con 0x559e9dacd2b0 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 -- 172.21.15.8:0/1232035937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 msgr2=0x559e9dacd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dacd690 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f15d8004af0 tx=0x7f15d8032df0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 -- 172.21.15.8:0/1232035937 shutdown_connections 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dad67b0 0x559e9dad8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 --2- 172.21.15.8:0/1232035937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dacd690 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 -- 172.21.15.8:0/1232035937 >> 172.21.15.8:0/1232035937 conn(0x559e9dac7990 msgr2=0x559e9dac7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.608+0000 7f15e7458740 1 -- 172.21.15.8:0/1232035937 shutdown_connections 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 -- 172.21.15.8:0/1232035937 wait complete. 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 -- start start 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 Processor -- start 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacd2b0 0x559e9dad3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dad4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9d708430 con 0x559e9dad67b0 2024-02-19T16:32:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9d6c4e00 con 0x559e9dacd2b0 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9d9b9fd0 con 0x559e9dacdbd0 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15e7458740 10 client.0 _ll_register_callbacks cb 0x559e9da0ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15ded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dad4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15ded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dad4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35814/0 (socket says 172.21.15.8:35814) 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15ded9d640 1 -- 172.21.15.8:0/3434068443 learned_addr learned my addr 172.21.15.8:0/3434068443 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.609+0000 7f15de59c640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacd2b0 0x559e9dad3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15ded9d640 1 -- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 msgr2=0x559e9dad3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15ded9d640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15ded9d640 1 -- 172.21.15.8:0/3434068443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacd2b0 msgr2=0x559e9dad3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15ded9d640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacd2b0 0x559e9dad3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15ded9d640 1 -- 172.21.15.8:0/3434068443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e010ec20 con 0x559e9dad67b0 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15d7fff640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15ded9d640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dad4090 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f15d8003c20 tx=0x7f15d8003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f34f9d421295d2f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3434068443 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80326b0 con 0x559e9dad67b0 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3434068443 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f15d8032810 con 0x559e9dad67b0 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e9d7347e0 con 0x559e9dad67b0 2024-02-19T16:32:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3434068443 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d80329d0 con 0x559e9dad67b0 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 10 client.17766 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559e9da33610 con 0x559e9dad67b0 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3434068443 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e9d7347e0 con 0x559e9dad67b0 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3434068443 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f15d804d740 con 0x559e9dad67b0 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 10 client.17766 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 20 client.17766 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.610+0000 7f15e7458740 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9dae6790 0x559e9dae8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.611+0000 7f15e7458740 4 client.17766 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.611+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e9d771720 con 0x559e9dae6790 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.611+0000 7f15de59c640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9dae6790 0x559e9dae8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.611+0000 7f15de59c640 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9dae6790 0x559e9dae8bb0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad29f16484c463c server_cookie=c741054bde848732 in_seq=0 out_seq=0 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.611+0000 7f15c5ffb640 10 client.17766 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3434068443 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e9d771720 con 0x559e9dae6790 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15c5ffb640 10 client.17766 handle_command_reply: tid=0 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 client.17766 shutdown 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 msgr2=0x559e9dad4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dad4090 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f15d8003c20 tx=0x7f15d8003c50 comp rx=0 tx=0).stop 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 shutdown_connections 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dad4090 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9dae6790 0x559e9dae8bb0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacdbd0 0x559e9dad3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 --2- 172.21.15.8:0/3434068443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacd2b0 0x559e9dad3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15c5ffb640 0 client.17766 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.612+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 >> 172.21.15.8:0/3434068443 conn(0x559e9dac7990 msgr2=0x559e9dac9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.613+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 shutdown_connections 2024-02-19T16:32:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.613+0000 7f15e7458740 1 -- 172.21.15.8:0/3434068443 wait complete. 2024-02-19T16:32:26.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.613+0000 7f15e7458740 20 client.17766 trim_cache size 0 max 16384 2024-02-19T16:32:26.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 Processor -- start 2024-02-19T16:32:26.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 -- start start 2024-02-19T16:32:26.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacd2b0 0x559e9dad5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dad3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dae1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9dace420 con 0x559e9dad67b0 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9dac9cd0 con 0x559e9dacdbd0 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15e7458740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9dae9210 con 0x559e9dacd2b0 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15ded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dae1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dad3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dad3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:45834/0 (socket says 172.21.15.8:45834) 2024-02-19T16:32:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15d7fff640 1 -- 172.21.15.8:0/4249293047 learned_addr learned my addr 172.21.15.8:0/4249293047 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15ded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dae1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35820/0 (socket says 172.21.15.8:35820) 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.631+0000 7f15de59c640 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacd2b0 0x559e9dad5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15ded9d640 1 -- 172.21.15.8:0/4249293047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacd2b0 msgr2=0x559e9dad5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15ded9d640 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacd2b0 0x559e9dad5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15ded9d640 1 -- 172.21.15.8:0/4249293047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 msgr2=0x559e9dad3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15ded9d640 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dad3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15ded9d640 1 -- 172.21.15.8:0/4249293047 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9d9cfac0 con 0x559e9dad67b0 2024-02-19T16:32:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15ded9d640 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dae1b30 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f15d804db20 tx=0x7f15d80324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdae160369bf6e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15c7fff640 1 -- 172.21.15.8:0/4249293047 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8003ea0 con 0x559e9dad67b0 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15c7fff640 1 -- 172.21.15.8:0/4249293047 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15d8003cc0 con 0x559e9dad67b0 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.632+0000 7f15c7fff640 1 -- 172.21.15.8:0/4249293047 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80043c0 con 0x559e9dad67b0 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- 172.21.15.8:0/4249293047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 msgr2=0x559e9dae1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dae1b30 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f15d804db20 tx=0x7f15d80324c0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- 172.21.15.8:0/4249293047 shutdown_connections 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dad67b0 0x559e9dae1b30 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dad3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- 172.21.15.8:0/4249293047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dacd2b0 0x559e9dad5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- 172.21.15.8:0/4249293047 >> 172.21.15.8:0/4249293047 conn(0x559e9dac7990 msgr2=0x559e9dad2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- 172.21.15.8:0/4249293047 shutdown_connections 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- 172.21.15.8:0/4249293047 wait complete. 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- start start 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 Processor -- start 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dade070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dade5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dad67b0 0x559e9dad9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9dad9e00 con 0x559e9dacd2b0 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9dad9f70 con 0x559e9dacdbd0 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9dada130 con 0x559e9dad67b0 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15e7458740 10 client.0 _ll_register_callbacks cb 0x7f15c8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15de59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dade070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15de59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dade070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35836/0 (socket says 172.21.15.8:35836) 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.633+0000 7f15de59c640 1 -- 172.21.15.8:0/3843704363 learned_addr learned my addr 172.21.15.8:0/3843704363 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15d7fff640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dade5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15ded9d640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dad67b0 0x559e9dad9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15de59c640 1 -- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dad67b0 msgr2=0x559e9dad9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15de59c640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dad67b0 0x559e9dad9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15de59c640 1 -- 172.21.15.8:0/3843704363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 msgr2=0x559e9dade5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15de59c640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dade5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15de59c640 1 -- 172.21.15.8:0/3843704363 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e9da33610 con 0x559e9dacd2b0 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15d7fff640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dade5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15de59c640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dade070 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f15d0097810 tx=0x7f15d00d0ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf500513dfbe18ec server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3843704363 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d009ac10 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3843704363 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f15d009ae40 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15d8034780 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3843704363 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d009a980 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15e7458740 10 client.17772 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559e9d9f56a0 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.634+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3843704363 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15d8034780 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3843704363 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f15d00ce070 con 0x559e9dacd2b0 2024-02-19T16:32:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15e7458740 10 client.17772 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15e7458740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15e7458740 20 client.17772 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15e7458740 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9db42350 0x559e9db44770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15e7458740 4 client.17772 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559e9d7c0890 con 0x559e9db42350 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15d7fff640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9db42350 0x559e9db44770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15c5ffb640 10 client.17772 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.635+0000 7f15d7fff640 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9db42350 0x559e9db44770 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28420e4170645007 server_cookie=a8c9c73b8b49c406 in_seq=0 out_seq=0 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15c5ffb640 1 -- 172.21.15.8:0/3843704363 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x559e9d7c0890 con 0x559e9db42350 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15c5ffb640 10 client.17772 handle_command_reply: tid=0 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 client.17772 shutdown 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 msgr2=0x559e9dade070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dade070 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f15d0097810 tx=0x7f15d00d0ab0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 shutdown_connections 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x559e9db42350 0x559e9db44770 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559e9dad67b0 0x559e9dad9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559e9dacdbd0 0x559e9dade5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15c5ffb640 0 client.17772 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 --2- 172.21.15.8:0/3843704363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559e9dacd2b0 0x559e9dade070 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 >> 172.21.15.8:0/3843704363 conn(0x559e9dac7990 msgr2=0x559e9dac8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 shutdown_connections 2024-02-19T16:32:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 1 -- 172.21.15.8:0/3843704363 wait complete. 2024-02-19T16:32:26.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.636+0000 7f15e7458740 20 client.17772 trim_cache size 0 max 16384 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "scrub active (2830 inodes in the stack)", 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": { 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout: "90affaab-6bf6-468a-86e3-5fa4870690b2": { 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout: "path": "/", 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout: "tag": "90affaab-6bf6-468a-86e3-5fa4870690b2", 2024-02-19T16:32:26.649 INFO:teuthology.orchestra.run.smithi008.stdout: "options": "recursive,force" 2024-02-19T16:32:26.650 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:32:26.650 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:32:26.650 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15cc0a2230 msgr2=0x7f15cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15cc0a2230 0x7f15cc0a46f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15d8006010 tx=0x7f15d804a000 comp rx=0 tx=0).stop 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 msgr2=0x7f15e0100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 0x7f15e0100bd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f15d000e9f0 tx=0x7f15d000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 shutdown_connections 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f15cc0a2230 0x7f15cc0a46f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e0107510 0x7f15e0101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15e01003b0 0x7f15e0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 --2- 172.21.15.8:0/1132073601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e00ff9f0 0x7f15e0100bd0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 >> 172.21.15.8:0/1132073601 conn(0x7f15e0076920 msgr2=0x7f15e00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 shutdown_connections 2024-02-19T16:32:26.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:26.637+0000 7f15e4881640 1 -- 172.21.15.8:0/1132073601 wait complete. 2024-02-19T16:32:26.652 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:26 smithi008 ceph-mon[32573]: pgmap v297: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:32:26.653 INFO:tasks.cephfs.filesystem:scrub status for tag:90affaab-6bf6-468a-86e3-5fa4870690b2 - {'path': '/', 'tag': '90affaab-6bf6-468a-86e3-5fa4870690b2', 'options': 'recursive,force'} 2024-02-19T16:32:26.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:26 smithi191 ceph-mon[37203]: pgmap v297: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:32:26.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:26 smithi156 ceph-mon[36502]: pgmap v297: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:32:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub queued for path: / 2024-02-19T16:32:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.771 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:27 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub queued for path: / 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:32:27.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:27 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub queued for path: / 2024-02-19T16:32:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:32:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:32:27.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:32:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:32:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:27.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:27 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:28 smithi008 ceph-mon[32573]: pgmap v298: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:32:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:28 smithi008 ceph-mon[32573]: scrub summary: active paths [/] 2024-02-19T16:32:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:28 smithi008 ceph-mon[32573]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:28 smithi191 ceph-mon[37203]: pgmap v298: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:32:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:28 smithi191 ceph-mon[37203]: scrub summary: active paths [/] 2024-02-19T16:32:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:28 smithi191 ceph-mon[37203]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:28 smithi156 ceph-mon[36502]: pgmap v298: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:32:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:28 smithi156 ceph-mon[36502]: scrub summary: active paths [/] 2024-02-19T16:32:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:28 smithi156 ceph-mon[36502]: scrub summary: idle+waiting paths [/] 2024-02-19T16:32:30.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:30 smithi008 ceph-mon[32573]: pgmap v299: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:32:30.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:30 smithi191 ceph-mon[37203]: pgmap v299: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:32:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:30 smithi156 ceph-mon[36502]: pgmap v299: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:32:32.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:32 smithi008 ceph-mon[32573]: pgmap v300: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:32.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:32 smithi191 ceph-mon[37203]: pgmap v300: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:32.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:32 smithi156 ceph-mon[36502]: pgmap v300: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:34 smithi008 ceph-mon[32573]: pgmap v301: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:34 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:32:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:34 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:32:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:34 smithi191 ceph-mon[37203]: pgmap v301: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:34 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:32:34.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:34 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:32:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:34 smithi156 ceph-mon[36502]: pgmap v301: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:34 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:32:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:34 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:32:35.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:35 smithi008 ceph-mon[32573]: scrub summary: idle 2024-02-19T16:32:35.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:35 smithi191 ceph-mon[37203]: scrub summary: idle 2024-02-19T16:32:35.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:35 smithi156 ceph-mon[36502]: scrub summary: idle 2024-02-19T16:32:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:36 smithi008 ceph-mon[32573]: pgmap v302: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:36.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:36 smithi191 ceph-mon[37203]: pgmap v302: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:36 smithi156 ceph-mon[36502]: pgmap v302: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:38 smithi008 ceph-mon[32573]: pgmap v303: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:38 smithi191 ceph-mon[37203]: pgmap v303: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:38.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:38 smithi156 ceph-mon[36502]: pgmap v303: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-19T16:32:40.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:40 smithi008 ceph-mon[32573]: pgmap v304: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:40.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:40 smithi191 ceph-mon[37203]: pgmap v304: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:40 smithi156 ceph-mon[36502]: pgmap v304: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-02-19T16:32:42.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:42 smithi008 ceph-mon[32573]: pgmap v305: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:42 smithi191 ceph-mon[37203]: pgmap v305: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:42 smithi156 ceph-mon[36502]: pgmap v305: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:44.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:44 smithi008 ceph-mon[32573]: pgmap v306: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:44.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:44 smithi191 ceph-mon[37203]: pgmap v306: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:44.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:44 smithi156 ceph-mon[36502]: pgmap v306: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:46.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:46 smithi008 ceph-mon[32573]: pgmap v307: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:46.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:46 smithi191 ceph-mon[37203]: pgmap v307: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:46.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:46 smithi156 ceph-mon[36502]: pgmap v307: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:48.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:48 smithi008 ceph-mon[32573]: pgmap v308: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:48.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:48 smithi191 ceph-mon[37203]: pgmap v308: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:48.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:48 smithi156 ceph-mon[36502]: pgmap v308: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:50.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:50 smithi008 ceph-mon[32573]: pgmap v309: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:50.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:50 smithi191 ceph-mon[37203]: pgmap v309: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:50.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:50 smithi156 ceph-mon[36502]: pgmap v309: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:52.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:52 smithi191 ceph-mon[37203]: pgmap v310: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:52.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:52 smithi156 ceph-mon[36502]: pgmap v310: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:53.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:52 smithi008 ceph-mon[32573]: pgmap v310: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:54.627 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-19T16:32:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 -- 172.21.15.8:0/3028036140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570101e70 msgr2=0x7ff57010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 --2- 172.21.15.8:0/3028036140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570101e70 0x7ff57010e330 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff568009ca0 tx=0x7ff56802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 -- 172.21.15.8:0/3028036140 shutdown_connections 2024-02-19T16:32:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 --2- 172.21.15.8:0/3028036140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570101e70 0x7ff57010e330 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 --2- 172.21.15.8:0/3028036140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 0x7ff570101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 --2- 172.21.15.8:0/3028036140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff570107570 0x7ff570107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 -- 172.21.15.8:0/3028036140 >> 172.21.15.8:0/3028036140 conn(0x7ff5700fcf80 msgr2=0x7ff5700ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 -- 172.21.15.8:0/3028036140 shutdown_connections 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.742+0000 7ff575e46640 1 -- 172.21.15.8:0/3028036140 wait complete. 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 Processor -- start 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 -- start start 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 0x7ff5700737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff570101e70 0x7ff570073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 0x7ff570074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701a5d20 con 0x7ff570107570 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701a5e90 con 0x7ff5701014b0 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff575e46640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701a6050 con 0x7ff570101e70 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 0x7ff570074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 0x7ff5700737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 0x7ff570074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47076/0 (socket says 172.21.15.8:47076) 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 -- 172.21.15.8:0/1870986965 learned_addr learned my addr 172.21.15.8:0/1870986965 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56effd640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff570101e70 0x7ff570073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 -- 172.21.15.8:0/1870986965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff570101e70 msgr2=0x7ff570073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff570101e70 0x7ff570073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 -- 172.21.15.8:0/1870986965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 msgr2=0x7ff5700737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 0x7ff5700737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56ffff640 1 -- 172.21.15.8:0/1870986965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568009900 con 0x7ff570107570 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.743+0000 7ff56f7fe640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 0x7ff5700737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.744+0000 7ff56ffff640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 0x7ff570074270 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff568005e30 tx=0x7ff568035aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.744+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568035ea0 con 0x7ff570107570 2024-02-19T16:32:54.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.744+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5701a62d0 con 0x7ff570107570 2024-02-19T16:32:54.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.744+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff568003e50 con 0x7ff570107570 2024-02-19T16:32:54.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.744+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568004830 con 0x7ff570107570 2024-02-19T16:32:54.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.744+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5701a6780 con 0x7ff570107570 2024-02-19T16:32:54.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.745+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57004f250 con 0x7ff570107570 2024-02-19T16:32:54.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.746+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff56803b040 con 0x7ff570107570 2024-02-19T16:32:54.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.747+0000 7ff56cff9640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5580a22d0 0x7ff5580a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:54.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.747+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7ff5680f0cc0 con 0x7ff570107570 2024-02-19T16:32:54.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.747+0000 7ff56f7fe640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5580a22d0 0x7ff5580a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:54.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.747+0000 7ff56f7fe640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5580a22d0 0x7ff5580a4790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff560009fc0 tx=0x7ff560005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:54.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.751+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff5680b9660 con 0x7ff570107570 2024-02-19T16:32:54.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:54 smithi191 ceph-mon[37203]: pgmap v311: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:54.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:54 smithi156 ceph-mon[36502]: pgmap v311: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:54.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.955+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 --> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ff570109930 con 0x7ff5580a22d0 2024-02-19T16:32:54.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.962+0000 7ff56cff9640 1 -- 172.21.15.8:0/1870986965 <== mgr.15633 v2:172.21.15.8:6800/4279763273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7ff570109930 con 0x7ff5580a22d0 2024-02-19T16:32:54.963 INFO:teuthology.orchestra.run.smithi008.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-19T00:00:00", "created": "2024-02-19T16:22:53", "first": "2024-02-19T16:23:00", "last": "2024-02-19T16:32:00", "last_pruned": "2024-02-19T16:32:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-02-19T16:32:54.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5580a22d0 msgr2=0x7ff5580a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5580a22d0 0x7ff5580a4790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff560009fc0 tx=0x7ff560005f90 comp rx=0 tx=0).stop 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 msgr2=0x7ff570074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 0x7ff570074270 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff568005e30 tx=0x7ff568035aa0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 shutdown_connections 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff570107570 0x7ff570074270 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5580a22d0 0x7ff5580a4790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff570101e70 0x7ff570073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 --2- 172.21.15.8:0/1870986965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5701014b0 0x7ff5700737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:54.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 >> 172.21.15.8:0/1870986965 conn(0x7ff5700fcf80 msgr2=0x7ff5700fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:54.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 shutdown_connections 2024-02-19T16:32:54.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:54.964+0000 7ff575e46640 1 -- 172.21.15.8:0/1870986965 wait complete. 2024-02-19T16:32:54.974 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:54 smithi008 ceph-mon[32573]: pgmap v311: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:54.976 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 -- 172.21.15.8:0/1171200465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 msgr2=0x7f2f24100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 --2- 172.21.15.8:0/1171200465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f24100ac0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009900 tx=0x7f2f1402bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 -- 172.21.15.8:0/1171200465 shutdown_connections 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 --2- 172.21.15.8:0/1171200465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f24101000 0x7f2f2410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 --2- 172.21.15.8:0/1171200465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f24100ac0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 --2- 172.21.15.8:0/1171200465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24105390 0x7f2f24105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 -- 172.21.15.8:0/1171200465 >> 172.21.15.8:0/1171200465 conn(0x7f2f24077240 msgr2=0x7f2f24077650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:55.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 -- 172.21.15.8:0/1171200465 shutdown_connections 2024-02-19T16:32:55.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 -- 172.21.15.8:0/1171200465 wait complete. 2024-02-19T16:32:55.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.078+0000 7f2f2b919640 1 Processor -- start 2024-02-19T16:32:55.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 -- start start 2024-02-19T16:32:55.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f241989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:55.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f24101000 0x7f2f24198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:55.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24105390 0x7f2f241a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24199790 con 0x7f2f24100640 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241a3f90 con 0x7f2f24101000 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241a40d0 con 0x7f2f24105390 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f241989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f241989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47104/0 (socket says 172.21.15.8:47104) 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 -- 172.21.15.8:0/2957552940 learned_addr learned my addr 172.21.15.8:0/2957552940 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:55.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f29eb6640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24105390 0x7f2f241a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f28eb4640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f24101000 0x7f2f24198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 -- 172.21.15.8:0/2957552940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24105390 msgr2=0x7f2f241a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24105390 0x7f2f241a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 -- 172.21.15.8:0/2957552940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f24101000 msgr2=0x7f2f24198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f24101000 0x7f2f24198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 -- 172.21.15.8:0/2957552940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009490 con 0x7f2f24100640 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f296b5640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f241989d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000ea90 tx=0x7f2f2000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:55.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000cde0 con 0x7f2f24100640 2024-02-19T16:32:55.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f200035f0 con 0x7f2f24100640 2024-02-19T16:32:55.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20010690 con 0x7f2f24100640 2024-02-19T16:32:55.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.079+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f14009560 con 0x7f2f24100640 2024-02-19T16:32:55.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.080+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241a46d0 con 0x7f2f24100640 2024-02-19T16:32:55.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.080+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2404f250 con 0x7f2f24100640 2024-02-19T16:32:55.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.081+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f2f20003ad0 con 0x7f2f24100640 2024-02-19T16:32:55.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.082+0000 7f2f1a7fc640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2f100a2350 0x7f2f100a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:55.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.082+0000 7f2f28eb4640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2f100a2350 0x7f2f100a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:55.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.082+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f2f20014070 con 0x7f2f24100640 2024-02-19T16:32:55.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.083+0000 7f2f28eb4640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2f100a2350 0x7f2f100a4810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2f14006010 tx=0x7f2f14005e40 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:55.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.086+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f2f20097090 con 0x7f2f24100640 2024-02-19T16:32:55.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.291+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 --> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2f241105b0 con 0x7f2f100a2350 2024-02-19T16:32:55.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.297+0000 7f2f1a7fc640 1 -- 172.21.15.8:0/2957552940 <== mgr.15633 v2:172.21.15.8:6800/4279763273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2f241105b0 con 0x7f2f100a2350 2024-02-19T16:32:55.299 INFO:teuthology.orchestra.run.smithi008.stdout:/ 1m 6m3h 2024-02-19T16:32:55.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2f100a2350 msgr2=0x7f2f100a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:55.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2f100a2350 0x7f2f100a4810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2f14006010 tx=0x7f2f14005e40 comp rx=0 tx=0).stop 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 msgr2=0x7f2f241989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f241989d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000ea90 tx=0x7f2f2000ef50 comp rx=0 tx=0).stop 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 shutdown_connections 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f2f100a2350 0x7f2f100a4810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24105390 0x7f2f241a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f24101000 0x7f2f24198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 --2- 172.21.15.8:0/2957552940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f24100640 0x7f2f241989d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 >> 172.21.15.8:0/2957552940 conn(0x7f2f24077240 msgr2=0x7f2f240fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:55.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 shutdown_connections 2024-02-19T16:32:55.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:55.300+0000 7f2f2b919640 1 -- 172.21.15.8:0/2957552940 wait complete. 2024-02-19T16:32:55.312 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-19T16:32:55.322 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-19T16:32:56.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-19T16:32:56.654 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-19T16:32:56.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 -- 172.21.15.8:0/4272456793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84106480 msgr2=0x7fea84106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 --2- 172.21.15.8:0/4272456793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84106480 0x7fea84106860 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fea80009900 tx=0x7fea8002bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 -- 172.21.15.8:0/4272456793 shutdown_connections 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 --2- 172.21.15.8:0/4272456793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 0x7fea8410d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 --2- 172.21.15.8:0/4272456793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84100640 0x7fea84100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 --2- 172.21.15.8:0/4272456793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84106480 0x7fea84106860 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 -- 172.21.15.8:0/4272456793 >> 172.21.15.8:0/4272456793 conn(0x7fea84077190 msgr2=0x7fea840775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 -- 172.21.15.8:0/4272456793 shutdown_connections 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 -- 172.21.15.8:0/4272456793 wait complete. 2024-02-19T16:32:56.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 Processor -- start 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.771+0000 7fea8c18a640 1 -- start start 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8c18a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 0x7fea84102050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8c18a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 0x7fea84102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8c18a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84106480 0x7fea84102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8c18a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea841a5ef0 con 0x7fea84100640 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8c18a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea841a6060 con 0x7fea84101000 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8c18a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea841a6220 con 0x7fea84106480 2024-02-19T16:32:56.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 0x7fea84102050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 0x7fea84102050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44062/0 (socket says 172.21.15.8:44062) 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 -- 172.21.15.8:0/2850709042 learned_addr learned my addr 172.21.15.8:0/2850709042 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8a727640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84106480 0x7fea84102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89725640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 0x7fea84102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 -- 172.21.15.8:0/2850709042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84106480 msgr2=0x7fea84102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84106480 0x7fea84102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 -- 172.21.15.8:0/2850709042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 msgr2=0x7fea84102590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 0x7fea84102590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 -- 172.21.15.8:0/2850709042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80009560 con 0x7fea84100640 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea8a727640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84106480 0x7fea84102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89725640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 0x7fea84102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea89f26640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 0x7fea84102050 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fea80000c00 tx=0x7fea8002c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.772+0000 7fea7affd640 1 -- 172.21.15.8:0/2850709042 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80004390 con 0x7fea84100640 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.773+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea841a64a0 con 0x7fea84100640 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.773+0000 7fea7affd640 1 -- 172.21.15.8:0/2850709042 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea800044f0 con 0x7fea84100640 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.773+0000 7fea7affd640 1 -- 172.21.15.8:0/2850709042 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80003d00 con 0x7fea84100640 2024-02-19T16:32:56.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.773+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea841a6a00 con 0x7fea84100640 2024-02-19T16:32:56.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:56 smithi191 ceph-mon[37203]: pgmap v312: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:56.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:56 smithi191 ceph-mon[37203]: from='client.17778 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:32:56.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:56 smithi191 ceph-mon[37203]: from='client.17784 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:32:56.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea7affd640 1 -- 172.21.15.8:0/2850709042 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fea80004df0 con 0x7fea84100640 2024-02-19T16:32:56.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 Processor -- start 2024-02-19T16:32:56.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 -- start start 2024-02-19T16:32:56.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685bbbd0 0x5567685c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685c47b0 0x5567685c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676852a280 con 0x5567685bbbd0 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685277b0 con 0x5567685c47b0 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea8ed67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556768522f90 con 0x5567685bb2b0 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.775+0000 7fea7affd640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fea700a2230 0x7fea700a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea89f26640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685bbbd0 0x5567685c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685bb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:58768/0 (socket says 172.21.15.8:58768) 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 -- 172.21.15.8:0/1496583502 learned_addr learned my addr 172.21.15.8:0/1496583502 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea89725640 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685c47b0 0x5567685c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fea700a2230 0x7fea700a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea7affd640 1 -- 172.21.15.8:0/2850709042 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fea800f09f0 con 0x7fea84100640 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 -- 172.21.15.8:0/1496583502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685c47b0 msgr2=0x5567685c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685c47b0 0x5567685c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 -- 172.21.15.8:0/1496583502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685bbbd0 msgr2=0x5567685c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685bbbd0 0x5567685c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 -- 172.21.15.8:0/1496583502 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567684e1570 con 0x5567685bb2b0 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fea700a2230 0x7fea700a46f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0099e0 tx=0x7fea7c018000 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea8a727640 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685bb690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009fd0 tx=0x7fea7c00f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1360be3c9b504884 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea6ffff640 1 -- 172.21.15.8:0/1496583502 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c004450 con 0x5567685bb2b0 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea6ffff640 1 -- 172.21.15.8:0/1496583502 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7c0115f0 con 0x5567685bb2b0 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.776+0000 7fea6ffff640 1 -- 172.21.15.8:0/1496583502 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c026400 con 0x5567685bb2b0 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 -- 172.21.15.8:0/1496583502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 msgr2=0x5567685bb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685bb690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009fd0 tx=0x7fea7c00f940 comp rx=0 tx=0).stop 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 -- 172.21.15.8:0/1496583502 shutdown_connections 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685c47b0 0x5567685c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685bbbd0 0x5567685c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1496583502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685bb690 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 -- 172.21.15.8:0/1496583502 >> 172.21.15.8:0/1496583502 conn(0x5567685b5990 msgr2=0x5567685b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 -- 172.21.15.8:0/1496583502 shutdown_connections 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.777+0000 7fea8ed67740 1 -- 172.21.15.8:0/1496583502 wait complete. 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 -- start start 2024-02-19T16:32:56.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 Processor -- start 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bbbd0 0x5567685c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567681f6430 con 0x5567685c47b0 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567681b2e00 con 0x5567685bbbd0 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567684a7fd0 con 0x5567685bb2b0 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8ed67740 10 client.0 _ll_register_callbacks cb 0x5567684fdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea89f26640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8a727640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea89f26640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.8:58782/0 (socket says 172.21.15.8:58782) 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea8a727640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44086/0 (socket says 172.21.15.8:44086) 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea89f26640 1 -- 172.21.15.8:0/1426848121 learned_addr learned my addr 172.21.15.8:0/1426848121 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.778+0000 7fea89725640 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bbbd0 0x5567685c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea89f26640 1 -- 172.21.15.8:0/1426848121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bbbd0 msgr2=0x5567685c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea89f26640 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bbbd0 0x5567685c15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea89f26640 1 -- 172.21.15.8:0/1426848121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 msgr2=0x5567685c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea89f26640 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea89f26640 1 -- 172.21.15.8:0/1426848121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c019040 con 0x5567685bb2b0 2024-02-19T16:32:56.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea89f26640 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685c4250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea800bce80 tx=0x7fea8008f140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44afcf16d1cbad0e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea6d7fa640 1 -- 172.21.15.8:0/1426848121 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800c4590 con 0x5567685bb2b0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea6d7fa640 1 -- 172.21.15.8:0/1426848121 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fea800035f0 con 0x5567685bb2b0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567682227e0 con 0x5567685bb2b0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea6d7fa640 1 -- 172.21.15.8:0/1426848121 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea80003e60 con 0x5567685bb2b0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea8ed67740 10 client.27646 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556768521610 con 0x5567685bb2b0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea6d7fa640 1 -- 172.21.15.8:0/1426848121 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea800d9080 con 0x5567685bb2b0 2024-02-19T16:32:56.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.779+0000 7fea6d7fa640 1 -- 172.21.15.8:0/1426848121 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea8003c070 con 0x5567685bb2b0 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea8ed67740 10 client.27646 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea8ed67740 10 client.27646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea8ed67740 20 client.27646 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5567685d46d0 0x5567685d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea8ed67740 4 client.27646 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55676825f720 con 0x5567685d46d0 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea89725640 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5567685d46d0 0x5567685d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea6d7fa640 10 client.27646 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.780+0000 7fea89725640 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5567685d46d0 0x5567685d6ac0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4422fad018763c9d server_cookie=61528d5de0356199 in_seq=0 out_seq=0 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea6d7fa640 1 -- 172.21.15.8:0/1426848121 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55676825f720 con 0x5567685d46d0 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea6d7fa640 10 client.27646 handle_command_reply: tid=0 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 client.27646 shutdown 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 msgr2=0x5567685c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685c4250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea800bce80 tx=0x7fea8008f140 comp rx=0 tx=0).stop 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 shutdown_connections 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bbbd0 0x5567685c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x5567685d46d0 0x5567685d6ac0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 --2- 172.21.15.8:0/1426848121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bb2b0 0x5567685c4250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea6d7fa640 0 client.27646 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 >> 172.21.15.8:0/1426848121 conn(0x5567685b5990 msgr2=0x5567685b7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 shutdown_connections 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 1 -- 172.21.15.8:0/1426848121 wait complete. 2024-02-19T16:32:56.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.781+0000 7fea8ed67740 20 client.27646 trim_cache size 0 max 16384 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.797+0000 7fea8ed67740 1 Processor -- start 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.797+0000 7fea8ed67740 1 -- start start 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8ed67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685baf10 con 0x5567685c47b0 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8ed67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685b51d0 con 0x5567685bb2b0 2024-02-19T16:32:56.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8ed67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685afe60 con 0x5567685bbbd0 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44094/0 (socket says 172.21.15.8:44094) 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 -- 172.21.15.8:0/3608571641 learned_addr learned my addr 172.21.15.8:0/3608571641 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea89f26640 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea89725640 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 -- 172.21.15.8:0/3608571641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 msgr2=0x5567685c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 -- 172.21.15.8:0/3608571641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 msgr2=0x5567685c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 -- 172.21.15.8:0/3608571641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567684bdac0 con 0x5567685c47b0 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea89f26640 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea8a727640 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685cfa70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009fd0 tx=0x7fea7c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744fcd957546ad44 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea6f7fe640 1 -- 172.21.15.8:0/3608571641 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c011cc0 con 0x5567685c47b0 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea6f7fe640 1 -- 172.21.15.8:0/3608571641 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7c00ec30 con 0x5567685c47b0 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.798+0000 7fea6f7fe640 1 -- 172.21.15.8:0/3608571641 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00e470 con 0x5567685c47b0 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 -- 172.21.15.8:0/3608571641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 msgr2=0x5567685cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685cfa70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009fd0 tx=0x7fea7c009740 comp rx=0 tx=0).stop 2024-02-19T16:32:56.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 -- 172.21.15.8:0/3608571641 shutdown_connections 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685cfa70 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 --2- 172.21.15.8:0/3608571641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 -- 172.21.15.8:0/3608571641 >> 172.21.15.8:0/3608571641 conn(0x5567685b5990 msgr2=0x5567685b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 -- 172.21.15.8:0/3608571641 shutdown_connections 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 -- 172.21.15.8:0/3608571641 wait complete. 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 -- start start 2024-02-19T16:32:56.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.799+0000 7fea8ed67740 1 Processor -- start 2024-02-19T16:32:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685cc8b0 con 0x5567685c47b0 2024-02-19T16:32:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685bf0e0 con 0x5567685bb2b0 2024-02-19T16:32:56.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567685bf270 con 0x5567685bbbd0 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8ed67740 10 client.0 _ll_register_callbacks cb 0x7fea80009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685beba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44102/0 (socket says 172.21.15.8:44102) 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea89725640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 -- 172.21.15.8:0/2469619589 learned_addr learned my addr 172.21.15.8:0/2469619589 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea89f26640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 -- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 msgr2=0x5567685be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685be660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 -- 172.21.15.8:0/2469619589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 msgr2=0x5567685cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 -- 172.21.15.8:0/2469619589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556768521610 con 0x5567685c47b0 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea89725640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea89f26640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea8a727640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685beba0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00f980 tx=0x7fea7c00f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e73be4187d22c90 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.800+0000 7fea6d7fa640 1 -- 172.21.15.8:0/2469619589 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00eda0 con 0x5567685c47b0 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea6d7fa640 1 -- 172.21.15.8:0/2469619589 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fea7c011040 con 0x5567685c47b0 2024-02-19T16:32:56.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea7c0115f0 con 0x5567685c47b0 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea6d7fa640 1 -- 172.21.15.8:0/2469619589 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea7c011200 con 0x5567685c47b0 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 10 client.17802 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5567684e1570 con 0x5567685c47b0 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea6d7fa640 1 -- 172.21.15.8:0/2469619589 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7c0115f0 con 0x5567685c47b0 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea6d7fa640 1 -- 172.21.15.8:0/2469619589 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea7c024030 con 0x5567685c47b0 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 10 client.17802 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:56.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 20 client.17802 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556768631010 0x556768633430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 4 client.17802 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567682ae890 con 0x556768631010 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.801+0000 7fea89f26640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556768631010 0x556768633430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea89f26640 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556768631010 0x556768633430 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af03967d6d5583 server_cookie=4f1fe7d832dde432 in_seq=0 out_seq=0 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea6d7fa640 10 client.17802 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea6d7fa640 1 -- 172.21.15.8:0/2469619589 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5567682ae890 con 0x556768631010 2024-02-19T16:32:56.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea6d7fa640 10 client.17802 handle_command_reply: tid=0 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 client.17802 shutdown 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 msgr2=0x5567685beba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685beba0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00f980 tx=0x7fea7c00f470 comp rx=0 tx=0).stop 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 shutdown_connections 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5567685c47b0 0x5567685beba0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x556768631010 0x556768633430 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5567685bbbd0 0x5567685be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 --2- 172.21.15.8:0/2469619589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567685bb2b0 0x5567685cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea6d7fa640 0 client.17802 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 >> 172.21.15.8:0/2469619589 conn(0x5567685b5990 msgr2=0x5567685b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 shutdown_connections 2024-02-19T16:32:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 1 -- 172.21.15.8:0/2469619589 wait complete. 2024-02-19T16:32:56.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.802+0000 7fea8ed67740 20 client.17802 trim_cache size 0 max 16384 2024-02-19T16:32:56.814 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:32:56.814 INFO:teuthology.orchestra.run.smithi008.stdout: "status": "no active scrubs running", 2024-02-19T16:32:56.815 INFO:teuthology.orchestra.run.smithi008.stdout: "scrubs": {} 2024-02-19T16:32:56.815 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:32:56.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fea700a2230 msgr2=0x7fea700a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fea700a2230 0x7fea700a46f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0099e0 tx=0x7fea7c018000 comp rx=0 tx=0).stop 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 msgr2=0x7fea84102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 0x7fea84102050 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fea80000c00 tx=0x7fea8002c630 comp rx=0 tx=0).stop 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 shutdown_connections 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fea700a2230 0x7fea700a46f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84106480 0x7fea84102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea84101000 0x7fea84102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 --2- 172.21.15.8:0/2850709042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84100640 0x7fea84102050 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 >> 172.21.15.8:0/2850709042 conn(0x7fea84077190 msgr2=0x7fea84069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 shutdown_connections 2024-02-19T16:32:56.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.803+0000 7fea8c18a640 1 -- 172.21.15.8:0/2850709042 wait complete. 2024-02-19T16:32:56.817 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:56 smithi008 ceph-mon[32573]: pgmap v312: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:56.817 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:56 smithi008 ceph-mon[32573]: from='client.17778 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:32:56.817 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:56 smithi008 ceph-mon[32573]: from='client.17784 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:32:56.818 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-19T16:32:56.818 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:32:56.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:56 smithi156 ceph-mon[36502]: pgmap v312: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:32:56.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:56 smithi156 ceph-mon[36502]: from='client.17778 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:32:56.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:56 smithi156 ceph-mon[36502]: from='client.17784 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 -- 172.21.15.8:0/2236903812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b8100620 msgr2=0x7f57b8100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2236903812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b8100620 0x7f57b8100a00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009930 tx=0x7f57b002bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 -- 172.21.15.8:0/2236903812 shutdown_connections 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2236903812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b81056f0 0x7f57b810d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2236903812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100fd0 0x7f57b8105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2236903812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b8100620 0x7f57b8100a00 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 -- 172.21.15.8:0/2236903812 >> 172.21.15.8:0/2236903812 conn(0x7f57b8077490 msgr2=0x7f57b80778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:56.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 -- 172.21.15.8:0/2236903812 shutdown_connections 2024-02-19T16:32:56.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.920+0000 7f57bce5c640 1 -- 172.21.15.8:0/2236903812 wait complete. 2024-02-19T16:32:56.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 Processor -- start 2024-02-19T16:32:56.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 -- start start 2024-02-19T16:32:56.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100620 0x7f57b819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 0x7f57b819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b81979d0 0x7f57b8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b81983c0 con 0x7f57b81056f0 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8198530 con 0x7f57b81979d0 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57bce5c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b81986f0 con 0x7f57b8100620 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 0x7f57b819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 0x7f57b819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44130/0 (socket says 172.21.15.8:44130) 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 -- 172.21.15.8:0/2307143465 learned_addr learned my addr 172.21.15.8:0/2307143465 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b659c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100620 0x7f57b819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b6d9d640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b81979d0 0x7f57b8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 -- 172.21.15.8:0/2307143465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100620 msgr2=0x7f57b819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100620 0x7f57b819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 -- 172.21.15.8:0/2307143465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b81979d0 msgr2=0x7f57b8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b81979d0 0x7f57b8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 -- 172.21.15.8:0/2307143465 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0009560 con 0x7f57b81056f0 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b659c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100620 0x7f57b819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.921+0000 7f57b5d9b640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 0x7f57b819dd60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f57a000e9c0 tx=0x7f57a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.922+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0009700 con 0x7f57b81056f0 2024-02-19T16:32:56.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.922+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b81a46e0 con 0x7f57b81056f0 2024-02-19T16:32:56.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.922+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57a000cd80 con 0x7f57b81056f0 2024-02-19T16:32:56.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.922+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0010430 con 0x7f57b81056f0 2024-02-19T16:32:56.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.922+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b81a4c10 con 0x7f57b81056f0 2024-02-19T16:32:56.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.923+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b8068730 con 0x7f57b81056f0 2024-02-19T16:32:56.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.924+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f57a0005000 con 0x7f57b81056f0 2024-02-19T16:32:56.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.924+0000 7f57af7fe640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57a40a2230 0x7f57a40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:56.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.925+0000 7f57b659c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57a40a2230 0x7f57a40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:56.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.925+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f57a0014070 con 0x7f57b81056f0 2024-02-19T16:32:56.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.925+0000 7f57b659c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57a40a2230 0x7f57a40a46f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f57b0005ab0 tx=0x7f57b0005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:56.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:56.928+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f57a0097380 con 0x7f57b81056f0 2024-02-19T16:32:57.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.148+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57b8108980 con 0x7f57b81056f0 2024-02-19T16:32:57.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.148+0000 7f57af7fe640 1 -- 172.21.15.8:0/2307143465 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f57a009aa40 con 0x7f57b81056f0 2024-02-19T16:32:57.150 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:32:57.150 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:32:57.151 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:32:57.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57a40a2230 msgr2=0x7f57a40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57a40a2230 0x7f57a40a46f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f57b0005ab0 tx=0x7f57b0005fd0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 msgr2=0x7f57b819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 0x7f57b819dd60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f57a000e9c0 tx=0x7f57a000ee80 comp rx=0 tx=0).stop 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 shutdown_connections 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b81979d0 0x7f57b8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57b81056f0 0x7f57b819dd60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f57a40a2230 0x7f57a40a46f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 --2- 172.21.15.8:0/2307143465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57b8100620 0x7f57b819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.150+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 >> 172.21.15.8:0/2307143465 conn(0x7f57b8077490 msgr2=0x7f57b80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.151+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 shutdown_connections 2024-02-19T16:32:57.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.151+0000 7f57bce5c640 1 -- 172.21.15.8:0/2307143465 wait complete. 2024-02-19T16:32:57.163 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:32:57.163 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-19T16:32:57.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 -- 172.21.15.8:0/2126091421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881056a0 msgr2=0x7fb38810d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 --2- 172.21.15.8:0/2126091421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881056a0 0x7fb38810d270 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb378009930 tx=0x7fb37802bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:57.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 -- 172.21.15.8:0/2126091421 shutdown_connections 2024-02-19T16:32:57.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 --2- 172.21.15.8:0/2126091421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881056a0 0x7fb38810d270 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 --2- 172.21.15.8:0/2126091421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb388100f80 0x7fb388104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 --2- 172.21.15.8:0/2126091421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3881005d0 0x7fb3881009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 -- 172.21.15.8:0/2126091421 >> 172.21.15.8:0/2126091421 conn(0x7fb3880774f0 msgr2=0x7fb388077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 -- 172.21.15.8:0/2126091421 shutdown_connections 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 -- 172.21.15.8:0/2126091421 wait complete. 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.266+0000 7fb38ed25640 1 Processor -- start 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 -- start start 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 0x7fb38819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3881056a0 0x7fb38819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb388196830 0x7fb388196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881972e0 con 0x7fb3881005d0 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388197450 con 0x7fb3881056a0 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881975c0 con 0x7fb388196830 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 0x7fb38819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 0x7fb38819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44152/0 (socket says 172.21.15.8:44152) 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 -- 172.21.15.8:0/1437934420 learned_addr learned my addr 172.21.15.8:0/1437934420 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb388196830 0x7fb388196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb387fff640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3881056a0 0x7fb38819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 -- 172.21.15.8:0/1437934420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb388196830 msgr2=0x7fb388196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb388196830 0x7fb388196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 -- 172.21.15.8:0/1437934420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3881056a0 msgr2=0x7fb38819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3881056a0 0x7fb38819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 -- 172.21.15.8:0/1437934420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378009560 con 0x7fb3881005d0 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38cac1640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 0x7fb38819c6c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb38000ebc0 tx=0x7fb38000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb385ffb640 1 -- 172.21.15.8:0/1437934420 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800099b0 con 0x7fb3881005d0 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb385ffb640 1 -- 172.21.15.8:0/1437934420 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb38000eec0 con 0x7fb3881005d0 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb385ffb640 1 -- 172.21.15.8:0/1437934420 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380018b60 con 0x7fb3881005d0 2024-02-19T16:32:57.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.267+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3881a3570 con 0x7fb3881005d0 2024-02-19T16:32:57.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.268+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3881a3a80 con 0x7fb3881005d0 2024-02-19T16:32:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.270+0000 7fb385ffb640 1 -- 172.21.15.8:0/1437934420 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb380018cc0 con 0x7fb3881005d0 2024-02-19T16:32:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 Processor -- start 2024-02-19T16:32:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 -- start start 2024-02-19T16:32:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 0x55b6d7da5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7d55960 con 0x55b6d7da3750 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7d09310 con 0x55b6d7d9a250 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb3919d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7d06840 con 0x55b6d7d9ab70 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 0x55b6d7da5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb38cac1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 0x55b6d7da5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44156/0 (socket says 172.21.15.8:44156) 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 -- 172.21.15.8:0/277012992 learned_addr learned my addr 172.21.15.8:0/277012992 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb385ffb640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb3700a22b0 0x7fb3700a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 -- 172.21.15.8:0/277012992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7d9ab70 msgr2=0x55b6d7da3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 -- 172.21.15.8:0/277012992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 msgr2=0x55b6d7d9a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d9a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 -- 172.21.15.8:0/277012992 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d7cc4060 con 0x55b6d7da3750 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb38cac1640 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb3700a22b0 0x7fb3700a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb385ffb640 1 -- 172.21.15.8:0/1437934420 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb380014070 con 0x7fb3881005d0 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.271+0000 7fb387fff640 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 0x55b6d7da5b40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009c00 tx=0x7fb37c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b90536b72a1ff3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb36effd640 1 -- 172.21.15.8:0/277012992 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c004050 con 0x55b6d7da3750 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb36effd640 1 -- 172.21.15.8:0/277012992 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb37c005500 con 0x55b6d7da3750 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb36effd640 1 -- 172.21.15.8:0/277012992 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c004ca0 con 0x55b6d7da3750 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 -- 172.21.15.8:0/277012992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 msgr2=0x55b6d7da5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb3700a22b0 0x7fb3700a4770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb378005940 tx=0x7fb378009260 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 0x55b6d7da5b40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009c00 tx=0x7fb37c011810 comp rx=0 tx=0).stop 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 -- 172.21.15.8:0/277012992 shutdown_connections 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7da3750 0x55b6d7da5b40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 --2- 172.21.15.8:0/277012992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 -- 172.21.15.8:0/277012992 >> 172.21.15.8:0/277012992 conn(0x55b6d7d94980 msgr2=0x55b6d7d94d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 -- 172.21.15.8:0/277012992 shutdown_connections 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.272+0000 7fb3919d9740 1 -- 172.21.15.8:0/277012992 wait complete. 2024-02-19T16:32:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 -- start start 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 Processor -- start 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 0x55b6d7dad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9ab70 0x55b6d7dadb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7d02020 con 0x55b6d7d9a250 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d79d5430 con 0x55b6d7d9ab70 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7991e00 con 0x55b6d7da3750 2024-02-19T16:32:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb3919d9740 10 client.0 _ll_register_callbacks cb 0x55b6d7ca9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb38cac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 0x55b6d7dad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb38cac1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 0x55b6d7dad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44168/0 (socket says 172.21.15.8:44168) 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb38cac1640 1 -- 172.21.15.8:0/2023685716 learned_addr learned my addr 172.21.15.8:0/2023685716 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.273+0000 7fb387fff640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9ab70 0x55b6d7dadb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb38cac1640 1 -- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 msgr2=0x55b6d7da7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb38cac1640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb38cac1640 1 -- 172.21.15.8:0/2023685716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9ab70 msgr2=0x55b6d7dadb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb38cac1640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9ab70 0x55b6d7dadb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb38cac1640 1 -- 172.21.15.8:0/2023685716 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c009830 con 0x55b6d7d9a250 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb38cac1640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 0x55b6d7dad610 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb380097b00 tx=0x7fb38009ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59b692185bc5640 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb367fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb367fff640 1 -- 172.21.15.8:0/2023685716 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38009a780 con 0x55b6d7d9a250 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb367fff640 1 -- 172.21.15.8:0/2023685716 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb38009f1d0 con 0x55b6d7d9a250 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6d7a017e0 con 0x55b6d7d9a250 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb367fff640 1 -- 172.21.15.8:0/2023685716 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38009a8e0 con 0x55b6d7d9a250 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 10 client.17820 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b6d7d006a0 con 0x55b6d7d9a250 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb367fff640 1 -- 172.21.15.8:0/2023685716 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b6d7a017e0 con 0x55b6d7d9a250 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb367fff640 1 -- 172.21.15.8:0/2023685716 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb38009a180 con 0x55b6d7d9a250 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 10 client.17820 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 10 client.17820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 20 client.17820 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7db7ec0 0x55b6d7dba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 4 client.17820 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.274+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6d7a3e720 con 0x55b6d7db7ec0 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.275+0000 7fb387fff640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7db7ec0 0x55b6d7dba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.275+0000 7fb367fff640 10 client.17820 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.275+0000 7fb387fff640 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7db7ec0 0x55b6d7dba2d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=500dcc34b37d3d63 server_cookie=d4bf262b4fccb5ca in_seq=0 out_seq=0 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb367fff640 1 -- 172.21.15.8:0/2023685716 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b6d7a3e720 con 0x55b6d7db7ec0 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb367fff640 10 client.17820 handle_command_reply: tid=0 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 client.17820 shutdown 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 msgr2=0x55b6d7dad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 0x55b6d7dad610 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb380097b00 tx=0x7fb38009ad40 comp rx=0 tx=0).stop 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 shutdown_connections 2024-02-19T16:32:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7db7ec0 0x55b6d7dba2d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9ab70 0x55b6d7dadb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 --2- 172.21.15.8:0/2023685716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9a250 0x55b6d7dad610 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb367fff640 0 client.17820 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 >> 172.21.15.8:0/2023685716 conn(0x55b6d7d94980 msgr2=0x55b6d7d966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 shutdown_connections 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 1 -- 172.21.15.8:0/2023685716 wait complete. 2024-02-19T16:32:57.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.276+0000 7fb3919d9740 20 client.17820 trim_cache size 0 max 16384 2024-02-19T16:32:57.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 Processor -- start 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 -- start start 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7c87130 con 0x55b6d7d9ab70 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7d95980 con 0x55b6d7d9a250 2024-02-19T16:32:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb3919d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7d99370 con 0x55b6d7da3750 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44170/0 (socket says 172.21.15.8:44170) 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 -- 172.21.15.8:0/3659443532 learned_addr learned my addr 172.21.15.8:0/3659443532 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb38cac1640 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 -- 172.21.15.8:0/3659443532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 msgr2=0x55b6d7da1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 -- 172.21.15.8:0/3659443532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 msgr2=0x55b6d7d8fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d8fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 -- 172.21.15.8:0/3659443532 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d7c9cc20 con 0x55b6d7d9ab70 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb387fff640 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da75e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb37c014040 tx=0x7fb37c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e94ae9a2c55523f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.289+0000 7fb36e7fc640 1 -- 172.21.15.8:0/3659443532 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0040d0 con 0x55b6d7d9ab70 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb36e7fc640 1 -- 172.21.15.8:0/3659443532 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb37c004230 con 0x55b6d7d9ab70 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb36e7fc640 1 -- 172.21.15.8:0/3659443532 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c02c7f0 con 0x55b6d7d9ab70 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 -- 172.21.15.8:0/3659443532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 msgr2=0x55b6d7da75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da75e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb37c014040 tx=0x7fb37c009680 comp rx=0 tx=0).stop 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 -- 172.21.15.8:0/3659443532 shutdown_connections 2024-02-19T16:32:57.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7da1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7da75e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3659443532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7d8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 -- 172.21.15.8:0/3659443532 >> 172.21.15.8:0/3659443532 conn(0x55b6d7d94980 msgr2=0x55b6d7d93b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 -- 172.21.15.8:0/3659443532 shutdown_connections 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 -- 172.21.15.8:0/3659443532 wait complete. 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 -- start start 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.290+0000 7fb3919d9740 1 Processor -- start 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7db1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7db1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7e12780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7db2480 con 0x55b6d7d9ab70 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7e12cc0 con 0x55b6d7d9a250 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7e12e00 con 0x55b6d7da3750 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb3919d9740 10 client.0 _ll_register_callbacks cb 0x7fb38000eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7db1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7db1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44176/0 (socket says 172.21.15.8:44176) 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 -- 172.21.15.8:0/3307510820 learned_addr learned my addr 172.21.15.8:0/3307510820 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb38d2c2640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7e12780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb38cac1640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7db1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 -- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 msgr2=0x55b6d7e12780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7e12780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 -- 172.21.15.8:0/3307510820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 msgr2=0x55b6d7db1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7db1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 -- 172.21.15.8:0/3307510820 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6d7cc4060 con 0x55b6d7d9ab70 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb387fff640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7db1bb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009b20 tx=0x7fb37c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980829f59fd728f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.291+0000 7fb367fff640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:57.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb367fff640 1 -- 172.21.15.8:0/3307510820 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0035f0 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb367fff640 1 -- 172.21.15.8:0/3307510820 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb37c003750 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb37c005500 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb367fff640 1 -- 172.21.15.8:0/3307510820 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb37c003930 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 10 client.17826 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b6d7cc28c0 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb367fff640 1 -- 172.21.15.8:0/3307510820 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb37c005500 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb367fff640 1 -- 172.21.15.8:0/3307510820 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb37c021040 con 0x55b6d7d9ab70 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 10 client.17826 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:57.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 10 client.17826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 20 client.17826 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7e14250 0x55b6d7e16670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 4 client.17826 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 --> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6d7a8d890 con 0x55b6d7e14250 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.292+0000 7fb38cac1640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7e14250 0x55b6d7e16670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb367fff640 10 client.17826 ms_handle_connect on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb38cac1640 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7e14250 0x55b6d7e16670 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f94070e48669615b server_cookie=e11c1a4d7398ba4c in_seq=0 out_seq=0 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb367fff640 1 -- 172.21.15.8:0/3307510820 <== mds.2 v2:172.21.15.191:6836/1523568014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6d7a8d890 con 0x55b6d7e14250 2024-02-19T16:32:57.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb367fff640 10 client.17826 handle_command_reply: tid=0 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 client.17826 shutdown 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 msgr2=0x55b6d7db1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7db1bb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009b20 tx=0x7fb37c003f60 comp rx=0 tx=0).stop 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 shutdown_connections 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:6836/1523568014,v1:172.21.15.191:6837/1523568014] conn(0x55b6d7e14250 0x55b6d7e16670 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6d7da3750 0x55b6d7e12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb367fff640 0 client.17826 ms_handle_reset on v2:172.21.15.191:6836/1523568014 2024-02-19T16:32:57.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b6d7d9ab70 0x55b6d7db1bb0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 --2- 172.21.15.8:0/3307510820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6d7d9a250 0x55b6d7db1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 >> 172.21.15.8:0/3307510820 conn(0x55b6d7d94980 msgr2=0x55b6d7d95d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 shutdown_connections 2024-02-19T16:32:57.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 1 -- 172.21.15.8:0/3307510820 wait complete. 2024-02-19T16:32:57.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.293+0000 7fb3919d9740 20 client.17826 trim_cache size 0 max 16384 2024-02-19T16:32:57.306 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb3700a22b0 msgr2=0x7fb3700a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb3700a22b0 0x7fb3700a4770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb378005940 tx=0x7fb378009260 comp rx=0 tx=0).stop 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 msgr2=0x7fb38819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 0x7fb38819c6c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb38000ebc0 tx=0x7fb38000c6a0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 shutdown_connections 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb3700a22b0 0x7fb3700a4770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb388196830 0x7fb388196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3881056a0 0x7fb38819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 --2- 172.21.15.8:0/1437934420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3881005d0 0x7fb38819c6c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 >> 172.21.15.8:0/1437934420 conn(0x7fb3880774f0 msgr2=0x7fb3880fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 shutdown_connections 2024-02-19T16:32:57.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.294+0000 7fb38ed25640 1 -- 172.21.15.8:0/1437934420 wait complete. 2024-02-19T16:32:57.309 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:32:57.309 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-19T16:32:57.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 -- 172.21.15.8:0/4201066965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3041005f0 msgr2=0x7fb3041009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/4201066965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3041005f0 0x7fb3041009d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb300009900 tx=0x7fb30002bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:57.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 -- 172.21.15.8:0/4201066965 shutdown_connections 2024-02-19T16:32:57.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/4201066965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3041056c0 0x7fb30410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/4201066965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb304100fa0 0x7fb304104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/4201066965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3041005f0 0x7fb3041009d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 -- 172.21.15.8:0/4201066965 >> 172.21.15.8:0/4201066965 conn(0x7fb3040774c0 msgr2=0x7fb3040778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 -- 172.21.15.8:0/4201066965 shutdown_connections 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.412+0000 7fb30c5d7640 1 -- 172.21.15.8:0/4201066965 wait complete. 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 Processor -- start 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 -- start start 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 0x7fb304198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3041056c0 0x7fb304198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb304199420 0x7fb3041a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041a4070 con 0x7fb3041005f0 2024-02-19T16:32:57.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041a41e0 con 0x7fb3041056c0 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30c5d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041a43a0 con 0x7fb304199420 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 0x7fb304198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 0x7fb304198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44194/0 (socket says 172.21.15.8:44194) 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 -- 172.21.15.8:0/289227616 learned_addr learned my addr 172.21.15.8:0/289227616 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30ab74640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb304199420 0x7fb3041a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb309b72640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3041056c0 0x7fb304198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 -- 172.21.15.8:0/289227616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb304199420 msgr2=0x7fb3041a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb304199420 0x7fb3041a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 -- 172.21.15.8:0/289227616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3041056c0 msgr2=0x7fb304198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3041056c0 0x7fb304198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 -- 172.21.15.8:0/289227616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009560 con 0x7fb3041005f0 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30ab74640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb304199420 0x7fb3041a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.413+0000 7fb30a373640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 0x7fb304198950 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb300005940 tx=0x7fb30002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.414+0000 7fb2fb7fe640 1 -- 172.21.15.8:0/289227616 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000042d0 con 0x7fb3041005f0 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.414+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3041a4620 con 0x7fb3041005f0 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.414+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3041a4b80 con 0x7fb3041005f0 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.414+0000 7fb2fb7fe640 1 -- 172.21.15.8:0/289227616 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb300004430 con 0x7fb3041005f0 2024-02-19T16:32:57.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.414+0000 7fb2fb7fe640 1 -- 172.21.15.8:0/289227616 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300003be0 con 0x7fb3041005f0 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.416+0000 7fb2fb7fe640 1 -- 172.21.15.8:0/289227616 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb300003d40 con 0x7fb3041005f0 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.416+0000 7fb30f1b8740 1 Processor -- start 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.416+0000 7fb30f1b8740 1 -- start start 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e07630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e10210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e12b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220dc2960 con 0x556220e10750 2024-02-19T16:32:57.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220d76310 con 0x556220e07250 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220d73840 con 0x556220e07b70 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb2fb7fe640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb2f00a2230 0x7fb2f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb309b72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e12b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e10210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb309b72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e12b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44208/0 (socket says 172.21.15.8:44208) 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb309b72640 1 -- 172.21.15.8:0/2610684591 learned_addr learned my addr 172.21.15.8:0/2610684591 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30ab74640 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e07630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30ab74640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb2f00a2230 0x7fb2f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb2fb7fe640 1 -- 172.21.15.8:0/289227616 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb3000f0140 con 0x7fb3041005f0 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 -- 172.21.15.8:0/2610684591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 msgr2=0x556220e07630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e07630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 -- 172.21.15.8:0/2610684591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 msgr2=0x556220e12b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e12b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 -- 172.21.15.8:0/2610684591 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556220d31060 con 0x556220e07b70 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb309b72640 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e12b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30ab74640 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e07630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30a373640 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e10210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3000ed580 tx=0x7fb30008f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9211b746c9eab0e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.417+0000 7fb30ab74640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb2f00a2230 0x7fb2f00a46f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0099e0 tx=0x7fb2fc009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb2f8bfe640 1 -- 172.21.15.8:0/2610684591 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000bcf20 con 0x556220e07b70 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb2f8bfe640 1 -- 172.21.15.8:0/2610684591 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3000bc820 con 0x556220e07b70 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb2f8bfe640 1 -- 172.21.15.8:0/2610684591 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000b8e90 con 0x556220e07b70 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2610684591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 msgr2=0x556220e10210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e10210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3000ed580 tx=0x7fb30008f890 comp rx=0 tx=0).stop 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2610684591 shutdown_connections 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e12b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e10210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2610684591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e07630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2610684591 >> 172.21.15.8:0/2610684591 conn(0x556220e01980 msgr2=0x556220e01d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.418+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2610684591 shutdown_connections 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2610684591 wait complete. 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 -- start start 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 Processor -- start 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e0e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220d6f020 con 0x556220e10750 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220a42430 con 0x556220e07b70 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562209fee00 con 0x556220e07250 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30f1b8740 10 client.0 _ll_register_callbacks cb 0x556220d16cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30ab74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e0e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30ab74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e0e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44224/0 (socket says 172.21.15.8:44224) 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30ab74640 1 -- 172.21.15.8:0/521680233 learned_addr learned my addr 172.21.15.8:0/521680233 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb30a373640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.419+0000 7fb309b72640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e0dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30ab74640 1 -- 172.21.15.8:0/521680233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 msgr2=0x556220e0d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30ab74640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30ab74640 1 -- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 msgr2=0x556220e0dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30ab74640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e0dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30ab74640 1 -- 172.21.15.8:0/521680233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3041a4b80 con 0x556220e10750 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30a373640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30ab74640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e0e180 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc01f990 tx=0x7fb2fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db221a744b68b471 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb2edffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb2edffb640 1 -- 172.21.15.8:0/521680233 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc024070 con 0x556220e10750 2024-02-19T16:32:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb2edffb640 1 -- 172.21.15.8:0/521680233 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb2fc00fe40 con 0x556220e10750 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556220a6e7e0 con 0x556220e10750 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb2edffb640 1 -- 172.21.15.8:0/521680233 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2fc01fe50 con 0x556220e10750 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30f1b8740 10 client.17838 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556220d6d6a0 con 0x556220e10750 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb2edffb640 1 -- 172.21.15.8:0/521680233 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556220a6e7e0 con 0x556220e10750 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb2edffb640 1 -- 172.21.15.8:0/521680233 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb2fc004020 con 0x556220e10750 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.420+0000 7fb30f1b8740 10 client.17838 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:57.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30f1b8740 10 client.17838 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30f1b8740 20 client.17838 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e209c0 0x556220e22de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30f1b8740 4 client.17838 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556220aab720 con 0x556220e209c0 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30a373640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e209c0 0x556220e22de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb30a373640 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e209c0 0x556220e22de0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ecd340a38f281ed2 server_cookie=c741120b650e76b3 in_seq=0 out_seq=0 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.421+0000 7fb2edffb640 10 client.17838 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb2edffb640 1 -- 172.21.15.8:0/521680233 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556220aab720 con 0x556220e209c0 2024-02-19T16:32:57.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb2edffb640 10 client.17838 handle_command_reply: tid=0 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 client.17838 shutdown 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 msgr2=0x556220e0e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e0e180 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc01f990 tx=0x7fb2fc004a40 comp rx=0 tx=0).stop 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 shutdown_connections 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e209c0 0x556220e22de0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e0e180 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb2edffb640 0 client.17838 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/521680233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 >> 172.21.15.8:0/521680233 conn(0x556220e01980 msgr2=0x556220e036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 shutdown_connections 2024-02-19T16:32:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 1 -- 172.21.15.8:0/521680233 wait complete. 2024-02-19T16:32:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.422+0000 7fb30f1b8740 20 client.17838 trim_cache size 0 max 16384 2024-02-19T16:32:57.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.434+0000 7fb30f1b8740 1 Processor -- start 2024-02-19T16:32:57.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.434+0000 7fb30f1b8740 1 -- start start 2024-02-19T16:32:57.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e1b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220cf4130 con 0x556220e10750 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220e12b40 con 0x556220e07b70 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220dfca40 con 0x556220e07250 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e1b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e1b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44230/0 (socket says 172.21.15.8:44230) 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 -- 172.21.15.8:0/2731446814 learned_addr learned my addr 172.21.15.8:0/2731446814 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30a373640 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb309b72640 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 -- 172.21.15.8:0/2731446814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 msgr2=0x556220e0dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 -- 172.21.15.8:0/2731446814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 msgr2=0x556220e034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 -- 172.21.15.8:0/2731446814 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556220d09c20 con 0x556220e10750 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb309b72640 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb30ab74640 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e1b8f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc006010 tx=0x7fb2fc005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed45c33b493c1e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb2effff640 1 -- 172.21.15.8:0/2731446814 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc024040 con 0x556220e10750 2024-02-19T16:32:57.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb2effff640 1 -- 172.21.15.8:0/2731446814 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2fc01f720 con 0x556220e10750 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.435+0000 7fb2effff640 1 -- 172.21.15.8:0/2731446814 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc003ed0 con 0x556220e10750 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2731446814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 msgr2=0x556220e1b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e1b8f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc006010 tx=0x7fb2fc005c80 comp rx=0 tx=0).stop 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2731446814 shutdown_connections 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e1b8f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07b70 0x556220e034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/2731446814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07250 0x556220e0dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2731446814 >> 172.21.15.8:0/2731446814 conn(0x556220e01980 msgr2=0x556220e02db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2731446814 shutdown_connections 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- 172.21.15.8:0/2731446814 wait complete. 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- start start 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 Processor -- start 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e14080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e75240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220e75780 con 0x556220e10750 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220e758f0 con 0x556220e07250 2024-02-19T16:32:57.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556220e75ab0 con 0x556220e07b70 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.436+0000 7fb30f1b8740 10 client.0 _ll_register_callbacks cb 0x7fb2fc0115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e75240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30a373640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e14080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e75240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44232/0 (socket says 172.21.15.8:44232) 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 -- 172.21.15.8:0/1469417901 learned_addr learned my addr 172.21.15.8:0/1469417901 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb309b72640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 -- 172.21.15.8:0/1469417901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 msgr2=0x556220e145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 -- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 msgr2=0x556220e14080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e14080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 -- 172.21.15.8:0/1469417901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556220d6d6a0 con 0x556220e10750 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30a373640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e14080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30ab74640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e75240 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc023010 tx=0x7fb2fc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e041f4d2936c9e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb2edffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb2edffb640 1 -- 172.21.15.8:0/1469417901 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc024040 con 0x556220e10750 2024-02-19T16:32:57.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb2edffb640 1 -- 172.21.15.8:0/1469417901 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb2fc01d970 con 0x556220e10750 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3000bc820 con 0x556220e10750 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb2edffb640 1 -- 172.21.15.8:0/1469417901 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2fc01db30 con 0x556220e10750 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30f1b8740 10 client.17844 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556220d2f8c0 con 0x556220e10750 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.437+0000 7fb2edffb640 1 -- 172.21.15.8:0/1469417901 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3000bc820 con 0x556220e10750 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb2edffb640 1 -- 172.21.15.8:0/1469417901 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb2fc040070 con 0x556220e10750 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 10 client.17844 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 10 client.17844 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-02-19T16:32:57.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 20 client.17844 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e7d000 0x556220e7f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 4 client.17844 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 --> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556220afa890 con 0x556220e7d000 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30a373640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e7d000 0x556220e7f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30a373640 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e7d000 0x556220e7f420 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b73d327948a9a81 server_cookie=4ea59fa07c909d2a in_seq=0 out_seq=0 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb2edffb640 10 client.17844 ms_handle_connect on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb2edffb640 1 -- 172.21.15.8:0/1469417901 <== mds.1 v2:172.21.15.156:6838/3057541491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556220afa890 con 0x556220e7d000 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb2edffb640 10 client.17844 handle_command_reply: tid=0 2024-02-19T16:32:57.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 client.17844 shutdown 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 msgr2=0x556220e75240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e75240 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc023010 tx=0x7fb2fc003970 comp rx=0 tx=0).stop 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 shutdown_connections 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:6838/3057541491,v1:172.21.15.156:6839/3057541491] conn(0x556220e7d000 0x556220e7f420 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556220e10750 0x556220e75240 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556220e07b70 0x556220e145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 --2- 172.21.15.8:0/1469417901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556220e07250 0x556220e14080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb2edffb640 0 client.17844 ms_handle_reset on v2:172.21.15.156:6838/3057541491 2024-02-19T16:32:57.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.438+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 >> 172.21.15.8:0/1469417901 conn(0x556220e01980 msgr2=0x556220e03520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 shutdown_connections 2024-02-19T16:32:57.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30f1b8740 1 -- 172.21.15.8:0/1469417901 wait complete. 2024-02-19T16:32:57.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30f1b8740 20 client.17844 trim_cache size 0 max 16384 2024-02-19T16:32:57.453 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:32:57.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb2f00a2230 msgr2=0x7fb2f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb2f00a2230 0x7fb2f00a46f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0099e0 tx=0x7fb2fc009500 comp rx=0 tx=0).stop 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 msgr2=0x7fb304198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 0x7fb304198950 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb300005940 tx=0x7fb30002c670 comp rx=0 tx=0).stop 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 shutdown_connections 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb2f00a2230 0x7fb2f00a46f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb304199420 0x7fb3041a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3041056c0 0x7fb304198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 --2- 172.21.15.8:0/289227616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3041005f0 0x7fb304198950 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 >> 172.21.15.8:0/289227616 conn(0x7fb3040774c0 msgr2=0x7fb3040fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 shutdown_connections 2024-02-19T16:32:57.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.439+0000 7fb30c5d7640 1 -- 172.21.15.8:0/289227616 wait complete. 2024-02-19T16:32:57.456 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-19T16:32:57.456 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-19T16:32:57.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.559+0000 7fee3cdf6640 1 -- 172.21.15.8:0/4197618590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee380686f0 msgr2=0x7fee38068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.559+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/4197618590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee380686f0 0x7fee38068b70 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fee20009930 tx=0x7fee2002bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:57.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.559+0000 7fee3cdf6640 1 -- 172.21.15.8:0/4197618590 shutdown_connections 2024-02-19T16:32:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.559+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/4197618590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380690b0 0x7fee3810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/4197618590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee380686f0 0x7fee38068b70 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/4197618590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee381074f0 0x7fee381078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- 172.21.15.8:0/4197618590 >> 172.21.15.8:0/4197618590 conn(0x7fee380774f0 msgr2=0x7fee38077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- 172.21.15.8:0/4197618590 shutdown_connections 2024-02-19T16:32:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- 172.21.15.8:0/4197618590 wait complete. 2024-02-19T16:32:57.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 Processor -- start 2024-02-19T16:32:57.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- start start 2024-02-19T16:32:57.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 0x7fee3819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee380690b0 0x7fee3819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 0x7fee38197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38197f70 con 0x7fee381074f0 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee381980e0 con 0x7fee380686f0 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.560+0000 7fee3cdf6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee381982a0 con 0x7fee380690b0 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 0x7fee38197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee3659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 0x7fee3819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 0x7fee38197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44248/0 (socket says 172.21.15.8:44248) 2024-02-19T16:32:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee3659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 0x7fee3819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.8:49636/0 (socket says 172.21.15.8:49636) 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 -- 172.21.15.8:0/1749622036 learned_addr learned my addr 172.21.15.8:0/1749622036 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee35d9b640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee380690b0 0x7fee3819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 -- 172.21.15.8:0/1749622036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee380690b0 msgr2=0x7fee3819dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee380690b0 0x7fee3819dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 -- 172.21.15.8:0/1749622036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 msgr2=0x7fee3819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 0x7fee3819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 -- 172.21.15.8:0/1749622036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009560 con 0x7fee381074f0 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee3659c640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 0x7fee3819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee36d9d640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 0x7fee38197920 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fee2800eb10 tx=0x7fee2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee2f7fe640 1 -- 172.21.15.8:0/1749622036 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28004b20 con 0x7fee381074f0 2024-02-19T16:32:57.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee2f7fe640 1 -- 172.21.15.8:0/1749622036 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee28004c80 con 0x7fee381074f0 2024-02-19T16:32:57.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee38198580 con 0x7fee381074f0 2024-02-19T16:32:57.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee2f7fe640 1 -- 172.21.15.8:0/1749622036 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28010650 con 0x7fee381074f0 2024-02-19T16:32:57.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.561+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee381a4810 con 0x7fee381074f0 2024-02-19T16:32:57.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.563+0000 7fee2f7fe640 1 -- 172.21.15.8:0/1749622036 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fee28005020 con 0x7fee381074f0 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 Processor -- start 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee2f7fe640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fee240a2230 0x7fee240a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee35d9b640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fee240a2230 0x7fee240a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 -- start start 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee2f7fe640 1 -- 172.21.15.8:0/1749622036 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fee28014070 con 0x7fee381074f0 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0ee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde0a9960 con 0x563fde0f7750 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde05d310 con 0x563fde0eeb70 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde05a840 con 0x563fde0ee250 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44262/0 (socket says 172.21.15.8:44262) 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee36d9d640 1 -- 172.21.15.8:0/689938405 learned_addr learned my addr 172.21.15.8:0/689938405 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee3659c640 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee35d9b640 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.564+0000 7fee35d9b640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fee240a2230 0x7fee240a46f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee20005ab0 tx=0x7fee20005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee36d9d640 1 -- 172.21.15.8:0/689938405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 msgr2=0x563fde0ee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee36d9d640 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0ee630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee36d9d640 1 -- 172.21.15.8:0/689938405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 msgr2=0x563fde0f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee36d9d640 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee36d9d640 1 -- 172.21.15.8:0/689938405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fde018060 con 0x563fde0f7750 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee3659c640 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0ee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:57.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee36d9d640 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f9b40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fee280a32e0 tx=0x7fee28009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4db498efe0b3da2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee2cff9640 1 -- 172.21.15.8:0/689938405 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280a3810 con 0x563fde0f7750 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee2cff9640 1 -- 172.21.15.8:0/689938405 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee28099370 con 0x563fde0f7750 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.565+0000 7fee2cff9640 1 -- 172.21.15.8:0/689938405 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2809b4f0 con 0x563fde0f7750 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- 172.21.15.8:0/689938405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 msgr2=0x563fde0f9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f9b40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fee280a32e0 tx=0x7fee28009a70 comp rx=0 tx=0).stop 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- 172.21.15.8:0/689938405 shutdown_connections 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f9b40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/689938405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0ee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- 172.21.15.8:0/689938405 >> 172.21.15.8:0/689938405 conn(0x563fde0e8980 msgr2=0x563fde0e8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- 172.21.15.8:0/689938405 shutdown_connections 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- 172.21.15.8:0/689938405 wait complete. 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- start start 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 Processor -- start 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde056020 con 0x563fde0f7750 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fddd29430 con 0x563fde0eeb70 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fddce5e00 con 0x563fde0ee250 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.566+0000 7fee3f9cb740 10 client.0 _ll_register_callbacks cb 0x563fddffdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44272/0 (socket says 172.21.15.8:44272) 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee35d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 -- 172.21.15.8:0/2633411557 learned_addr learned my addr 172.21.15.8:0/2633411557 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee3659c640 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 -- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 msgr2=0x563fde0f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 -- 172.21.15.8:0/2633411557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 msgr2=0x563fde0f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 -- 172.21.15.8:0/2633411557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee381a4810 con 0x563fde0f7750 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee36d9d640 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f51e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fee280974d0 tx=0x7fee28097910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51205e3103ba38a server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee1dffb640 1 -- 172.21.15.8:0/2633411557 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280a3810 con 0x563fde0f7750 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee1dffb640 1 -- 172.21.15.8:0/2633411557 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fee280a3380 con 0x563fde0f7750 2024-02-19T16:32:57.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fddd557e0 con 0x563fde0f7750 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee1dffb640 1 -- 172.21.15.8:0/2633411557 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee280a3540 con 0x563fde0f7750 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee3f9cb740 10 client.17856 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563fde0546a0 con 0x563fde0f7750 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.567+0000 7fee1dffb640 1 -- 172.21.15.8:0/2633411557 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563fddd557e0 con 0x563fde0f7750 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee1dffb640 1 -- 172.21.15.8:0/2633411557 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fee2801f060 con 0x563fde0f7750 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3f9cb740 10 client.17856 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3f9cb740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3f9cb740 20 client.17856 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:57.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde107840 0x563fde109c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3f9cb740 4 client.17856 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-19T16:32:57.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fddd92720 con 0x563fde107840 2024-02-19T16:32:57.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3659c640 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde107840 0x563fde109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee1dffb640 10 client.17856 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:57.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.568+0000 7fee3659c640 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde107840 0x563fde109c30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3a0734d077faf7 server_cookie=fa71781cf5476269 in_seq=0 out_seq=0 2024-02-19T16:32:57.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee1dffb640 1 -- 172.21.15.8:0/2633411557 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563fddd92720 con 0x563fde107840 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee1dffb640 10 client.17856 handle_command_reply: tid=0 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 client.17856 shutdown 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 msgr2=0x563fde0f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f51e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fee280974d0 tx=0x7fee28097910 comp rx=0 tx=0).stop 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 shutdown_connections 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde0f51e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde107840 0x563fde109c30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/2633411557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee1dffb640 0 client.17856 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:57.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 >> 172.21.15.8:0/2633411557 conn(0x563fde0e8980 msgr2=0x563fde0ea6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 shutdown_connections 2024-02-19T16:32:57.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 1 -- 172.21.15.8:0/2633411557 wait complete. 2024-02-19T16:32:57.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.569+0000 7fee3f9cb740 20 client.17856 trim_cache size 0 max 16384 2024-02-19T16:32:57.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.581+0000 7fee3f9cb740 1 Processor -- start 2024-02-19T16:32:57.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.581+0000 7fee3f9cb740 1 -- start start 2024-02-19T16:32:57.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.581+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.581+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fddfdb130 con 0x563fde0f7750 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde10a290 con 0x563fde0eeb70 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde107670 con 0x563fde0ee250 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde102ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44276/0 (socket says 172.21.15.8:44276) 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 -- 172.21.15.8:0/3141335946 learned_addr learned my addr 172.21.15.8:0/3141335946 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee3659c640 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee35d9b640 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 -- 172.21.15.8:0/3141335946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 msgr2=0x563fde0f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 -- 172.21.15.8:0/3141335946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 msgr2=0x563fde0f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 -- 172.21.15.8:0/3141335946 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fddff0c20 con 0x563fde0f7750 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee36d9d640 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde102ad0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fee28097a40 tx=0x7fee28009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2af611b459cb95 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee1ffff640 1 -- 172.21.15.8:0/3141335946 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2809a9d0 con 0x563fde0f7750 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee1ffff640 1 -- 172.21.15.8:0/3141335946 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee28079e90 con 0x563fde0f7750 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.582+0000 7fee1ffff640 1 -- 172.21.15.8:0/3141335946 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28084080 con 0x563fde0f7750 2024-02-19T16:32:57.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- 172.21.15.8:0/3141335946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 msgr2=0x563fde102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde102ad0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fee28097a40 tx=0x7fee28009a70 comp rx=0 tx=0).stop 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- 172.21.15.8:0/3141335946 shutdown_connections 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde102ad0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0eeb70 0x563fde0f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/3141335946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0ee250 0x563fde0f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- 172.21.15.8:0/3141335946 >> 172.21.15.8:0/3141335946 conn(0x563fde0e8980 msgr2=0x563fde0e7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- 172.21.15.8:0/3141335946 shutdown_connections 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- 172.21.15.8:0/3141335946 wait complete. 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- start start 2024-02-19T16:32:57.593 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 Processor -- start 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0ee250 0x563fde0f1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0eeb70 0x563fde0f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde1620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde0f2040 con 0x563fde0f7750 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde162600 con 0x563fde0ee250 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.583+0000 7fee3f9cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fde162740 con 0x563fde0eeb70 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee3f9cb740 10 client.0 _ll_register_callbacks cb 0x7fee30005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-19T16:32:57.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde1620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde1620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44284/0 (socket says 172.21.15.8:44284) 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 -- 172.21.15.8:0/1875877615 learned_addr learned my addr 172.21.15.8:0/1875877615 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee3659c640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0ee250 0x563fde0f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee35d9b640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0eeb70 0x563fde0f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 -- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0eeb70 msgr2=0x563fde0f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0eeb70 0x563fde0f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 -- 172.21.15.8:0/1875877615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0ee250 msgr2=0x563fde0f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0ee250 0x563fde0f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 -- 172.21.15.8:0/1875877615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fde0546a0 con 0x563fde0f7750 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee36d9d640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde1620c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fee28099320 tx=0x7fee280a3380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ac6157cfa7a67d server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee1dffb640 1 -- 172.21.15.8:0/1875877615 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28084240 con 0x563fde0f7750 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee1dffb640 1 -- 172.21.15.8:0/1875877615 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fee280a3810 con 0x563fde0f7750 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fde0e40f0 con 0x563fde0f7750 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee1dffb640 1 -- 172.21.15.8:0/1875877615 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee280843a0 con 0x563fde0f7750 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee3f9cb740 10 client.17862 fetch_fsmap learned FSMap version 30 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.584+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563fde0168c0 con 0x563fde0f7750 2024-02-19T16:32:57.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee1dffb640 1 -- 172.21.15.8:0/1875877615 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563fde0e40f0 con 0x563fde0f7750 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee1dffb640 1 -- 172.21.15.8:0/1875877615 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fee2801f070 con 0x563fde0f7750 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3f9cb740 10 client.17862 fetch_fsmap finished waiting for FSMap version 30 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3f9cb740 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3f9cb740 20 client.17862 populate_metadata read hostname 'smithi008' 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde163960 0x563fde165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3f9cb740 4 client.17862 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 --> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fee28099370 con 0x563fde163960 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.585+0000 7fee3659c640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde163960 0x563fde165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:57.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3659c640 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde163960 0x563fde165d50 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c44349572a1088b server_cookie=d4cbfdbde5d2fdb0 in_seq=0 out_seq=0 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee1dffb640 10 client.17862 ms_handle_connect on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee1dffb640 1 -- 172.21.15.8:0/1875877615 <== mds.0 v2:172.21.15.191:6838/2229091821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fee28099370 con 0x563fde163960 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee1dffb640 10 client.17862 handle_command_reply: tid=0 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 client.17862 shutdown 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 msgr2=0x563fde1620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde1620c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fee28099320 tx=0x7fee280a3380 comp rx=0 tx=0).stop 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 shutdown_connections 2024-02-19T16:32:57.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563fde0f7750 0x563fde1620c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:6838/2229091821,v1:172.21.15.191:6839/2229091821] conn(0x563fde163960 0x563fde165d50 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563fde0eeb70 0x563fde0f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 --2- 172.21.15.8:0/1875877615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563fde0ee250 0x563fde0f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee1dffb640 0 client.17862 ms_handle_reset on v2:172.21.15.191:6838/2229091821 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 >> 172.21.15.8:0/1875877615 conn(0x563fde0e8980 msgr2=0x563fde10e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 shutdown_connections 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 1 -- 172.21.15.8:0/1875877615 wait complete. 2024-02-19T16:32:57.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3f9cb740 20 client.17862 trim_cache size 0 max 16384 2024-02-19T16:32:57.600 INFO:teuthology.orchestra.run.smithi008.stdout:[] 2024-02-19T16:32:57.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fee240a2230 msgr2=0x7fee240a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fee240a2230 0x7fee240a46f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee20005ab0 tx=0x7fee20005fd0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.586+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 msgr2=0x7fee38197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:57.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 0x7fee38197920 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fee2800eb10 tx=0x7fee2800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 shutdown_connections 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee381074f0 0x7fee38197920 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fee240a2230 0x7fee240a46f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee380690b0 0x7fee3819dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 --2- 172.21.15.8:0/1749622036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee380686f0 0x7fee3819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 >> 172.21.15.8:0/1749622036 conn(0x7fee380774f0 msgr2=0x7fee380ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 shutdown_connections 2024-02-19T16:32:57.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:57.587+0000 7fee3cdf6640 1 -- 172.21.15.8:0/1749622036 wait complete. 2024-02-19T16:32:57.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:57 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2307143465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:32:57.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:57 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2307143465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:32:58.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:57 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2307143465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:32:58.603 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-02-19T16:32:58.604 INFO:tasks.fwd_scrub:done joining 2024-02-19T16:32:58.604 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-19T16:32:58.614 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:32:58.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 -- 172.21.15.8:0/3839506817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 msgr2=0x7f880c10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:58.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 --2- 172.21.15.8:0/3839506817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c10e400 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f880402c0f0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 -- 172.21.15.8:0/3839506817 shutdown_connections 2024-02-19T16:32:58.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 --2- 172.21.15.8:0/3839506817 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c10e400 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 --2- 172.21.15.8:0/3839506817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f880c06a7a0 0x7f880c106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 --2- 172.21.15.8:0/3839506817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 0x7f880c06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 -- 172.21.15.8:0/3839506817 >> 172.21.15.8:0/3839506817 conn(0x7f880c077490 msgr2=0x7f880c0778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 -- 172.21.15.8:0/3839506817 shutdown_connections 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.727+0000 7f88147c7640 1 -- 172.21.15.8:0/3839506817 wait complete. 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 Processor -- start 2024-02-19T16:32:58.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 -- start start 2024-02-19T16:32:58.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 0x7f880c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:58.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:58.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f880c1978a0 0x7f880c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:58.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c198350 con 0x7f880c106880 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c1984c0 con 0x7f880c069df0 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f88147c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c198630 con 0x7f880c1978a0 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44290/0 (socket says 172.21.15.8:44290) 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 -- 172.21.15.8:0/2614889773 learned_addr learned my addr 172.21.15.8:0/2614889773 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8812563640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 0x7f880c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8812d64640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f880c1978a0 0x7f880c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 -- 172.21.15.8:0/2614889773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f880c1978a0 msgr2=0x7f880c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:58.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f880c1978a0 0x7f880c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 -- 172.21.15.8:0/2614889773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 msgr2=0x7f880c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 0x7f880c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8811d62640 1 -- 172.21.15.8:0/2614889773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8808009490 con 0x7f880c106880 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.728+0000 7f8812563640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 0x7f880c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.729+0000 7f8811d62640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c19dc20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e9f0 tx=0x7f87fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.729+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0096c0 con 0x7f880c106880 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.729+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87fc005500 con 0x7f880c106880 2024-02-19T16:32:58.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.729+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8804009900 con 0x7f880c106880 2024-02-19T16:32:58.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.729+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc010430 con 0x7f880c106880 2024-02-19T16:32:58.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.729+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c1a4890 con 0x7f880c106880 2024-02-19T16:32:58.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.730+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c102c10 con 0x7f880c106880 2024-02-19T16:32:58.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.731+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f87fc005000 con 0x7f880c106880 2024-02-19T16:32:58.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.732+0000 7f88037fe640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87f80a2230 0x7f87f80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:58.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.732+0000 7f8812563640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87f80a2230 0x7f87f80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:58.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.732+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f87fc014070 con 0x7f880c106880 2024-02-19T16:32:58.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.732+0000 7f8812563640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87f80a2230 0x7f87f80a46f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8808011630 tx=0x7f8808005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:58.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.735+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f87fc097290 con 0x7f880c106880 2024-02-19T16:32:58.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:58 smithi156 ceph-mon[36502]: pgmap v313: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:58.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.918+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f880c068730 con 0x7f880c106880 2024-02-19T16:32:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.920+0000 7f88037fe640 1 -- 172.21.15.8:0/2614889773 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+25301 (secure 0 0 0) 0x7f87fc09a950 con 0x7f880c106880 2024-02-19T16:32:58.922 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:32:58.922 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":104,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:32:16.464249+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/1905912286":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/543580195":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2140457421":"2024-02-20T16:22:52.539698+0000","172.21.15.8:6801/354789085":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/354789085":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2267597929":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/3384401098":"2024-02-20T16:22:52.539698+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87f80a2230 msgr2=0x7f87f80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87f80a2230 0x7f87f80a46f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8808011630 tx=0x7f8808005f90 comp rx=0 tx=0).stop 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 msgr2=0x7f880c19dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c19dc20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e9f0 tx=0x7f87fc00eeb0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 shutdown_connections 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f87f80a2230 0x7f87f80a46f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f880c1978a0 0x7f880c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f880c106880 0x7f880c19dc20 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 --2- 172.21.15.8:0/2614889773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f880c069df0 0x7f880c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:58.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 >> 172.21.15.8:0/2614889773 conn(0x7f880c077490 msgr2=0x7f880c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:58.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 shutdown_connections 2024-02-19T16:32:58.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:58.922+0000 7f88147c7640 1 -- 172.21.15.8:0/2614889773 wait complete. 2024-02-19T16:32:58.933 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:58 smithi008 ceph-mon[32573]: pgmap v313: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:58.934 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:32:58.935 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:32:59.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:58 smithi191 ceph-mon[37203]: pgmap v313: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:32:59.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 -- 172.21.15.8:0/4057268673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad8100610 msgr2=0x7faad81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 --2- 172.21.15.8:0/4057268673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad8100610 0x7faad81009f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7faad4009930 tx=0x7faad402bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:59.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 -- 172.21.15.8:0/4057268673 shutdown_connections 2024-02-19T16:32:59.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 --2- 172.21.15.8:0/4057268673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad81056e0 0x7faad810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 --2- 172.21.15.8:0/4057268673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faad8100fc0 0x7faad8105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 --2- 172.21.15.8:0/4057268673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad8100610 0x7faad81009f0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 -- 172.21.15.8:0/4057268673 >> 172.21.15.8:0/4057268673 conn(0x7faad8076770 msgr2=0x7faad8076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 -- 172.21.15.8:0/4057268673 shutdown_connections 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.038+0000 7faadfe1c640 1 -- 172.21.15.8:0/4057268673 wait complete. 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 Processor -- start 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 -- start start 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad8100610 0x7faad81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 0x7faad8198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faad8199470 0x7faad81a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a40c0 con 0x7faad81056e0 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a4230 con 0x7faad8199470 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadfe1c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a43f0 con 0x7faad8100610 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 0x7faad8198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 0x7faad8198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44310/0 (socket says 172.21.15.8:44310) 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 -- 172.21.15.8:0/1823508547 learned_addr learned my addr 172.21.15.8:0/1823508547 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faaddbb8640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad8100610 0x7faad81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faade3b9640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faad8199470 0x7faad81a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 -- 172.21.15.8:0/1823508547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad8100610 msgr2=0x7faad81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad8100610 0x7faad81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 -- 172.21.15.8:0/1823508547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faad8199470 msgr2=0x7faad81a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faad8199470 0x7faad81a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.039+0000 7faadd3b7640 1 -- 172.21.15.8:0/1823508547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad4009560 con 0x7faad81056e0 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.040+0000 7faadd3b7640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 0x7faad8198ee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9c0 tx=0x7faac800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.040+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8009700 con 0x7faad81056e0 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.040+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faac800cd80 con 0x7faad81056e0 2024-02-19T16:32:59.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.040+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad81a46d0 con 0x7faad81056e0 2024-02-19T16:32:59.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.040+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8010430 con 0x7faad81056e0 2024-02-19T16:32:59.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.040+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81a4be0 con 0x7faad81056e0 2024-02-19T16:32:59.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.041+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad8101ae0 con 0x7faad81056e0 2024-02-19T16:32:59.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.042+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7faac8005000 con 0x7faad81056e0 2024-02-19T16:32:59.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.042+0000 7faaceffd640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7faac40a2230 0x7faac40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.043+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7faac8014070 con 0x7faad81056e0 2024-02-19T16:32:59.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.043+0000 7faaddbb8640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7faac40a2230 0x7faac40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.043+0000 7faaddbb8640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7faac40a2230 0x7faac40a46f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faad4005ab0 tx=0x7faad4005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.047+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7faac8097120 con 0x7faad81056e0 2024-02-19T16:32:59.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.227+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faad8108910 con 0x7faad81056e0 2024-02-19T16:32:59.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.227+0000 7faaceffd640 1 -- 172.21.15.8:0/1823508547 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7faac809a7e0 con 0x7faad81056e0 2024-02-19T16:32:59.229 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:32:59.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7faac40a2230 msgr2=0x7faac40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7faac40a2230 0x7faac40a46f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faad4005ab0 tx=0x7faad4005fd0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 msgr2=0x7faad8198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 0x7faad8198ee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9c0 tx=0x7faac800ee80 comp rx=0 tx=0).stop 2024-02-19T16:32:59.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 shutdown_connections 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faad8199470 0x7faad81a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faad81056e0 0x7faad8198ee0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7faac40a2230 0x7faac40a46f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 --2- 172.21.15.8:0/1823508547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad8100610 0x7faad81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 >> 172.21.15.8:0/1823508547 conn(0x7faad8076770 msgr2=0x7faad80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 shutdown_connections 2024-02-19T16:32:59.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.229+0000 7faadfe1c640 1 -- 172.21.15.8:0/1823508547 wait complete. 2024-02-19T16:32:59.242 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:32:59.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 -- 172.21.15.8:0/1177069550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070100f80 msgr2=0x7fe070104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 --2- 172.21.15.8:0/1177069550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070100f80 0x7fe070104fc0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe058009930 tx=0x7fe05802bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 -- 172.21.15.8:0/1177069550 shutdown_connections 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 --2- 172.21.15.8:0/1177069550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0701056a0 0x7fe07010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 --2- 172.21.15.8:0/1177069550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070100f80 0x7fe070104fc0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 --2- 172.21.15.8:0/1177069550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0701005d0 0x7fe0701009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 -- 172.21.15.8:0/1177069550 >> 172.21.15.8:0/1177069550 conn(0x7fe0700774f0 msgr2=0x7fe070077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 -- 172.21.15.8:0/1177069550 shutdown_connections 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 -- 172.21.15.8:0/1177069550 wait complete. 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 Processor -- start 2024-02-19T16:32:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.346+0000 7fe07581b640 1 -- start start 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0701005d0 0x7fe07019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0701056a0 0x7fe07019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 0x7fe070197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070198350 con 0x7fe070197960 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701984c0 con 0x7fe0701005d0 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070198680 con 0x7fe0701056a0 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 0x7fe070197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 0x7fe070197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44328/0 (socket says 172.21.15.8:44328) 2024-02-19T16:32:59.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 -- 172.21.15.8:0/3904356034 learned_addr learned my addr 172.21.15.8:0/3904356034 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06effd640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0701005d0 0x7fe07019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe067fff640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0701056a0 0x7fe07019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 -- 172.21.15.8:0/3904356034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0701056a0 msgr2=0x7fe07019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0701056a0 0x7fe07019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 -- 172.21.15.8:0/3904356034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0701005d0 msgr2=0x7fe07019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0701005d0 0x7fe07019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 -- 172.21.15.8:0/3904356034 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe058009560 con 0x7fe070197960 2024-02-19T16:32:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe06f7fe640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 0x7fe070197e10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe06800eb10 tx=0x7fe06800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068004b20 con 0x7fe070197960 2024-02-19T16:32:59.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe068004c80 con 0x7fe070197960 2024-02-19T16:32:59.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.347+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0701a4670 con 0x7fe070197960 2024-02-19T16:32:59.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.348+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068010650 con 0x7fe070197960 2024-02-19T16:32:59.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.348+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0701a4b00 con 0x7fe070197960 2024-02-19T16:32:59.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.349+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0700682e0 con 0x7fe070197960 2024-02-19T16:32:59.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.350+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fe068005020 con 0x7fe070197960 2024-02-19T16:32:59.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.350+0000 7fe074819640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe05c0a2230 0x7fe05c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.350+0000 7fe06effd640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe05c0a2230 0x7fe05c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.350+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fe068014070 con 0x7fe070197960 2024-02-19T16:32:59.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.351+0000 7fe06effd640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe05c0a2230 0x7fe05c0a46f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe070198aa0 tx=0x7fe060005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.355+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fe068097290 con 0x7fe070197960 2024-02-19T16:32:59.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.537+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe070100f80 con 0x7fe070197960 2024-02-19T16:32:59.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.538+0000 7fe074819640 1 -- 172.21.15.8:0/3904356034 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe06809a950 con 0x7fe070197960 2024-02-19T16:32:59.539 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:32:59.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe05c0a2230 msgr2=0x7fe05c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe05c0a2230 0x7fe05c0a46f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe070198aa0 tx=0x7fe060005f90 comp rx=0 tx=0).stop 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 msgr2=0x7fe070197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 0x7fe070197e10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe06800eb10 tx=0x7fe06800c2d0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 shutdown_connections 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe070197960 0x7fe070197e10 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fe05c0a2230 0x7fe05c0a46f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0701056a0 0x7fe07019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 --2- 172.21.15.8:0/3904356034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0701005d0 0x7fe07019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 >> 172.21.15.8:0/3904356034 conn(0x7fe0700774f0 msgr2=0x7fe0700fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 shutdown_connections 2024-02-19T16:32:59.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.540+0000 7fe07581b640 1 -- 172.21.15.8:0/3904356034 wait complete. 2024-02-19T16:32:59.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:32:59.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 -- 172.21.15.8:0/74575114 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc1064b0 msgr2=0x7f3dbc106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/74575114 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc1064b0 0x7f3dbc106890 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009900 tx=0x7f3dac02bc30 comp rx=0 tx=0).stop 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 -- 172.21.15.8:0/74575114 shutdown_connections 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/74575114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dbc101090 0x7f3dbc10d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/74575114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dbc1006d0 0x7f3dbc100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/74575114 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc1064b0 0x7f3dbc106890 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 -- 172.21.15.8:0/74575114 >> 172.21.15.8:0/74575114 conn(0x7f3dbc0773d0 msgr2=0x7f3dbc0777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 -- 172.21.15.8:0/74575114 shutdown_connections 2024-02-19T16:32:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 -- 172.21.15.8:0/74575114 wait complete. 2024-02-19T16:32:59.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 Processor -- start 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.656+0000 7f3dc11c9640 1 -- start start 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dbc1006d0 0x7f3dbc19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 0x7f3dbc19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dbc1064b0 0x7f3dbc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc197ef0 con 0x7f3dbc101090 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc198060 con 0x7f3dbc1006d0 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc198220 con 0x7f3dbc1064b0 2024-02-19T16:32:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 0x7f3dbc19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 0x7f3dbc19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44350/0 (socket says 172.21.15.8:44350) 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 -- 172.21.15.8:0/295934450 learned_addr learned my addr 172.21.15.8:0/295934450 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dbad9d640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dbc1006d0 0x7f3dbc19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dbb59e640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dbc1064b0 0x7f3dbc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 -- 172.21.15.8:0/295934450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dbc1064b0 msgr2=0x7f3dbc197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dbc1064b0 0x7f3dbc197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 -- 172.21.15.8:0/295934450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dbc1006d0 msgr2=0x7f3dbc19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dbc1006d0 0x7f3dbc19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 -- 172.21.15.8:0/295934450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac009560 con 0x7f3dbc101090 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db3fff640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 0x7f3dbc19dc20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e9f0 tx=0x7f3da400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4009700 con 0x7f3dbc101090 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc198500 con 0x7f3dbc101090 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da400cd80 con 0x7f3dbc101090 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.657+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4018920 con 0x7f3dbc101090 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.658+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dbc1a4710 con 0x7f3dbc101090 2024-02-19T16:32:59.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.658+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dbc101b10 con 0x7f3dbc101090 2024-02-19T16:32:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.660+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f3da4018a80 con 0x7f3dbc101090 2024-02-19T16:32:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.660+0000 7f3db8d99640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3da80a2230 0x7f3da80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.660+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f3da4014070 con 0x7f3dbc101090 2024-02-19T16:32:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.660+0000 7f3dbad9d640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3da80a2230 0x7f3da80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.661+0000 7f3dbad9d640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3da80a2230 0x7f3da80a46f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006010 tx=0x7f3dac005e40 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.664+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f3da4053230 con 0x7f3dbc101090 2024-02-19T16:32:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:59 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2614889773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:32:59.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:32:59 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1823508547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:32:59.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.845+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3dbc108910 con 0x7f3dbc101090 2024-02-19T16:32:59.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.846+0000 7f3db8d99640 1 -- 172.21.15.8:0/295934450 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3da4097170 con 0x7f3dbc101090 2024-02-19T16:32:59.847 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:32:59.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3da80a2230 msgr2=0x7f3da80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3da80a2230 0x7f3da80a46f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006010 tx=0x7f3dac005e40 comp rx=0 tx=0).stop 2024-02-19T16:32:59.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 msgr2=0x7f3dbc19dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 0x7f3dbc19dc20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e9f0 tx=0x7f3da400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 shutdown_connections 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f3da80a2230 0x7f3da80a46f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dbc1064b0 0x7f3dbc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3dbc101090 0x7f3dbc19dc20 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 --2- 172.21.15.8:0/295934450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dbc1006d0 0x7f3dbc19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.847+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 >> 172.21.15.8:0/295934450 conn(0x7f3dbc0773d0 msgr2=0x7f3dbc0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.848+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 shutdown_connections 2024-02-19T16:32:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.848+0000 7f3dc11c9640 1 -- 172.21.15.8:0/295934450 wait complete. 2024-02-19T16:32:59.859 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:59 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2614889773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:32:59.859 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:32:59 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1823508547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:32:59.860 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:32:59.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 -- 172.21.15.8:0/599030521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 msgr2=0x7fd6ec10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 --2- 172.21.15.8:0/599030521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 0x7fd6ec10b560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009860 tx=0x7fd6dc02bc20 comp rx=0 tx=0).stop 2024-02-19T16:32:59.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 -- 172.21.15.8:0/599030521 shutdown_connections 2024-02-19T16:32:59.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 --2- 172.21.15.8:0/599030521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 --2- 172.21.15.8:0/599030521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 0x7fd6ec10b560 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 --2- 172.21.15.8:0/599030521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6ec102760 0x7fd6ec102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 -- 172.21.15.8:0/599030521 >> 172.21.15.8:0/599030521 conn(0x7fd6ec077550 msgr2=0x7fd6ec077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 -- 172.21.15.8:0/599030521 shutdown_connections 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.963+0000 7fd6f362a640 1 -- 172.21.15.8:0/599030521 wait complete. 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 Processor -- start 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 -- start start 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6ec102760 0x7fd6ec19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 0x7fd6ec19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1980b0 con 0x7fd6ec10bb30 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec198220 con 0x7fd6ec102760 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f362a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1983e0 con 0x7fd6ec103080 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f1bc7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f1bc7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44370/0 (socket says 172.21.15.8:44370) 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f1bc7640 1 -- 172.21.15.8:0/577265936 learned_addr learned my addr 172.21.15.8:0/577265936 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:32:59.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f0bc5640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 0x7fd6ec19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.964+0000 7fd6f13c6640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6ec102760 0x7fd6ec19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f1bc7640 1 -- 172.21.15.8:0/577265936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 msgr2=0x7fd6ec19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f1bc7640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 0x7fd6ec19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f1bc7640 1 -- 172.21.15.8:0/577265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6ec102760 msgr2=0x7fd6ec19d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f1bc7640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6ec102760 0x7fd6ec19d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f1bc7640 1 -- 172.21.15.8:0/577265936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4009900 con 0x7fd6ec10bb30 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f1bc7640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec197a60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400efc0 tx=0x7fd6e400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4003970 con 0x7fd6ec10bb30 2024-02-19T16:32:59.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc009490 con 0x7fd6ec10bb30 2024-02-19T16:32:59.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4004020 con 0x7fd6ec10bb30 2024-02-19T16:32:59.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400b9b0 con 0x7fd6ec10bb30 2024-02-19T16:32:59.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.965+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec1a4770 con 0x7fd6ec10bb30 2024-02-19T16:32:59.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.966+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec068730 con 0x7fd6ec10bb30 2024-02-19T16:32:59.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.967+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fd6e400a040 con 0x7fd6ec10bb30 2024-02-19T16:32:59.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.968+0000 7fd6e27fc640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6d80a2230 0x7fd6d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:32:59.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.968+0000 7fd6f13c6640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6d80a2230 0x7fd6d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:32:59.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.968+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fd6e40cea50 con 0x7fd6ec10bb30 2024-02-19T16:32:59.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.968+0000 7fd6f13c6640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6d80a2230 0x7fd6d80a46f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec198ba0 tx=0x7fd6e8005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:32:59.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:32:59.971+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd6e4097280 con 0x7fd6ec10bb30 2024-02-19T16:33:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:59 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2614889773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:00.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:32:59 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1823508547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:33:00.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.154+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd6ec106c20 con 0x7fd6ec10bb30 2024-02-19T16:33:00.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.154+0000 7fd6e27fc640 1 -- 172.21.15.8:0/577265936 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v104) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd6e409a940 con 0x7fd6ec10bb30 2024-02-19T16:33:00.156 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:33:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6d80a2230 msgr2=0x7fd6d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6d80a2230 0x7fd6d80a46f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec198ba0 tx=0x7fd6e8005f90 comp rx=0 tx=0).stop 2024-02-19T16:33:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 msgr2=0x7fd6ec197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec197a60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400efc0 tx=0x7fd6e400c800 comp rx=0 tx=0).stop 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 shutdown_connections 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6ec10bb30 0x7fd6ec197a60 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd6d80a2230 0x7fd6d80a46f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ec103080 0x7fd6ec19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 --2- 172.21.15.8:0/577265936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6ec102760 0x7fd6ec19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 >> 172.21.15.8:0/577265936 conn(0x7fd6ec077550 msgr2=0x7fd6ec0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 shutdown_connections 2024-02-19T16:33:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.156+0000 7fd6f362a640 1 -- 172.21.15.8:0/577265936 wait complete. 2024-02-19T16:33:00.169 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 -- 172.21.15.8:0/2054916109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 msgr2=0x7f9fbc101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2054916109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 0x7f9fbc101950 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009890 tx=0x7f9fb002bb70 comp rx=0 tx=0).stop 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 -- 172.21.15.8:0/2054916109 shutdown_connections 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2054916109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc101e90 0x7f9fbc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2054916109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 0x7f9fbc101950 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2054916109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 -- 172.21.15.8:0/2054916109 >> 172.21.15.8:0/2054916109 conn(0x7f9fbc0fcfa0 msgr2=0x7f9fbc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 -- 172.21.15.8:0/2054916109 shutdown_connections 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.270+0000 7f9fc225b640 1 -- 172.21.15.8:0/2054916109 wait complete. 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 Processor -- start 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 -- start start 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 0x7f9fbc19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc101e90 0x7f9fbc19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc197f00 con 0x7f9fbc107590 2024-02-19T16:33:00.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc198070 con 0x7f9fbc101e90 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fc225b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc198230 con 0x7f9fbc1014d0 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44384/0 (socket says 172.21.15.8:44384) 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 -- 172.21.15.8:0/2811250628 learned_addr learned my addr 172.21.15.8:0/2811250628 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbb7fe640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 0x7f9fbc19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbaffd640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc101e90 0x7f9fbc19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 -- 172.21.15.8:0/2811250628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 msgr2=0x7f9fbc19d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:00.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 0x7f9fbc19d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 -- 172.21.15.8:0/2811250628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc101e90 msgr2=0x7f9fbc19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc101e90 0x7f9fbc19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.271+0000 7f9fbbfff640 1 -- 172.21.15.8:0/2811250628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac009560 con 0x7f9fbc107590 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.272+0000 7f9fbbfff640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc1978b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400eb10 tx=0x7f9fb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.272+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4004b20 con 0x7f9fbc107590 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.272+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fb4004c80 con 0x7f9fbc107590 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.272+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb0009490 con 0x7f9fbc107590 2024-02-19T16:33:00.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.272+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4010650 con 0x7f9fbc107590 2024-02-19T16:33:00.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.272+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc1a45c0 con 0x7f9fbc107590 2024-02-19T16:33:00.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.273+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc0682e0 con 0x7f9fbc107590 2024-02-19T16:33:00.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.274+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f9fb4005020 con 0x7f9fbc107590 2024-02-19T16:33:00.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.274+0000 7f9fb8ff9640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9fa40a2230 0x7f9fa40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:00.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.274+0000 7f9fbb7fe640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9fa40a2230 0x7f9fa40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:00.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.275+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f9fb4014070 con 0x7f9fbc107590 2024-02-19T16:33:00.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.275+0000 7f9fbb7fe640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9fa40a2230 0x7f9fa40a46f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc1989f0 tx=0x7f9fac009260 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:00.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.279+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f9fb4097310 con 0x7f9fbc107590 2024-02-19T16:33:00.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.511+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f9fbc1098e0 con 0x7f9fbc107590 2024-02-19T16:33:00.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.520+0000 7f9fb8ff9640 1 -- 172.21.15.8:0/2811250628 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+736933 (secure 0 0 0) 0x7f9fb409a9d0 con 0x7f9fbc107590 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout: "epoch": 26, 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout: "active_gid": 15633, 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout: "active_name": "x", 2024-02-19T16:33:00.532 INFO:teuthology.orchestra.run.smithi008.stdout: "active_addrs": { 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:6800", 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 4279763273 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2024-02-19T16:33:00.533 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:6801", 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 4279763273 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: "active_addr": "172.21.15.8:6801/4279763273", 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: "active_change": "2024-02-19T16:22:52.539953+0000", 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: "active_mgr_features": 4540701547738038271, 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: "available": true, 2024-02-19T16:33:00.534 INFO:teuthology.orchestra.run.smithi008.stdout: "standbys": [ 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "gid": 25406, 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "y", 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr_features": 4540701547738038271, 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "available_modules": [ 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alerts", 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.535 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.536 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How frequently to reexamine health status", 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.537 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.538 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.539 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.540 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.541 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.542 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.543 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.544 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.545 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_destination": { 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_destination", 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.546 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email address to send alerts to", 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.547 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_from_name": { 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_from_name", 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "Ceph", 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.548 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email From: name", 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_host": { 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_host", 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.549 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP server", 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.550 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_password": { 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_password", 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.551 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Password to authenticate with", 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_port": { 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_port", 2024-02-19T16:33:00.552 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "465", 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP port", 2024-02-19T16:33:00.553 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_sender": { 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_sender", 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.554 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP envelope sender", 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.555 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_ssl": { 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_ssl", 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.556 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.557 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_user": { 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_user", 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.558 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "User to authenticate as", 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "balancer", 2024-02-19T16:33:00.559 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "active": { 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "active", 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.560 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically balance PGs across cluster", 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.561 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_time": { 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_time", 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0000", 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.562 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "beginning time of day to automatically balance", 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_weekday": { 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_weekday", 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:00.563 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.564 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_max_iterations": { 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_max_iterations", 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "25", 2024-02-19T16:33:00.565 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "250", 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_metrics": { 2024-02-19T16:33:00.566 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_metrics", 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "pgs,objects,bytes", 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_step": { 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_step", 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.5", 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0.001", 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "0.999", 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "aggressiveness of optimization", 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.569 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "end_time": { 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_time", 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2359", 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ending time of day to automatically balance", 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "end_weekday": { 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_weekday", 2024-02-19T16:33:00.571 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-19T16:33:00.572 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.573 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.574 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.575 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.576 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.577 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.578 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.579 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.580 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "min_score": { 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_score", 2024-02-19T16:33:00.581 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-19T16:33:00.582 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.583 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "upmap", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "crush-compat", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "none", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "read", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap", 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap-read" 2024-02-19T16:33:00.584 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Balancer mode", 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_ids": { 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_ids", 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.585 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.586 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.587 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.588 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_deviation": { 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_deviation", 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.589 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_optimizations": { 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_optimizations", 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.590 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.591 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm", 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_down_multiplier": { 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_down_multiplier", 2024-02-19T16:33:00.592 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.593 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_refresh_rate": { 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_refresh_rate", 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "20", 2024-02-19T16:33:00.594 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_starting_port": { 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_starting_port", 2024-02-19T16:33:00.595 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4721", 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-19T16:33:00.596 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_ptrace": { 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_ptrace", 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.597 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.598 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_interval": { 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_interval", 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:00.599 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.600 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_memory_target_ratio": { 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_memory_target_ratio", 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.7", 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.601 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm_log_destination": { 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm_log_destination", 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.602 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "file", 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "file,syslog", 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "syslog" 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.603 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "cgroups_split": { 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cgroups_split", 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.604 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.605 INFO:teuthology.orchestra.run.smithi008.stdout: "config_checks_enabled": { 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_checks_enabled", 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.606 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "config_dashboard": { 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_dashboard", 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.607 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.608 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_alertmanager": { 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_alertmanager", 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.609 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_base": { 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_base", 2024-02-19T16:33:00.610 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Container image name, without the tag", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.611 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_elasticsearch": { 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_elasticsearch", 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.612 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "elasticsearch container image", 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.613 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_grafana": { 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_grafana", 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.614 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_haproxy": { 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_haproxy", 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.615 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "HAproxy container image", 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.616 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_agent": { 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_agent", 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-19T16:33:00.617 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger agent container image", 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.618 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_collector": { 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_collector", 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.619 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger collector container image", 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_query": { 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_query", 2024-02-19T16:33:00.620 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger query container image", 2024-02-19T16:33:00.621 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_keepalived": { 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_keepalived", 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.622 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Keepalived container image", 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.623 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_loki": { 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_loki", 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-19T16:33:00.624 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Loki container image", 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.625 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_node_exporter": { 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_node_exporter", 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.626 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_nvmeof": { 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_nvmeof", 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.627 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Nvme-of container image", 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.628 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_prometheus": { 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_prometheus", 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.629 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.630 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_promtail": { 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_promtail", 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.631 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Promtail container image", 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_snmp_gateway": { 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_snmp_gateway", 2024-02-19T16:33:00.632 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SNMP Gateway container image", 2024-02-19T16:33:00.633 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "container_init": { 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_init", 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.634 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Run podman/docker with `--init`", 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.635 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "daemon_cache_timeout": { 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "daemon_cache_timeout", 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.636 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "default_cephadm_command_timeout": { 2024-02-19T16:33:00.637 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_cephadm_command_timeout", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "900", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-19T16:33:00.638 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "default_registry": { 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_registry", 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.639 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io", 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.640 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "device_cache_timeout": { 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_cache_timeout", 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1800", 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.641 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache device inventory", 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "device_enhanced_scan": { 2024-02-19T16:33:00.642 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_enhanced_scan", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-19T16:33:00.643 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "facts_cache_timeout": { 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "facts_cache_timeout", 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.644 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache host facts data", 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.645 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "host_check_interval": { 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "host_check_interval", 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.646 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to perform a host check", 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "hw_monitoring": { 2024-02-19T16:33:00.647 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hw_monitoring", 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.648 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "inventory_list_all": { 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "inventory_list_all", 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.649 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.650 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.651 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.652 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.653 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "log_refresh_metadata": { 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_refresh_metadata", 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.654 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.655 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.656 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.657 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.658 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.659 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.660 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.661 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-19T16:33:00.662 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "*", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.663 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "max_count_per_host": { 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_count_per_host", 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.664 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of daemons per service per host", 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "max_osd_draining_count": { 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_osd_draining_count", 2024-02-19T16:33:00.665 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.666 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "migration_current": { 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "migration_current", 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.667 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "internal - do not modify", 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2024-02-19T16:33:00.668 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "root", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm-package", 2024-02-19T16:33:00.669 INFO:teuthology.orchestra.run.smithi008.stdout: "root" 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "mode for remote execution of cephadm", 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "oob_default_addr": { 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "oob_default_addr", 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.670 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "169.254.1.1", 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.671 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_alerts_path": { 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_alerts_path", 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.672 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_insecure": { 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_insecure", 2024-02-19T16:33:00.673 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-19T16:33:00.674 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_password": { 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_password", 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.675 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.676 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_url": { 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_url", 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.677 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_username": { 2024-02-19T16:33:00.678 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_username", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-19T16:33:00.679 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "secure_monitoring_stack": { 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "secure_monitoring_stack", 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.680 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.681 INFO:teuthology.orchestra.run.smithi008.stdout: "service_discovery_port": { 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "service_discovery_port", 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8765", 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.682 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "cephadm service discovery port", 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "ssh_config_file": { 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssh_config_file", 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.683 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.684 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "use_agent": { 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_agent", 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.685 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "use_repo_digest": { 2024-02-19T16:33:00.686 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_repo_digest", 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.687 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_failed_host_check": { 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_failed_host_check", 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.688 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if the host check fails", 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.689 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_daemons": { 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_daemons", 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.690 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_hosts": { 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_hosts", 2024-02-19T16:33:00.691 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.692 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crash", 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.693 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.694 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.695 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.696 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.697 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.698 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.699 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.700 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.701 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "retain_interval": { 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retain_interval", 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "31536000", 2024-02-19T16:33:00.702 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_recent_interval": { 2024-02-19T16:33:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_recent_interval", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1209600", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dashboard", 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.705 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.706 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_HOST": { 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-19T16:33:00.707 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.708 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.709 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.710 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_ENABLED": { 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_ENABLED", 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.711 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-19T16:33:00.712 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.713 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "ENABLE_BROWSABLE_API": { 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.714 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.715 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.716 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-19T16:33:00.717 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.718 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.719 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.720 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.721 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.722 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.723 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-19T16:33:00.724 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.725 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.726 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.727 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.728 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_PASSWORD": { 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-19T16:33:00.729 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.730 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.731 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.732 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_URL": { 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_URL", 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.733 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_USERNAME": { 2024-02-19T16:33:00.734 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.735 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.736 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.737 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.738 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.739 INFO:teuthology.orchestra.run.smithi008.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.740 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.741 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.742 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_HOST": { 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.743 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.744 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.745 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.746 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-19T16:33:00.747 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.748 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.749 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.750 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.751 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.752 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.753 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-19T16:33:00.754 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.755 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.756 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.757 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_ENABLED": { 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.758 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-19T16:33:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.760 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.761 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-19T16:33:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.763 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.764 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "45", 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ACCESS_KEY": { 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.766 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-19T16:33:00.767 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.768 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SECRET_KEY": { 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.769 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.770 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SSL_VERIFY": { 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.771 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.772 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.773 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.774 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-19T16:33:00.775 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.776 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "cross_origin_url": { 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cross_origin_url", 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.777 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.778 INFO:teuthology.orchestra.run.smithi008.stdout: "crt_file": { 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crt_file", 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.779 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "debug": { 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "debug", 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.780 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable/disable debug options", 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.781 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "jwt_token_ttl": { 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "jwt_token_ttl", 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "28800", 2024-02-19T16:33:00.782 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2024-02-19T16:33:00.783 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.784 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.785 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.786 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.787 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.788 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.789 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.790 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.791 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "motd": { 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "motd", 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "The message of the day", 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect_resolve_ip_addr": { 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "redirect_resolve_ip_addr", 2024-02-19T16:33:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.795 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2024-02-19T16:33:00.796 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:00.797 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8080", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.798 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.799 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl_server_port": { 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl_server_port", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8443", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.801 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.802 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "redirect", 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect" 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.803 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2024-02-19T16:33:00.804 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "url_prefix": { 2024-02-19T16:33:00.805 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url_prefix", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.806 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.807 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_monitoring": { 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_monitoring", 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.808 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.811 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.811 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.811 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "monitor device health metrics", 2024-02-19T16:33:00.811 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.811 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.811 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.812 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.813 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.814 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.815 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.816 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.817 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.818 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.819 INFO:teuthology.orchestra.run.smithi008.stdout: "mark_out_threshold": { 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mark_out_threshold", 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2419200", 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.820 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_name": { 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_name", 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.821 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "device_health_metrics", 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.822 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "retention_period": { 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retention_period", 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15552000", 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.823 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain device health metrics", 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_frequency": { 2024-02-19T16:33:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_frequency", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-19T16:33:00.825 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "self_heal": { 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "self_heal", 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.827 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:00.828 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and check device health", 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.829 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_threshold": { 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_threshold", 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7257600", 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.830 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.831 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "diskprediction_local", 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.832 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.833 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.834 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.835 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.836 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.837 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.838 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.839 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "predict_interval": { 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predict_interval", 2024-02-19T16:33:00.840 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.841 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: "predictor_model": { 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predictor_model", 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.842 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "prophetstor", 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.843 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.844 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.845 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "influx", 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": false, 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "influxdb python module not found", 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "batch_size": { 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "batch_size", 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.846 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5000", 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.847 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "database": { 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "database", 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "ceph", 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.848 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": { 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hostname", 2024-02-19T16:33:00.849 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server hostname", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.850 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "30", 2024-02-19T16:33:00.851 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "5", 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.852 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.853 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.854 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.855 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.856 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.857 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.858 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.859 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "password": { 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "password", 2024-02-19T16:33:00.860 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.861 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.861 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.861 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.861 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.861 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "password of InfluxDB server user", 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "port": { 2024-02-19T16:33:00.862 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "port", 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8086", 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.863 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server port", 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.864 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "false", 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.865 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "threads": { 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threads", 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:00.866 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "32", 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "username": { 2024-02-19T16:33:00.867 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "username", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "username of InfluxDB server user", 2024-02-19T16:33:00.868 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "verify_ssl": { 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "verify_ssl", 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.869 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "true", 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.870 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "insights", 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.871 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.872 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.873 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.874 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.875 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.876 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.877 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.878 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.879 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "iostat", 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.880 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.881 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.882 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.883 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.884 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.885 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.886 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.887 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.888 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "k8sevents", 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.889 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_event_retention_days": { 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ceph_event_retention_days", 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7", 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.890 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "config_check_secs": { 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_check_secs", 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.891 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "10", 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.892 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.893 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.894 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.895 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.896 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.897 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.898 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.899 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.900 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "localpool", 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.901 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "failure_domain": { 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "failure_domain", 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "host", 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.902 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "failure domain for any created local pool", 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.903 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.904 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.905 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.906 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.907 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.908 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.909 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.910 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.911 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "min_size": { 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_size", 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.912 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default min_size for any created local pool", 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "num_rep": { 2024-02-19T16:33:00.913 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "num_rep", 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3", 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.914 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default replica count for any created local pool", 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_num": { 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_num", 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "128", 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default pg_num for any created local pool", 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "prefix": { 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prefix", 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name prefix for any created local pool", 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "subtree": { 2024-02-19T16:33:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "subtree", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "rack", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-19T16:33:00.919 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mds_autoscaler", 2024-02-19T16:33:00.920 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.921 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.922 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.923 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.924 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.925 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirroring", 2024-02-19T16:33:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "nfs", 2024-02-19T16:33:00.938 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.939 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.940 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.941 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.942 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.943 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.944 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.945 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.946 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2024-02-19T16:33:00.947 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "fail_fs": { 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "fail_fs", 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.949 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.951 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.952 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.953 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.954 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.955 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.956 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.957 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator": { 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.958 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "rook", 2024-02-19T16:33:00.959 INFO:teuthology.orchestra.run.smithi008.stdout: "test_orchestrator" 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Orchestrator backend", 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.960 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_perf_query", 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.961 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.962 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.963 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.964 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.965 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.966 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.967 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.968 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.969 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_support", 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.970 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.971 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.972 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.973 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.974 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.975 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.976 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.977 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.978 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.979 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_autoscaler", 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.980 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.981 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.982 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.983 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.984 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.985 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.986 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:00.987 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.988 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.989 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.990 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "threshold": { 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threshold", 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1.0", 2024-02-19T16:33:00.991 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "scaling threshold", 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:00.992 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "progress", 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_pg_recovery_event": { 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_pg_recovery_event", 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.993 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.994 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:00.995 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.996 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:00.997 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:00.998 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:00.999 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.000 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.001 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.002 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.003 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.004 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "max_completed_events": { 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_completed_events", 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "50", 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.005 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "number of past completed events to remember", 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.006 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long the module is going to sleep", 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.007 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus", 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.008 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "cache": { 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cache", 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.009 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "exclude_perf_counters": { 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "exclude_perf_counters", 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.010 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.011 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.012 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.015 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.015 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.015 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.015 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.016 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.017 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.018 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.019 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.020 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.022 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools": { 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools", 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.023 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.024 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300", 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.025 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_interval": { 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_interval", 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.026 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15.0", 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.027 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2024-02-19T16:33:01.028 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:01.029 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "9283", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-19T16:33:01.030 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "stale_cache_strategy": { 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stale_cache_strategy", 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.031 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "log", 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.032 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "default", 2024-02-19T16:33:01.033 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "default", 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "error" 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.034 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.035 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.036 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.037 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.038 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.039 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.040 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.041 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.042 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.043 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.044 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.045 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_snap_create": { 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_snap_create", 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.046 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "mirror_snapshot_schedule": { 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirror_snapshot_schedule", 2024-02-19T16:33:01.047 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.048 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "trash_purge_schedule": { 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "trash_purge_schedule", 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.049 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.050 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "restful", 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_auth": { 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_auth", 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.051 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.052 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.053 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.054 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.055 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.056 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.057 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.058 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.058 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.058 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.058 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.058 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.058 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.059 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.060 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.061 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.062 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.063 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.064 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.065 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rgw", 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.066 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.067 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.068 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.069 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.070 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.071 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.072 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.073 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.074 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rook", 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "drive_group_interval": { 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "drive_group_interval", 2024-02-19T16:33:01.075 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300.0", 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-19T16:33:01.076 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.077 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.078 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.079 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.080 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.081 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.082 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.083 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.084 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.084 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.084 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.084 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.084 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.085 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "storage_class": { 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "storage_class", 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "local", 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.086 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.087 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "selftest", 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.088 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.089 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.090 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.091 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.092 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.093 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.094 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.095 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "roption1": { 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption1", 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.096 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.097 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "roption2": { 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption2", 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "xyz", 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.098 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption1": { 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption1", 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.099 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.100 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption2": { 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption2", 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.101 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption3": { 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption3", 2024-02-19T16:33:01.102 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.103 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption4": { 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption4", 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.104 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.105 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption5": { 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption5", 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.106 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption6": { 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption6", 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.107 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.108 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption7": { 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption7", 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.109 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "42", 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "testkey": { 2024-02-19T16:33:01.110 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testkey", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.111 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "testlkey": { 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testlkey", 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.112 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.113 INFO:teuthology.orchestra.run.smithi008.stdout: "testnewline": { 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testnewline", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.114 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2024-02-19T16:33:01.115 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_m_granularity": { 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_m_granularity", 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.116 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow minute scheduled snapshots", 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.117 INFO:teuthology.orchestra.run.smithi008.stdout: "dump_on_update": { 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dump_on_update", 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.118 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "dump database to debug log on update", 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.119 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.120 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.121 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.122 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.123 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.124 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.124 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.124 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.124 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.124 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.124 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.125 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.126 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.127 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.128 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stats", 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.129 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.130 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.131 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.132 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.134 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.136 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.137 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "status", 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.138 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.139 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.140 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.141 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.142 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.143 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.144 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.145 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.146 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telegraf", 2024-02-19T16:33:01.147 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "address": { 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "address", 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.148 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.149 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15", 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.150 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.151 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.152 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.153 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.154 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.155 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.156 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.157 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.158 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.159 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telemetry", 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.160 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_basic": { 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_basic", 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.161 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_crash": { 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_crash", 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.162 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.163 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_device": { 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_device", 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.164 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.165 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_ident": { 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_ident", 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.166 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_perf": { 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_perf", 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.167 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.168 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "contact": { 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "contact", 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.169 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "description": { 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "description", 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.170 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.171 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "device_url": { 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_url", 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.174 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.175 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "24", 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "8", 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.176 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: "last_opt_revision": { 2024-02-19T16:33:01.177 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "last_opt_revision", 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1", 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.178 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard": { 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard", 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.179 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.180 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard_description": { 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard_description", 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.181 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.182 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.183 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.184 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.185 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.186 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.187 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.188 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.189 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.190 INFO:teuthology.orchestra.run.smithi008.stdout: "organization": { 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "organization", 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.191 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.192 INFO:teuthology.orchestra.run.smithi008.stdout: "proxy": { 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "proxy", 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.193 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "url": { 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url", 2024-02-19T16:33:01.194 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.195 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.196 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "test_orchestrator", 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.197 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.198 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.199 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.200 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.201 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.202 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.203 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.204 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.205 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.206 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.207 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.208 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.209 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.210 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.211 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.212 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.213 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.214 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.214 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.214 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.214 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.214 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.214 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.215 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_clones": { 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_clones", 2024-02-19T16:33:01.216 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4", 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.217 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: "periodic_async_work": { 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "periodic_async_work", 2024-02-19T16:33:01.218 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.219 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Periodically check for async work", 2024-02-19T16:33:01.220 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.220 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.220 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.220 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.220 INFO:teuthology.orchestra.run.smithi008.stdout: "snapshot_clone_delay": { 2024-02-19T16:33:01.221 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snapshot_clone_delay", 2024-02-19T16:33:01.221 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.221 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.221 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.221 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.221 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.222 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.222 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.222 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-19T16:33:01.222 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.222 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.223 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.223 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.223 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.223 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.223 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.224 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix", 2024-02-19T16:33:01.224 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.224 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.224 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.225 INFO:teuthology.orchestra.run.smithi008.stdout: "discovery_interval": { 2024-02-19T16:33:01.225 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "discovery_interval", 2024-02-19T16:33:01.225 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.225 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.225 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.225 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "100", 2024-02-19T16:33:01.226 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.226 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.226 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.226 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.226 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.227 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.227 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.227 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.227 INFO:teuthology.orchestra.run.smithi008.stdout: "identifier": { 2024-02-19T16:33:01.227 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "identifier", 2024-02-19T16:33:01.228 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.228 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.228 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.228 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.228 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.229 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.229 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.229 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.229 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.229 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.229 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.230 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.230 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.230 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.230 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.231 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.231 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.231 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.231 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.231 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.232 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.232 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.232 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.232 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.232 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.232 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.233 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.234 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.234 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.234 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.234 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.234 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.235 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.235 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.235 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.235 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.235 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.235 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.236 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.237 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.238 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.239 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.240 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.241 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_host": { 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_host", 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.242 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_port": { 2024-02-19T16:33:01.243 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_port", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10051", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.244 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_sender": { 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_sender", 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.245 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.246 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: "gid": 25415, 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "z", 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr_features": 4540701547738038271, 2024-02-19T16:33:01.247 INFO:teuthology.orchestra.run.smithi008.stdout: "available_modules": [ 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alerts", 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.248 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How frequently to reexamine health status", 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.249 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.250 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.251 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.253 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.254 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.255 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.256 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.257 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_destination": { 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_destination", 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.258 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email address to send alerts to", 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_from_name": { 2024-02-19T16:33:01.259 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_from_name", 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "Ceph", 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.260 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email From: name", 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_host": { 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_host", 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.261 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP server", 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.262 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_password": { 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_password", 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.263 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Password to authenticate with", 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_port": { 2024-02-19T16:33:01.264 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_port", 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "465", 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.265 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP port", 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_sender": { 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_sender", 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.266 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP envelope sender", 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.267 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_ssl": { 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_ssl", 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.268 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_user": { 2024-02-19T16:33:01.269 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_user", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "User to authenticate as", 2024-02-19T16:33:01.270 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "balancer", 2024-02-19T16:33:01.271 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "active": { 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "active", 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.272 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically balance PGs across cluster", 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.273 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_time": { 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_time", 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0000", 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.274 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "beginning time of day to automatically balance", 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_weekday": { 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_weekday", 2024-02-19T16:33:01.275 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.276 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_max_iterations": { 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_max_iterations", 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.277 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "25", 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "250", 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.278 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_metrics": { 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_metrics", 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "pgs,objects,bytes", 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.279 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_step": { 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_step", 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.280 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.5", 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0.001", 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "0.999", 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "aggressiveness of optimization", 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.281 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "end_time": { 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_time", 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2359", 2024-02-19T16:33:01.282 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ending time of day to automatically balance", 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "end_weekday": { 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_weekday", 2024-02-19T16:33:01.283 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-19T16:33:01.284 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.285 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.286 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.287 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.288 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.289 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.290 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.291 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.292 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "min_score": { 2024-02-19T16:33:01.293 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_score", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-19T16:33:01.294 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.295 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "upmap", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "crush-compat", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "none", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "read", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap", 2024-02-19T16:33:01.296 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap-read" 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Balancer mode", 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_ids": { 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_ids", 2024-02-19T16:33:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.298 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.299 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.300 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_deviation": { 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_deviation", 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.301 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_optimizations": { 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_optimizations", 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.302 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.303 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm", 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.304 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_down_multiplier": { 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_down_multiplier", 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.305 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_refresh_rate": { 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_refresh_rate", 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.306 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "20", 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.307 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_starting_port": { 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_starting_port", 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4721", 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.308 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_ptrace": { 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_ptrace", 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.309 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.310 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_interval": { 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_interval", 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.311 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_memory_target_ratio": { 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_memory_target_ratio", 2024-02-19T16:33:01.312 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.7", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.313 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm_log_destination": { 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm_log_destination", 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.314 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "file", 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "file,syslog", 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "syslog" 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.315 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "cgroups_split": { 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cgroups_split", 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.316 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "config_checks_enabled": { 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_checks_enabled", 2024-02-19T16:33:01.317 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-19T16:33:01.318 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "config_dashboard": { 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_dashboard", 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.319 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.320 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_alertmanager": { 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_alertmanager", 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.321 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_base": { 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_base", 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.322 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Container image name, without the tag", 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.323 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_elasticsearch": { 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_elasticsearch", 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.324 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "elasticsearch container image", 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_grafana": { 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_grafana", 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.325 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.326 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_haproxy": { 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_haproxy", 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.327 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "HAproxy container image", 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_agent": { 2024-02-19T16:33:01.328 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_agent", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger agent container image", 2024-02-19T16:33:01.329 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_collector": { 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_collector", 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.330 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger collector container image", 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.331 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_query": { 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_query", 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.332 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger query container image", 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_keepalived": { 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_keepalived", 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.333 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Keepalived container image", 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.334 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_loki": { 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_loki", 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.335 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Loki container image", 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.336 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_node_exporter": { 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_node_exporter", 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.337 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_nvmeof": { 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_nvmeof", 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.338 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Nvme-of container image", 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.339 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_prometheus": { 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_prometheus", 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-19T16:33:01.340 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.341 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_promtail": { 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_promtail", 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.342 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Promtail container image", 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_snmp_gateway": { 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_snmp_gateway", 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.343 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SNMP Gateway container image", 2024-02-19T16:33:01.344 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "container_init": { 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_init", 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.345 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Run podman/docker with `--init`", 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.346 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "daemon_cache_timeout": { 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "daemon_cache_timeout", 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:01.347 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.348 INFO:teuthology.orchestra.run.smithi008.stdout: "default_cephadm_command_timeout": { 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_cephadm_command_timeout", 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "900", 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.349 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "default_registry": { 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_registry", 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.350 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io", 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.351 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: "device_cache_timeout": { 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_cache_timeout", 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.352 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1800", 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache device inventory", 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.353 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "device_enhanced_scan": { 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_enhanced_scan", 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.354 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "facts_cache_timeout": { 2024-02-19T16:33:01.355 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "facts_cache_timeout", 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.356 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache host facts data", 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "host_check_interval": { 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "host_check_interval", 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.357 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to perform a host check", 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.358 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "hw_monitoring": { 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hw_monitoring", 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.359 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.360 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "inventory_list_all": { 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "inventory_list_all", 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.361 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.362 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.363 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.364 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "log_refresh_metadata": { 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_refresh_metadata", 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.365 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.366 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.367 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.368 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.369 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.370 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.371 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.372 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "*", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-19T16:33:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "max_count_per_host": { 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_count_per_host", 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of daemons per service per host", 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "max_osd_draining_count": { 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_osd_draining_count", 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.377 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "migration_current": { 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "migration_current", 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.378 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "internal - do not modify", 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.379 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "root", 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.380 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm-package", 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "root" 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "mode for remote execution of cephadm", 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.381 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "oob_default_addr": { 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "oob_default_addr", 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "169.254.1.1", 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.382 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.383 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_alerts_path": { 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_alerts_path", 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.384 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.385 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-19T16:33:01.385 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.385 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.385 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.385 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.385 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_insecure": { 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_insecure", 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.386 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_password": { 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_password", 2024-02-19T16:33:01.387 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.388 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_url": { 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_url", 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.389 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_username": { 2024-02-19T16:33:01.390 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_username", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-19T16:33:01.391 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "secure_monitoring_stack": { 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "secure_monitoring_stack", 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.392 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.393 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "service_discovery_port": { 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "service_discovery_port", 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8765", 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.394 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "cephadm service discovery port", 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "ssh_config_file": { 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssh_config_file", 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.395 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.396 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "use_agent": { 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_agent", 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.397 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "use_repo_digest": { 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_repo_digest", 2024-02-19T16:33:01.398 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.399 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_failed_host_check": { 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_failed_host_check", 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.400 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if the host check fails", 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_daemons": { 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_daemons", 2024-02-19T16:33:01.401 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.402 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_hosts": { 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_hosts", 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.403 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.404 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crash", 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.405 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.406 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.407 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.408 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.409 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.410 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.411 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.412 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "retain_interval": { 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retain_interval", 2024-02-19T16:33:01.413 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "31536000", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.414 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_recent_interval": { 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_recent_interval", 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1209600", 2024-02-19T16:33:01.415 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.416 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dashboard", 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.417 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.418 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_HOST": { 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.419 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.420 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.421 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_ENABLED": { 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_ENABLED", 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.422 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.423 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.424 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "ENABLE_BROWSABLE_API": { 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.425 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-19T16:33:01.426 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.427 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.428 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.429 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.430 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.431 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.432 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.433 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-19T16:33:01.434 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.435 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.436 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.437 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.438 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_PASSWORD": { 2024-02-19T16:33:01.439 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.440 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.441 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.442 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_URL": { 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_URL", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.443 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_USERNAME": { 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.444 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.445 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.446 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.447 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.448 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.449 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.450 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.451 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_HOST": { 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.452 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-19T16:33:01.453 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.454 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.455 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.456 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.457 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.458 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.459 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-19T16:33:01.460 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.461 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.462 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.463 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.464 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-19T16:33:01.465 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.466 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_ENABLED": { 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.467 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.468 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.469 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.470 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.471 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8", 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.472 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.473 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "45", 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.474 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ACCESS_KEY": { 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.475 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.476 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.477 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SECRET_KEY": { 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.478 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SSL_VERIFY": { 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-19T16:33:01.479 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.480 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.481 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.482 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.483 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.484 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.485 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "cross_origin_url": { 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cross_origin_url", 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.486 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "crt_file": { 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crt_file", 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.487 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.488 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "debug": { 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "debug", 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.489 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable/disable debug options", 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "jwt_token_ttl": { 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "jwt_token_ttl", 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.490 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "28800", 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.491 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.492 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.493 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.494 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.495 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.496 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.497 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.498 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.499 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.500 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "motd": { 2024-02-19T16:33:01.501 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "motd", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "The message of the day", 2024-02-19T16:33:01.502 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect_resolve_ip_addr": { 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "redirect_resolve_ip_addr", 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.503 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:01.504 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.505 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.506 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8080", 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.507 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.508 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl_server_port": { 2024-02-19T16:33:01.509 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl_server_port", 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8443", 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.510 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.511 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "redirect", 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect" 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.512 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.513 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.514 INFO:teuthology.orchestra.run.smithi008.stdout: "url_prefix": { 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url_prefix", 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.515 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2024-02-19T16:33:01.516 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_monitoring": { 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_monitoring", 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.517 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "monitor device health metrics", 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.518 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.519 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.520 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.521 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.522 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.523 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.524 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.525 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "mark_out_threshold": { 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mark_out_threshold", 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.526 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2419200", 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.527 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_name": { 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_name", 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "device_health_metrics", 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.528 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "retention_period": { 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retention_period", 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.529 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15552000", 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain device health metrics", 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.530 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_frequency": { 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_frequency", 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.531 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "self_heal": { 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "self_heal", 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.532 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.533 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.534 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and check device health", 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_threshold": { 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_threshold", 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.535 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7257600", 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.536 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "diskprediction_local", 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.537 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.538 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.539 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.540 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.541 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.542 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.543 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.544 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.545 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "predict_interval": { 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predict_interval", 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.546 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.547 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "predictor_model": { 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predictor_model", 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "prophetstor", 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.548 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.549 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.550 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "influx", 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": false, 2024-02-19T16:33:01.551 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "influxdb python module not found", 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "batch_size": { 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "batch_size", 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5000", 2024-02-19T16:33:01.552 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "database": { 2024-02-19T16:33:01.553 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "database", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "ceph", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-19T16:33:01.554 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": { 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hostname", 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.555 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server hostname", 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.556 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "30", 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "5", 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.557 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.558 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.559 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.560 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.561 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.562 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.563 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.564 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.565 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "password": { 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "password", 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.566 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "password of InfluxDB server user", 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.567 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "port": { 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "port", 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8086", 2024-02-19T16:33:01.568 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server port", 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.569 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "false", 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.570 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "threads": { 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threads", 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.571 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "32", 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.572 INFO:teuthology.orchestra.run.smithi008.stdout: "username": { 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "username", 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.573 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "username of InfluxDB server user", 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "verify_ssl": { 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "verify_ssl", 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.574 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "true", 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.575 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "insights", 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.576 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.577 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.578 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.579 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.580 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.581 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.582 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.583 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "iostat", 2024-02-19T16:33:01.584 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.585 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.586 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.587 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.588 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.589 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.590 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.591 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.592 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "k8sevents", 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.593 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_event_retention_days": { 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ceph_event_retention_days", 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7", 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.594 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "config_check_secs": { 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_check_secs", 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.595 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "10", 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.596 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.597 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.598 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.599 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.600 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.601 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.602 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.603 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.604 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "localpool", 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: "failure_domain": { 2024-02-19T16:33:01.605 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "failure_domain", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "host", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "failure domain for any created local pool", 2024-02-19T16:33:01.606 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.607 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.608 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.609 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.610 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.611 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.612 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.613 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.614 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "min_size": { 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_size", 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.615 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default min_size for any created local pool", 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.616 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "num_rep": { 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "num_rep", 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3", 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.617 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default replica count for any created local pool", 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_num": { 2024-02-19T16:33:01.618 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_num", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "128", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default pg_num for any created local pool", 2024-02-19T16:33:01.619 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "prefix": { 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prefix", 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.620 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name prefix for any created local pool", 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.621 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "subtree": { 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "subtree", 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "rack", 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.622 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.623 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mds_autoscaler", 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.624 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.625 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.626 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.627 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.628 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.629 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.630 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.631 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.632 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirroring", 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.633 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.634 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.635 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.636 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.637 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.638 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.639 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.640 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "nfs", 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.641 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.642 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.643 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.644 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.645 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.646 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.647 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.648 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2024-02-19T16:33:01.649 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "fail_fs": { 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "fail_fs", 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.651 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.652 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.653 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.654 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.655 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.656 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.657 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.658 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.659 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator": { 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.660 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: "rook", 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: "test_orchestrator" 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Orchestrator backend", 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.661 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_perf_query", 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.662 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.663 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.664 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.665 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.666 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.667 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.668 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.669 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.670 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_support", 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.671 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.672 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.673 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.674 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.675 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.676 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.677 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.678 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.679 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_autoscaler", 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.680 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.681 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.682 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.683 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.684 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.685 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.686 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.687 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.688 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "threshold": { 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threshold", 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.689 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1.0", 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "scaling threshold", 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.690 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "progress", 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.691 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_pg_recovery_event": { 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_pg_recovery_event", 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.692 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2024-02-19T16:33:01.693 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.694 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.695 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.696 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.697 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.698 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.699 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.700 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.701 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "max_completed_events": { 2024-02-19T16:33:01.702 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_completed_events", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "50", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "number of past completed events to remember", 2024-02-19T16:33:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.704 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long the module is going to sleep", 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.705 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus", 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.706 INFO:teuthology.orchestra.run.smithi008.stdout: "cache": { 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cache", 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.707 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "exclude_perf_counters": { 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "exclude_perf_counters", 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.708 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.709 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools": { 2024-02-19T16:33:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools", 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300", 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_interval": { 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_interval", 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15.0", 2024-02-19T16:33:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "9283", 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "stale_cache_strategy": { 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stale_cache_strategy", 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "log", 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2024-02-19T16:33:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "default", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "default", 2024-02-19T16:33:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: "error" 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2024-02-19T16:33:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_snap_create": { 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_snap_create", 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "mirror_snapshot_schedule": { 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirror_snapshot_schedule", 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: "trash_purge_schedule": { 2024-02-19T16:33:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "trash_purge_schedule", 2024-02-19T16:33:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.744 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "restful", 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.745 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_auth": { 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_auth", 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.746 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.747 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.748 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.749 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.750 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.751 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.752 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.753 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.754 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.755 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:01.756 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.757 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.758 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.759 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rgw", 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.760 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.761 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.762 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.763 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.764 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.765 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.767 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.768 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rook", 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "drive_group_interval": { 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "drive_group_interval", 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.769 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300.0", 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.770 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.771 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.772 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.773 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.774 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.775 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.776 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.777 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "storage_class": { 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "storage_class", 2024-02-19T16:33:01.778 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "local", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.779 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "selftest", 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.780 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.781 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.782 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.783 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.784 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.785 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.786 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.787 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.788 INFO:teuthology.orchestra.run.smithi008.stdout: "roption1": { 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption1", 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.789 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "roption2": { 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption2", 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.790 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "xyz", 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.791 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption1": { 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption1", 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.792 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption2": { 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption2", 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.793 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.794 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption3": { 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption3", 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.795 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption4": { 2024-02-19T16:33:01.796 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption4", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.797 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption5": { 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption5", 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.798 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.799 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption6": { 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption6", 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.800 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption7": { 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption7", 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.801 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "42", 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.802 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "testkey": { 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testkey", 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.803 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "testlkey": { 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testlkey", 2024-02-19T16:33:01.804 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.805 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "testnewline": { 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testnewline", 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.806 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.807 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.808 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_m_granularity": { 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_m_granularity", 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.809 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow minute scheduled snapshots", 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "dump_on_update": { 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dump_on_update", 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.810 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "dump database to debug log on update", 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.811 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.812 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.813 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.814 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.815 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.816 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.817 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.818 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.819 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.820 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stats", 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.821 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.822 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.823 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.824 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.825 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.826 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.827 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.828 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.829 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "status", 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.830 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.831 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.832 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.833 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.834 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.835 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.836 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.837 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.838 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telegraf", 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "address": { 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "address", 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.839 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.840 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15", 2024-02-19T16:33:01.841 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.842 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.843 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.844 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.845 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.846 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.847 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.848 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.849 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.850 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telemetry", 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_basic": { 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_basic", 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.851 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.852 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_crash": { 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_crash", 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.853 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_device": { 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_device", 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.854 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.855 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_ident": { 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_ident", 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.856 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.857 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_perf": { 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_perf", 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.858 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "contact": { 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "contact", 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.859 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.860 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "description": { 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "description", 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.861 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.862 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "device_url": { 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_url", 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.863 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2024-02-19T16:33:01.864 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.865 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.866 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "24", 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "8", 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.867 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "last_opt_revision": { 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "last_opt_revision", 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1", 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.868 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard": { 2024-02-19T16:33:01.869 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.870 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard_description": { 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard_description", 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.871 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.872 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.873 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.874 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.875 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.876 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.877 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.878 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.879 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.880 INFO:teuthology.orchestra.run.smithi008.stdout: "organization": { 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "organization", 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.881 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "proxy": { 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "proxy", 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.882 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.883 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "url": { 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url", 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-19T16:33:01.884 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.885 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "test_orchestrator", 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.886 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.887 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.888 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.889 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.890 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.891 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.892 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.893 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.894 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.895 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.896 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.898 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.899 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.900 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.901 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.902 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_clones": { 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_clones", 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4", 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.903 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "periodic_async_work": { 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "periodic_async_work", 2024-02-19T16:33:01.904 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Periodically check for async work", 2024-02-19T16:33:01.905 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "snapshot_clone_delay": { 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snapshot_clone_delay", 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.906 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.907 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix", 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.908 INFO:teuthology.orchestra.run.smithi008.stdout: "discovery_interval": { 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "discovery_interval", 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "100", 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.909 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "identifier": { 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "identifier", 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.910 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.911 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.912 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.913 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.914 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.915 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.916 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.917 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.918 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.919 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.920 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_host": { 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_host", 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.921 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.922 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_port": { 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_port", 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10051", 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.923 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_sender": { 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_sender", 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.924 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.925 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: "modules": [ 2024-02-19T16:33:01.926 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "dashboard", 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "iostat", 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "nfs", 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "restful", 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "snap_schedule" 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "available_modules": [ 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.927 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alerts", 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.928 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How frequently to reexamine health status", 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.929 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.930 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.931 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.932 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.933 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.934 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.935 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.936 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_destination": { 2024-02-19T16:33:01.937 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_destination", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email address to send alerts to", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.938 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_from_name": { 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_from_name", 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "Ceph", 2024-02-19T16:33:01.939 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email From: name", 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.940 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_host": { 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_host", 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.941 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP server", 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_password": { 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_password", 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.942 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Password to authenticate with", 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.943 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_port": { 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_port", 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "465", 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.944 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP port", 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_sender": { 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_sender", 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.945 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP envelope sender", 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.946 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_ssl": { 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_ssl", 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.947 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_user": { 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_user", 2024-02-19T16:33:01.948 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "User to authenticate as", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.949 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "balancer", 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.950 INFO:teuthology.orchestra.run.smithi008.stdout: "active": { 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "active", 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.951 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically balance PGs across cluster", 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_time": { 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_time", 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.952 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0000", 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "beginning time of day to automatically balance", 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.953 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_weekday": { 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_weekday", 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2024-02-19T16:33:01.954 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_max_iterations": { 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_max_iterations", 2024-02-19T16:33:01.955 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "25", 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "250", 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.956 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_metrics": { 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_metrics", 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.957 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "pgs,objects,bytes", 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-19T16:33:01.958 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_step": { 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_step", 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.5", 2024-02-19T16:33:01.959 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0.001", 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "0.999", 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "aggressiveness of optimization", 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.960 INFO:teuthology.orchestra.run.smithi008.stdout: "end_time": { 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_time", 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2359", 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.961 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ending time of day to automatically balance", 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "end_weekday": { 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_weekday", 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.962 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-19T16:33:01.963 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.964 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.965 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.966 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.967 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:01.968 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:01.969 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.970 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "min_score": { 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_score", 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.973 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2024-02-19T16:33:01.974 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "upmap", 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.975 INFO:teuthology.orchestra.run.smithi008.stdout: "crush-compat", 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: "none", 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: "read", 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap", 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap-read" 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Balancer mode", 2024-02-19T16:33:01.976 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_ids": { 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_ids", 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.977 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.978 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.979 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_deviation": { 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_deviation", 2024-02-19T16:33:01.980 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-19T16:33:01.981 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_optimizations": { 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_optimizations", 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:01.982 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:01.983 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm", 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_down_multiplier": { 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_down_multiplier", 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.984 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.985 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_refresh_rate": { 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_refresh_rate", 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "20", 2024-02-19T16:33:01.986 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.987 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_starting_port": { 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_starting_port", 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4721", 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.988 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_ptrace": { 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_ptrace", 2024-02-19T16:33:01.989 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-19T16:33:01.990 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_interval": { 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_interval", 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.991 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.992 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_memory_target_ratio": { 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_memory_target_ratio", 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.7", 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.993 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm_log_destination": { 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm_log_destination", 2024-02-19T16:33:01.994 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "file", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "file,syslog", 2024-02-19T16:33:01.995 INFO:teuthology.orchestra.run.smithi008.stdout: "syslog" 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: "cgroups_split": { 2024-02-19T16:33:01.996 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cgroups_split", 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.997 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "config_checks_enabled": { 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_checks_enabled", 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:01.998 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:01.999 INFO:teuthology.orchestra.run.smithi008.stdout: "config_dashboard": { 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_dashboard", 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.000 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_alertmanager": { 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_alertmanager", 2024-02-19T16:33:02.001 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:02.002 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_base": { 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_base", 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.003 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Container image name, without the tag", 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.004 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_elasticsearch": { 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_elasticsearch", 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.005 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "elasticsearch container image", 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_grafana": { 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_grafana", 2024-02-19T16:33:02.006 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:02.007 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_haproxy": { 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_haproxy", 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.008 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "HAproxy container image", 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.009 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_agent": { 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_agent", 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.010 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger agent container image", 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_collector": { 2024-02-19T16:33:02.011 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_collector", 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.012 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger collector container image", 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_query": { 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_query", 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.013 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger query container image", 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.014 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_keepalived": { 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_keepalived", 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.015 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Keepalived container image", 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.016 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_loki": { 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_loki", 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.017 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Loki container image", 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_node_exporter": { 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_node_exporter", 2024-02-19T16:33:02.018 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.019 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_nvmeof": { 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_nvmeof", 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.020 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Nvme-of container image", 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_prometheus": { 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_prometheus", 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_promtail": { 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_promtail", 2024-02-19T16:33:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Promtail container image", 2024-02-19T16:33:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_snmp_gateway": { 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_snmp_gateway", 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SNMP Gateway container image", 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: "container_init": { 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_init", 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Run podman/docker with `--init`", 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "daemon_cache_timeout": { 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "daemon_cache_timeout", 2024-02-19T16:33:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "default_cephadm_command_timeout": { 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_cephadm_command_timeout", 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "900", 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "default_registry": { 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_registry", 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io", 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "device_cache_timeout": { 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_cache_timeout", 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.033 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1800", 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache device inventory", 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.034 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "device_enhanced_scan": { 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_enhanced_scan", 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.035 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "facts_cache_timeout": { 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "facts_cache_timeout", 2024-02-19T16:33:02.036 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache host facts data", 2024-02-19T16:33:02.037 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "host_check_interval": { 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "host_check_interval", 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.038 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to perform a host check", 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.039 INFO:teuthology.orchestra.run.smithi008.stdout: "hw_monitoring": { 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hw_monitoring", 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.040 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "inventory_list_all": { 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "inventory_list_all", 2024-02-19T16:33:02.041 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.042 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.043 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.044 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.045 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "log_refresh_metadata": { 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_refresh_metadata", 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.046 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.047 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.048 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.049 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.050 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.051 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.052 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-19T16:33:02.053 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-19T16:33:02.054 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.055 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "*", 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.056 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "max_count_per_host": { 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_count_per_host", 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.057 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of daemons per service per host", 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "max_osd_draining_count": { 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_osd_draining_count", 2024-02-19T16:33:02.058 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.059 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "migration_current": { 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "migration_current", 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.060 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "internal - do not modify", 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2024-02-19T16:33:02.061 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "root", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm-package", 2024-02-19T16:33:02.062 INFO:teuthology.orchestra.run.smithi008.stdout: "root" 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "mode for remote execution of cephadm", 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "oob_default_addr": { 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "oob_default_addr", 2024-02-19T16:33:02.063 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "169.254.1.1", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.064 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_alerts_path": { 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_alerts_path", 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-19T16:33:02.065 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.066 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_insecure": { 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_insecure", 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.067 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_password": { 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_password", 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.068 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.069 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_url": { 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_url", 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.070 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_username": { 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_username", 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.071 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.072 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "secure_monitoring_stack": { 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "secure_monitoring_stack", 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.073 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "service_discovery_port": { 2024-02-19T16:33:02.074 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "service_discovery_port", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8765", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "cephadm service discovery port", 2024-02-19T16:33:02.075 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "ssh_config_file": { 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssh_config_file", 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.076 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "use_agent": { 2024-02-19T16:33:02.077 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_agent", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-19T16:33:02.078 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "use_repo_digest": { 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_repo_digest", 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.079 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.080 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_failed_host_check": { 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_failed_host_check", 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.081 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if the host check fails", 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_daemons": { 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_daemons", 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.082 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.083 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_hosts": { 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_hosts", 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.084 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.085 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crash", 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.086 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.087 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.088 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.089 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.090 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.091 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.092 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.093 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.094 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.095 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.095 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.095 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.095 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.095 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.095 INFO:teuthology.orchestra.run.smithi008.stdout: "retain_interval": { 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retain_interval", 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "31536000", 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.096 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_recent_interval": { 2024-02-19T16:33:02.097 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_recent_interval", 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1209600", 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.098 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dashboard", 2024-02-19T16:33:02.099 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.100 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_HOST": { 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.102 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.104 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_ENABLED": { 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_ENABLED", 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.105 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-19T16:33:02.106 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.107 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "ENABLE_BROWSABLE_API": { 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.108 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.109 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.110 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-19T16:33:02.111 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.112 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.113 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.114 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.115 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.116 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.117 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-19T16:33:02.118 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.119 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-19T16:33:02.120 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.121 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.122 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.123 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_PASSWORD": { 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-19T16:33:02.125 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.126 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_URL": { 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_URL", 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.127 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.128 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_USERNAME": { 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.129 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.130 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.131 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.132 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-19T16:33:02.133 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.134 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.135 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_HOST": { 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-19T16:33:02.136 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.137 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.138 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-19T16:33:02.139 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.140 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.141 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.142 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.143 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-19T16:33:02.144 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.145 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.146 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.147 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.148 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-19T16:33:02.149 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.150 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_ENABLED": { 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.151 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.152 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.153 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.154 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.155 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8", 2024-02-19T16:33:02.156 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-19T16:33:02.157 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "45", 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.158 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ACCESS_KEY": { 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.159 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.160 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.161 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SECRET_KEY": { 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-19T16:33:02.162 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.163 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SSL_VERIFY": { 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.164 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.165 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:02.166 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-19T16:33:02.167 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.168 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.169 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.170 INFO:teuthology.orchestra.run.smithi008.stdout: "cross_origin_url": { 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cross_origin_url", 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.171 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "crt_file": { 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crt_file", 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.172 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.173 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "debug": { 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "debug", 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.174 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable/disable debug options", 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.175 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "jwt_token_ttl": { 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "jwt_token_ttl", 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "28800", 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.176 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2024-02-19T16:33:02.177 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.178 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.179 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.180 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.181 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.182 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.183 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.184 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.185 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.186 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "motd": { 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "motd", 2024-02-19T16:33:02.187 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "The message of the day", 2024-02-19T16:33:02.188 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect_resolve_ip_addr": { 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "redirect_resolve_ip_addr", 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.189 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.190 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.191 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.192 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8080", 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.193 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.194 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl_server_port": { 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl_server_port", 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.195 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8443", 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.196 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "redirect", 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.197 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect" 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.198 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.199 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.200 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "url_prefix": { 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url_prefix", 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.201 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2024-02-19T16:33:02.202 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_monitoring": { 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_monitoring", 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.203 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "monitor device health metrics", 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.204 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.205 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.206 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.207 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.208 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.209 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.210 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.211 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "mark_out_threshold": { 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mark_out_threshold", 2024-02-19T16:33:02.212 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2419200", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.213 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_name": { 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_name", 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.214 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "device_health_metrics", 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: "retention_period": { 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retention_period", 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15552000", 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.216 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain device health metrics", 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_frequency": { 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_frequency", 2024-02-19T16:33:02.217 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.218 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "self_heal": { 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "self_heal", 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.219 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.220 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.221 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and check device health", 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_threshold": { 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_threshold", 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.222 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7257600", 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.223 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "diskprediction_local", 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.224 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.226 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.227 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.228 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.229 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.230 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.231 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.232 INFO:teuthology.orchestra.run.smithi008.stdout: "predict_interval": { 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predict_interval", 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "predictor_model": { 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predictor_model", 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.234 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "prophetstor", 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.235 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.236 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "influx", 2024-02-19T16:33:02.237 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": false, 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "influxdb python module not found", 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "batch_size": { 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "batch_size", 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5000", 2024-02-19T16:33:02.238 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "database": { 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "database", 2024-02-19T16:33:02.239 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "ceph", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.240 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": { 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hostname", 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.241 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server hostname", 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:02.242 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "30", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "5", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-19T16:33:02.243 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.244 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.245 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.247 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.248 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.249 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.250 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.251 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "password": { 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "password", 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.252 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "password of InfluxDB server user", 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.253 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "port": { 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "port", 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8086", 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.254 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server port", 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.255 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "false", 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.256 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "threads": { 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threads", 2024-02-19T16:33:02.257 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "32", 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-19T16:33:02.258 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: "username": { 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "username", 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.259 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "username of InfluxDB server user", 2024-02-19T16:33:02.260 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "verify_ssl": { 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "verify_ssl", 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.261 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "true", 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.262 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "insights", 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.263 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.264 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.265 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.266 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.267 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.268 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.270 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.271 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "iostat", 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.272 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.273 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.274 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.275 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.277 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.278 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.279 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.280 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "k8sevents", 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_event_retention_days": { 2024-02-19T16:33:02.281 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ceph_event_retention_days", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-19T16:33:02.282 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "config_check_secs": { 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_check_secs", 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.283 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "10", 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.284 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.285 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.286 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.287 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.288 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.289 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.290 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.291 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.292 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "localpool", 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.293 INFO:teuthology.orchestra.run.smithi008.stdout: "failure_domain": { 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "failure_domain", 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "host", 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.294 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "failure domain for any created local pool", 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.295 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.296 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.297 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.298 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.299 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.300 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.301 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.302 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.303 INFO:teuthology.orchestra.run.smithi008.stdout: "min_size": { 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_size", 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.304 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default min_size for any created local pool", 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "num_rep": { 2024-02-19T16:33:02.305 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "num_rep", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default replica count for any created local pool", 2024-02-19T16:33:02.306 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_num": { 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_num", 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.307 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "128", 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default pg_num for any created local pool", 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.308 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "prefix": { 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prefix", 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.309 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name prefix for any created local pool", 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "subtree": { 2024-02-19T16:33:02.310 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "subtree", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "rack", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-19T16:33:02.311 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mds_autoscaler", 2024-02-19T16:33:02.312 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.313 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.314 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.315 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.316 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.317 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.318 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.319 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.320 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirroring", 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.321 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.322 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.323 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.324 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.325 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.326 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.327 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.328 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "nfs", 2024-02-19T16:33:02.329 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.330 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.331 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.332 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.333 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.334 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.335 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.336 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "fail_fs": { 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "fail_fs", 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.338 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.340 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.341 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.342 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.343 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.344 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.345 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.346 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.347 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator": { 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.348 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: "rook", 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: "test_orchestrator" 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Orchestrator backend", 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.349 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_perf_query", 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.350 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.351 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.352 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.353 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.354 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.355 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.356 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.357 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.358 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_support", 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.359 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.360 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.361 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.362 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.363 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.364 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.365 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.366 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.367 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.368 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_autoscaler", 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.369 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.370 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.371 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.372 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.373 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.374 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.375 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.376 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.377 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.378 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "threshold": { 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threshold", 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2024-02-19T16:33:02.379 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1.0", 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "scaling threshold", 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.380 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "progress", 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_pg_recovery_event": { 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_pg_recovery_event", 2024-02-19T16:33:02.381 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.382 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.383 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.384 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.385 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.386 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.387 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.388 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.389 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.390 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.391 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "max_completed_events": { 2024-02-19T16:33:02.392 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_completed_events", 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "50", 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.393 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "number of past completed events to remember", 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.394 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long the module is going to sleep", 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.395 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus", 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: "cache": { 2024-02-19T16:33:02.396 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cache", 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.397 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "exclude_perf_counters": { 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "exclude_perf_counters", 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.398 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.399 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.400 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.401 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.402 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.403 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.404 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.405 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.406 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools": { 2024-02-19T16:33:02.407 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.408 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.409 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300", 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.410 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_interval": { 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_interval", 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15.0", 2024-02-19T16:33:02.411 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.412 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.413 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:02.414 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "9283", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.415 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "stale_cache_strategy": { 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stale_cache_strategy", 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "log", 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.416 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.417 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "default", 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.418 INFO:teuthology.orchestra.run.smithi008.stdout: "default", 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "error" 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2024-02-19T16:33:02.419 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.421 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.422 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.423 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.424 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.425 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.426 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.427 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.428 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_snap_create": { 2024-02-19T16:33:02.429 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_snap_create", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.430 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "mirror_snapshot_schedule": { 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirror_snapshot_schedule", 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.431 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.432 INFO:teuthology.orchestra.run.smithi008.stdout: "trash_purge_schedule": { 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "trash_purge_schedule", 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.433 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "restful", 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.434 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_auth": { 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_auth", 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.435 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2024-02-19T16:33:02.436 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.437 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.438 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.439 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.440 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.443 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.444 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.445 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.446 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.447 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.448 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.449 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.450 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.451 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rgw", 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.452 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.453 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.454 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.455 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.456 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.457 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.458 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.459 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rook", 2024-02-19T16:33:02.460 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "drive_group_interval": { 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "drive_group_interval", 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300.0", 2024-02-19T16:33:02.461 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.462 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.463 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.464 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.465 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.466 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.467 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.468 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.468 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.468 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.468 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.468 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.468 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.469 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.470 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: "storage_class": { 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "storage_class", 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.471 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "local", 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-19T16:33:02.472 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "selftest", 2024-02-19T16:33:02.473 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.474 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.475 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.476 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.477 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.478 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.479 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.480 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.481 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "roption1": { 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption1", 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.482 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.483 INFO:teuthology.orchestra.run.smithi008.stdout: "roption2": { 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption2", 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "xyz", 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.484 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption1": { 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption1", 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.485 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.486 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption2": { 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption2", 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.487 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption3": { 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption3", 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2024-02-19T16:33:02.488 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.489 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption4": { 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption4", 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.490 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.491 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption5": { 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption5", 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.492 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption6": { 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption6", 2024-02-19T16:33:02.493 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.494 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption7": { 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption7", 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.495 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "42", 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.496 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "testkey": { 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testkey", 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.497 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: "testlkey": { 2024-02-19T16:33:02.498 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testlkey", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.499 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: "testnewline": { 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testnewline", 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.500 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.501 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.502 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_m_granularity": { 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_m_granularity", 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.503 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow minute scheduled snapshots", 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "dump_on_update": { 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dump_on_update", 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.504 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "dump database to debug log on update", 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.505 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.506 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.507 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.508 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.509 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.510 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.511 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.512 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.513 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stats", 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.514 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.516 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.517 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.518 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.519 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.520 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.521 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "status", 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.522 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.523 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.524 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.525 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.526 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.527 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.528 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.529 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.530 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telegraf", 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "address": { 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "address", 2024-02-19T16:33:02.531 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.532 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.533 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15", 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.534 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.535 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.536 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.537 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.538 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.539 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.540 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.541 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.542 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telemetry", 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_basic": { 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_basic", 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.543 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.544 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_crash": { 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_crash", 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.545 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.546 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_device": { 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_device", 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.547 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_ident": { 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_ident", 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.548 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.549 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_perf": { 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_perf", 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.550 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.551 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "contact": { 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "contact", 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.552 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "description": { 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "description", 2024-02-19T16:33:02.553 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.554 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "device_url": { 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_url", 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.555 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2024-02-19T16:33:02.556 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.557 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "24", 2024-02-19T16:33:02.558 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "8", 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "last_opt_revision": { 2024-02-19T16:33:02.559 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "last_opt_revision", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.560 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard": { 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard", 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.561 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard_description": { 2024-02-19T16:33:02.562 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard_description", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.563 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.564 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.565 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.566 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.567 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.568 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.569 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.570 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.571 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "organization": { 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "organization", 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.572 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.573 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "proxy": { 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "proxy", 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.574 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "url": { 2024-02-19T16:33:02.575 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.576 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "test_orchestrator", 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.578 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.579 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.580 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.581 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.582 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.583 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.584 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.585 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.586 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.587 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.588 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.589 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.590 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.591 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.592 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.593 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_clones": { 2024-02-19T16:33:02.594 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_clones", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-19T16:33:02.595 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "periodic_async_work": { 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "periodic_async_work", 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.596 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Periodically check for async work", 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.597 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "snapshot_clone_delay": { 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snapshot_clone_delay", 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.598 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix", 2024-02-19T16:33:02.599 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "discovery_interval": { 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "discovery_interval", 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "100", 2024-02-19T16:33:02.600 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "identifier": { 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "identifier", 2024-02-19T16:33:02.601 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.602 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2024-02-19T16:33:02.603 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.604 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.605 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.606 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.607 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.608 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.609 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.610 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2024-02-19T16:33:02.611 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.612 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_host": { 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_host", 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.613 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_port": { 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_port", 2024-02-19T16:33:02.614 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10051", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.615 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_sender": { 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_sender", 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2024-02-19T16:33:02.616 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2024-02-19T16:33:02.617 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: "services": { 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: "dashboard": "https://172.21.15.8:8443/" 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: "always_on_modules": { 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: "octopus": [ 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2024-02-19T16:33:02.618 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2024-02-19T16:33:02.619 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "pacific": [ 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2024-02-19T16:33:02.620 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "quincy": [ 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2024-02-19T16:33:02.621 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "reef": [ 2024-02-19T16:33:02.622 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2024-02-19T16:33:02.623 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "squid": [ 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2024-02-19T16:33:02.624 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: "last_failure_osd_epoch": 96, 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: "active_clients": [ 2024-02-19T16:33:02.625 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 2373667545 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.626 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "libcephsqlite", 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 2803356442 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.627 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 2874060145 2024-02-19T16:33:02.628 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2024-02-19T16:33:02.629 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 891149798 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.630 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 759690331 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2024-02-19T16:33:02.631 INFO:teuthology.orchestra.run.smithi008.stdout: { 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 3171683486 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-02-19T16:33:02.632 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:00 smithi191 ceph-mon[37203]: pgmap v314: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:00 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3904356034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:00 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/295934450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:00 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/577265936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:00 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2811250628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:00 smithi156 ceph-mon[36502]: pgmap v314: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T16:33:02.634 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:00 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3904356034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.635 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:00 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/295934450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.635 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:00 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/577265936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.635 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:00 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2811250628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9fa40a2230 msgr2=0x7f9fa40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9fa40a2230 0x7f9fa40a46f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc1989f0 tx=0x7f9fac009260 comp rx=0 tx=0).stop 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 msgr2=0x7f9fbc1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc1978b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400eb10 tx=0x7f9fb400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 shutdown_connections 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9fbc107590 0x7f9fbc1978b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc101e90 0x7f9fbc19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f9fa40a2230 0x7f9fa40a46f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:02.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.524+0000 7f9fc225b640 1 --2- 172.21.15.8:0/2811250628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc1014d0 0x7f9fbc19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:02.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.525+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 >> 172.21.15.8:0/2811250628 conn(0x7f9fbc0fcfa0 msgr2=0x7f9fbc0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:02.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.525+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 shutdown_connections 2024-02-19T16:33:02.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:00.525+0000 7f9fc225b640 1 -- 172.21.15.8:0/2811250628 wait complete. 2024-02-19T16:33:02.644 DEBUG:tasks.check_counter:Getting stats from a 2024-02-19T16:33:02.644 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-19T16:33:02.644 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:00 smithi008 ceph-mon[32573]: pgmap v314: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T16:33:02.645 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:00 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3904356034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.645 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:00 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/295934450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.645 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:00 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/577265936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:33:02.645 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:00 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2811250628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-19T16:33:02.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:02 smithi156 ceph-mon[36502]: pgmap v315: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:33:02.901 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:33:03.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:02 smithi008 ceph-mon[32573]: pgmap v315: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:33:03.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:02 smithi191 ceph-mon[37203]: pgmap v315: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:33:04.668 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:04 smithi008 ceph-mon[32573]: pgmap v316: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:33:04.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:04 smithi156 ceph-mon[36502]: pgmap v316: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:33:04.931 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:33:04.931 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:04.931 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 0, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 2, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 0, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 0, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.001279898, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000000000, 2024-02-19T16:33:04.932 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.000326070, 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:04.933 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 0 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 6, 2024-02-19T16:33:04.934 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 161, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 3344, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 183296, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.014411185, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.009815203, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.002274916, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 159, 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.010586794, 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000066583 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:04.936 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 183296 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 219, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 208, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 1568368, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 124320, 2024-02-19T16:33:04.937 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.042249026, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.014280385, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.032322500, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.004373846, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 207, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.010263900, 2024-02-19T16:33:04.938 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000049584 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 1568368, 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 124320 2024-02-19T16:33:04.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:04.940 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:04.941 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:04.942 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:04.943 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:04.943 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:04.943 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:04.943 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:04.943 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:04.943 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:04.944 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 149017, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 31, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:04.945 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2024-02-19T16:33:04.946 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2024-02-19T16:33:04.947 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 221, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 1537484, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 221, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 221, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 1537484, 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2024-02-19T16:33:04.948 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:04.949 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:04.949 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:04.949 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:04.949 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:04.949 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:33:05.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:04 smithi191 ceph-mon[37203]: pgmap v316: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T16:33:05.672 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-19T16:33:05.673 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-19T16:33:05.673 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-02-19T16:33:05.673 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-02-19T16:33:05.673 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-02-19T16:33:05.673 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-02-19T16:33:05.673 DEBUG:tasks.check_counter:Getting stats from d 2024-02-19T16:33:05.674 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-19T16:33:05.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:05 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:33:05.958 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:33:06.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:05 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:33:06.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:05 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T16:33:06.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:06 smithi008 ceph-mon[32573]: pgmap v317: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-02-19T16:33:06.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:06 smithi156 ceph-mon[36502]: pgmap v317: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-02-19T16:33:07.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:06 smithi191 ceph-mon[37203]: pgmap v317: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-02-19T16:33:08.434 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: pgmap v318: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.435 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.436 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.436 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.436 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:08 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.471 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:33:08.471 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 0, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 2, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 0, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 0, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.001189981, 2024-02-19T16:33:08.472 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000000000, 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.000300935, 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.473 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 0 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.474 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 225, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 214, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 1570000, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 128064, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.043417415, 2024-02-19T16:33:08.475 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.014437069, 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.034048271, 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.004324638, 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 213, 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.010274800, 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000048238 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.476 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 1570000, 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 128064 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:08.477 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 6, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 165, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 3344, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 184000, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.016234454, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.011517338, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.002248380, 2024-02-19T16:33:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 163, 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.010826805, 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000066422 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:08.479 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 184000 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2024-02-19T16:33:08.480 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:08.481 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:08.482 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:08.483 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:08.484 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 149057, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 32, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:08.485 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2024-02-19T16:33:08.486 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2024-02-19T16:33:08.487 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 227, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 1538264, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 227, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 227, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 1538264, 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2024-02-19T16:33:08.488 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:08.489 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:08.489 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:08.489 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:08.489 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:08.489 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:33:08.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: pgmap v318: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:33:08.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.529 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.529 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:08 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: pgmap v318: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.582 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.582 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.582 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:08.582 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:08 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:09.447 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:09 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-02-19T16:33:09.448 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:09 smithi008 ceph-mon[32573]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-19T16:33:09.448 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:09 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-02-19T16:33:09.448 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:09 smithi008 ceph-mon[32573]: Unable to set osd_memory_target on smithi191 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-02-19T16:33:09.477 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-19T16:33:09.477 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-19T16:33:09.478 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-02-19T16:33:09.478 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-02-19T16:33:09.478 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-02-19T16:33:09.478 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-02-19T16:33:09.478 DEBUG:tasks.check_counter:Getting stats from g 2024-02-19T16:33:09.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-19T16:33:09.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:09 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-02-19T16:33:09.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:09 smithi191 ceph-mon[37203]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-19T16:33:09.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:09 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-02-19T16:33:09.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:09 smithi191 ceph-mon[37203]: Unable to set osd_memory_target on smithi191 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-02-19T16:33:09.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:09 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-02-19T16:33:09.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:09 smithi156 ceph-mon[36502]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-19T16:33:09.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:09 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-02-19T16:33:09.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:09 smithi156 ceph-mon[36502]: Unable to set osd_memory_target on smithi191 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-02-19T16:33:09.755 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:33:10.134 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:10 smithi008 ceph-mon[32573]: pgmap v319: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-02-19T16:33:10.134 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:10 smithi008 ceph-mon[32573]: osdmap e105: 12 total, 12 up, 12 in 2024-02-19T16:33:10.134 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:10 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:10.135 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:10 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:10.481 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:10 smithi156 ceph-mon[36502]: pgmap v319: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-02-19T16:33:10.482 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:10 smithi156 ceph-mon[36502]: osdmap e105: 12 total, 12 up, 12 in 2024-02-19T16:33:10.482 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:10 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:10.482 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:10 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:10 smithi191 ceph-mon[37203]: pgmap v319: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-02-19T16:33:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:10 smithi191 ceph-mon[37203]: osdmap e105: 12 total, 12 up, 12 in 2024-02-19T16:33:10.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:10 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:10.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:10 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:11.156 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.156 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.157 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.157 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.157 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:33:11.157 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:33:11.157 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:11 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:11.527 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:33:11.527 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:11.527 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 4, 2024-02-19T16:33:11.527 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 4, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 3024, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 448, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.001501989, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000043513, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.000577550, 2024-02-19T16:33:11.528 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 3, 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000825948, 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000275316 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:11.529 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 448 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 221, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 210, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 1568912, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 125568, 2024-02-19T16:33:11.530 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.042531287, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.014249601, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.033469144, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.004240690, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 209, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.009500656, 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000045457 2024-02-19T16:33:11.531 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 1568912, 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 125568 2024-02-19T16:33:11.532 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 2, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 160, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 320, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 183024, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:11.533 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.014018403, 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.009626110, 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.001910073, 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 157, 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.009686803, 2024-02-19T16:33:11.534 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000061699 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 183024 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.535 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:11.536 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:11.537 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:11.538 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:11.539 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 148995, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 31, 2024-02-19T16:33:11.540 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:11.541 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2024-02-19T16:33:11.542 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2024-02-19T16:33:11.543 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 223, 2024-02-19T16:33:11.544 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 1537744, 2024-02-19T16:33:11.544 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:11.544 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 223, 2024-02-19T16:33:11.544 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2024-02-19T16:33:11.544 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2024-02-19T16:33:11.544 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 223, 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 1537744, 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:11.545 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:33:11.546 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.546 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.546 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.546 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.546 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:33:11.546 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:33:11.547 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:11 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:11.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-19T16:33:11.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T16:33:11.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T16:33:11.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:11 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' 2024-02-19T16:33:12.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:12 smithi008 ceph-mon[32573]: pgmap v321: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:33:12.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:12 smithi008 ceph-mon[32573]: Adjusting osd_memory_target on smithi008 to 177.2M 2024-02-19T16:33:12.181 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:12 smithi008 ceph-mon[32573]: Unable to set osd_memory_target on smithi008 to 185894195: error parsing value: Value '185894195' is below minimum 939524096 2024-02-19T16:33:12.208 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-19T16:33:12.208 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-19T16:33:12.208 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-02-19T16:33:12.208 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-02-19T16:33:12.208 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-02-19T16:33:12.208 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-02-19T16:33:12.208 DEBUG:tasks.check_counter:Getting stats from j 2024-02-19T16:33:12.209 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-19T16:33:12.454 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.a/config 2024-02-19T16:33:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:12 smithi191 ceph-mon[37203]: pgmap v321: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:33:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:12 smithi191 ceph-mon[37203]: Adjusting osd_memory_target on smithi008 to 177.2M 2024-02-19T16:33:12.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:12 smithi191 ceph-mon[37203]: Unable to set osd_memory_target on smithi008 to 185894195: error parsing value: Value '185894195' is below minimum 939524096 2024-02-19T16:33:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:12 smithi156 ceph-mon[36502]: pgmap v321: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:33:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:12 smithi156 ceph-mon[36502]: Adjusting osd_memory_target on smithi008 to 177.2M 2024-02-19T16:33:12.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:12 smithi156 ceph-mon[36502]: Unable to set osd_memory_target on smithi008 to 185894195: error parsing value: Value '185894195' is below minimum 939524096 2024-02-19T16:33:14.167 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:14 smithi008 ceph-mon[32573]: pgmap v322: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:33:14.185 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-02-19T16:33:14.185 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:14.185 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 4, 2024-02-19T16:33:14.185 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 4, 2024-02-19T16:33:14.185 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 3024, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 448, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.001369351, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000035706, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.000432886, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:14.186 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 3, 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000638342, 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000212780 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:14.187 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 448 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 2, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 165, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 320, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 183904, 2024-02-19T16:33:14.188 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.015483733, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.011098756, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.001821098, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 162, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.010376131, 2024-02-19T16:33:14.189 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000064050 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-19T16:33:14.190 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 183904 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 227, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 216, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 1570544, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 129312, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.046241693, 2024-02-19T16:33:14.191 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.014446925, 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.037711947, 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.004302560, 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 215, 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.010660242, 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000049582 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.192 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 1570544, 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 129312 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2024-02-19T16:33:14.193 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:14.194 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:14.195 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:14.196 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:14.197 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 148692, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 30, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2024-02-19T16:33:14.198 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:14.199 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-02-19T16:33:14.200 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 229, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 1538524, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 229, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2024-02-19T16:33:14.201 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 229, 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 1538524, 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-02-19T16:33:14.202 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-02-19T16:33:14.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:14 smithi191 ceph-mon[37203]: pgmap v322: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:33:14.580 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:14 smithi156 ceph-mon[36502]: pgmap v322: 129 pgs: 129 active+clean; 79 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-02-19T16:33:15.019 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-19T16:33:15.020 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-19T16:33:15.020 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-19T16:33:15.020 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-19T16:33:15.020 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-19T16:33:15.020 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-02-19T16:33:15.020 DEBUG:tasks.check_counter:Getting stats from b 2024-02-19T16:33:15.021 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-19T16:33:15.268 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:33:16.407 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:16 smithi156 ceph-mon[36502]: pgmap v323: 129 pgs: 129 active+clean; 79 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:16.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:16 smithi008 ceph-mon[32573]: pgmap v323: 129 pgs: 129 active+clean; 79 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:16.527 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:16 smithi191 ceph-mon[37203]: pgmap v323: 129 pgs: 129 active+clean; 79 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:16.986 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-19T16:33:16.986 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:16.986 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 464, 2024-02-19T16:33:16.986 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 590, 2024-02-19T16:33:16.986 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 115138, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1284512, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 49, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.041286586, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.011769860, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.015694929, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.008120636, 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:16.987 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 588, 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.032673562, 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000055567 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 38, 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000003071, 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000080 2024-02-19T16:33:16.988 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 310320 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 537, 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 534, 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1695038, 2024-02-19T16:33:16.989 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 233699, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.052953726, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.016183507, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.041461910, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.007325103, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 533, 2024-02-19T16:33:16.990 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.046920323, 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000088030 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 34, 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000004679, 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000137 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1642720, 2024-02-19T16:33:16.991 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 165408 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2173, 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2306, 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 645092, 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1110645, 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 13, 2024-02-19T16:33:16.992 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.112944869, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.044869082, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.066137157, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.012031159, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2304, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.056684581, 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000024602 2024-02-19T16:33:16.993 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2873, 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000383570, 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000133 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 132256 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.994 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-19T16:33:16.995 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 900, 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.174632483, 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000194036 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 44, 2024-02-19T16:33:16.996 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001210264, 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000027506 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 215, 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 211, 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 211, 2024-02-19T16:33:16.997 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.155085163, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000735000 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 4, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 11, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-02-19T16:33:16.998 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 181, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 2, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 167, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 12, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 162, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 6, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 60, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 60, 2024-02-19T16:33:16.999 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 6, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 513, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 245, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 4, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 5, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 3, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-02-19T16:33:17.000 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 1, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 2, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 4, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 33, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 1020, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 3452, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 26460392, 2024-02-19T16:33:17.001 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 1020, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 14, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 14, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 1006, 2024-02-19T16:33:17.002 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 10, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 98, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 2, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 137, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 126, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 6, 2024-02-19T16:33:17.003 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 88 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 5, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-02-19T16:33:17.004 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 7, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 7, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-02-19T16:33:17.005 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 1, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.006 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 276, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 276, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 2, 2024-02-19T16:33:17.007 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 2, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 4672716, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-02-19T16:33:17.008 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 72, 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.251011169, 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003486266 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 152, 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 160, 2024-02-19T16:33:17.009 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 8, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 29, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 31, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 2, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 181, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 209, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 28, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 60, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 80, 2024-02-19T16:33:17.010 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 20, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 53972, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 223612 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 351, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 1, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 215, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 17, 2024-02-19T16:33:17.011 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 43, 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 47, 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.062627217, 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001332493 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.012 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.013 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 124, 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.038166296, 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000307792 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-02-19T16:33:17.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.015 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.016 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.017 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-02-19T16:33:17.018 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.018 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.018 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.018 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.018 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001421724, 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001421724 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.019 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.020 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 12, 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002046479, 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000170539 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4, 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002733919, 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000683479 2024-02-19T16:33:17.021 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13, 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.022454281, 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001727252 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.022 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.023 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.024 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.025 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.007249345, 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003624672 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-02-19T16:33:17.027 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.028 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000976085, 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000976085 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7, 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.017409817, 2024-02-19T16:33:17.029 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002487116 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.030 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-02-19T16:33:17.031 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 204, 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 102, 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 612, 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:17.032 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:17.033 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:17.034 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:17.035 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:17.036 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 303366, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 218, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 1408, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 16, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-19T16:33:17.037 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9984, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 210, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:17.038 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 542210, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 4724, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.039 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 1185, 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 504683, 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 33, 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 1185, 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-02-19T16:33:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1185, 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.090982547, 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000920660 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1185, 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1223 2024-02-19T16:33:17.041 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 1185, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 1025, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 160, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2024-02-19T16:33:17.042 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 5, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 77, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 16, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 26, 2024-02-19T16:33:17.043 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1020, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-02-19T16:33:17.044 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 2, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-02-19T16:33:17.045 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-02-19T16:33:17.046 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 106, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-02-19T16:33:17.047 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 15, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 39, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 8 2024-02-19T16:33:17.048 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.049 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 6, 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 3, 2024-02-19T16:33:17.050 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 14, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 7, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 3170, 2024-02-19T16:33:17.051 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 2156481, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 3170, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3170, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2156481, 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:17.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-19T16:33:17.053 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 1185, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 505446, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1185, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 505446, 2024-02-19T16:33:17.054 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-02-19T16:33:17.055 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 1185, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 1185, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1185, 2024-02-19T16:33:17.056 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1185, 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.057 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 7, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 763, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 7, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:17.058 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 5, 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 763, 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.059 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x56093d2706e0": { 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 276, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 478412, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 276, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 72, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 478412, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.062 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:17.062 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-19T16:33:17.769 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:17 smithi156 ceph-mon[36502]: osdmap e106: 12 total, 12 up, 12 in 2024-02-19T16:33:17.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:17 smithi008 ceph-mon[32573]: osdmap e106: 12 total, 12 up, 12 in 2024-02-19T16:33:17.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:17 smithi191 ceph-mon[37203]: osdmap e106: 12 total, 12 up, 12 in 2024-02-19T16:33:17.801 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-02-19T16:33:17.801 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-02-19T16:33:17.801 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-02-19T16:33:17.801 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-02-19T16:33:17.802 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-02-19T16:33:17.802 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-02-19T16:33:17.802 DEBUG:tasks.check_counter:Getting stats from e 2024-02-19T16:33:17.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-19T16:33:18.058 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:33:18.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:18 smithi008 ceph-mon[32573]: pgmap v325: 129 pgs: 129 active+clean; 79 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:33:18.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:18 smithi191 ceph-mon[37203]: pgmap v325: 129 pgs: 129 active+clean; 79 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:33:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:18 smithi156 ceph-mon[36502]: pgmap v325: 129 pgs: 129 active+clean; 79 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-02-19T16:33:19.789 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 0, 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2, 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 0, 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 0, 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2024-02-19T16:33:19.790 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001168882, 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000000000, 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000328645, 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:19.791 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-02-19T16:33:19.792 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 231, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 218, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1573568, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 129760, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.034190257, 2024-02-19T16:33:19.793 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.008383961, 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.032670269, 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.003747945, 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 218, 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.008073022, 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000037032 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1573568, 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 129760 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:19.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 165, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 320, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 183904, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.007767916, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.004535435, 2024-02-19T16:33:19.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001568329, 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 162, 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.012020645, 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000074201 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:19.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 183904 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2024-02-19T16:33:19.798 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:19.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:19.800 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:19.801 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-19T16:33:19.802 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 149017, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 31, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:19.803 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:19.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-19T16:33:19.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-19T16:33:19.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:19.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:19.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2024-02-19T16:33:19.805 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:19.806 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 229, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1538524, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 229, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 229, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1538524, 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:19.807 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:19.808 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:19.808 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:19.808 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:19.808 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:19.808 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-19T16:33:20.424 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-02-19T16:33:20.424 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-02-19T16:33:20.424 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-02-19T16:33:20.424 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-02-19T16:33:20.425 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-02-19T16:33:20.425 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-02-19T16:33:20.425 DEBUG:tasks.check_counter:Getting stats from h 2024-02-19T16:33:20.425 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-19T16:33:20.671 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:33:20.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:20 smithi008 ceph-mon[32573]: pgmap v326: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-02-19T16:33:20.775 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:20 smithi156 ceph-mon[36502]: pgmap v326: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-02-19T16:33:20.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:20 smithi191 ceph-mon[37203]: pgmap v326: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-02-19T16:33:22.353 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-19T16:33:22.353 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:22.353 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1014, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 996, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1880224, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 368831, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.092077006, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.027737568, 2024-02-19T16:33:22.354 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.049673051, 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.021038116, 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 995, 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.031291387, 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000031448 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.355 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1624528, 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 132160 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 4, 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 165, 2024-02-19T16:33:22.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 876, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 444044, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.007703499, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.004204119, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.002003996, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000040846, 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:22.357 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 162, 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.012780443, 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000078891 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 443360 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1412, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1412, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 840625, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 438343, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:22.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.065294980, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.016219916, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.018065244, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.026818243, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1411, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.026143129, 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000018528 2024-02-19T16:33:22.360 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2024-02-19T16:33:22.361 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3709, 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.070875973, 2024-02-19T16:33:22.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000019109 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.363 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-02-19T16:33:22.365 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 163, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 2, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 161, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 160, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 5, 2024-02-19T16:33:22.366 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-02-19T16:33:22.367 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2024-02-19T16:33:22.368 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-02-19T16:33:22.369 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2024-02-19T16:33:22.370 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-02-19T16:33:22.371 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-02-19T16:33:22.372 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-02-19T16:33:22.373 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 276, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-02-19T16:33:22.374 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 2, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-02-19T16:33:22.375 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 4672716, 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 4672142, 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 276 2024-02-19T16:33:22.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 141, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 151, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 10, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 18, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 21, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 3, 2024-02-19T16:33:22.377 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 163, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 171, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 8, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 48516, 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 215420 2024-02-19T16:33:22.378 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-02-19T16:33:22.379 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.381 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-02-19T16:33:22.382 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.383 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.384 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.385 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.386 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-02-19T16:33:22.386 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.386 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-02-19T16:33:22.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.388 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.389 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.391 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-02-19T16:33:22.392 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.393 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.395 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-02-19T16:33:22.396 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2024-02-19T16:33:22.397 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 2254, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.398 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:22.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:22.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:22.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:22.402 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 186382, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 184, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:22.403 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9744, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 204, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-19T16:33:22.404 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 469640, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 4258, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-02-19T16:33:22.405 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 2171, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 2171, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2171, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.217980530, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000561022 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.406 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2171, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2172 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 2171, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 2171, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-02-19T16:33:22.407 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 1383, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 786, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-02-19T16:33:22.408 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-02-19T16:33:22.409 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-02-19T16:33:22.410 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-02-19T16:33:22.411 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 106, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 15, 2024-02-19T16:33:22.412 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 4, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 4, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-02-19T16:33:22.413 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-02-19T16:33:22.414 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 0, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:22.415 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 2426, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 2487436, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 2426, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:22.416 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2426, 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2487436, 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-02-19T16:33:22.417 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2171, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 478417, 2024-02-19T16:33:22.418 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2171, 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 478417, 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2171, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2171, 2024-02-19T16:33:22.420 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2171, 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2171, 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-02-19T16:33:22.421 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-02-19T16:33:22.422 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x564ba5c28960": { 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:22.423 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-02-19T16:33:22.424 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:22.425 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-19T16:33:22.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:22 smithi008 ceph-mon[32573]: pgmap v327: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:22.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:22 smithi191 ceph-mon[37203]: pgmap v327: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:22.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:22 smithi156 ceph-mon[36502]: pgmap v327: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:23.174 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-02-19T16:33:23.174 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-02-19T16:33:23.174 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-02-19T16:33:23.174 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-02-19T16:33:23.174 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-02-19T16:33:23.175 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-02-19T16:33:23.175 DEBUG:tasks.check_counter:Getting stats from k 2024-02-19T16:33:23.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-19T16:33:23.422 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.b/config 2024-02-19T16:33:24.749 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:24 smithi156 ceph-mon[36502]: pgmap v328: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:24.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:24 smithi008 ceph-mon[32573]: pgmap v328: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:24.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:24 smithi191 ceph-mon[37203]: pgmap v328: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-19T16:33:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-19T16:33:25.129 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:25.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 4, 2024-02-19T16:33:25.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 4, 2024-02-19T16:33:25.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 3024, 2024-02-19T16:33:25.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 448, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001327517, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000036215, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000488787, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:25.130 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3, 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000695709, 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000231903 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:25.131 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2, 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 168, 2024-02-19T16:33:25.132 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 320, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 184432, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.007662576, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.004757524, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001268949, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-19T16:33:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 165, 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.014924267, 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000090450 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:25.134 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 184432 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 231, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 220, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1571632, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 131808, 2024-02-19T16:33:25.135 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.034185554, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.008449810, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.032768390, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.003683137, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 219, 2024-02-19T16:33:25.136 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.007714019, 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000035223 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.137 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1571632, 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 131808 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:25.138 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:25.139 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:25.140 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:25.141 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:25.142 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148670, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 30, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-19T16:33:25.143 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:25.144 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 233, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1539044, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 233, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-19T16:33:25.146 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 233, 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1539044, 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-19T16:33:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-19T16:33:25.842 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-19T16:33:25.842 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-19T16:33:25.842 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-02-19T16:33:25.842 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-02-19T16:33:25.843 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-02-19T16:33:25.843 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-02-19T16:33:25.843 DEBUG:tasks.check_counter:Getting stats from c 2024-02-19T16:33:25.843 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-19T16:33:26.098 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:33:26.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:26 smithi008 ceph-mon[32573]: pgmap v329: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:33:26.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:26 smithi191 ceph-mon[37203]: pgmap v329: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:33:26.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:26 smithi156 ceph-mon[36502]: pgmap v329: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-19T16:33:28.190 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 395, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 380, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1656535, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 331449, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 22, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.051921737, 2024-02-19T16:33:28.191 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.011683097, 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.065219114, 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.004866832, 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 379, 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.051577884, 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000136089 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.192 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 54, 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000005969, 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000110 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1642896, 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 158784 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:28.193 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 70, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 237, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 18107, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 500055, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 13, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 6, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.015505845, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.006083190, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.006209441, 2024-02-19T16:33:28.194 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.001400425, 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 234, 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.020680059, 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000088376 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 20, 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000002583, 2024-02-19T16:33:28.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000129 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 449648 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 834, 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 864, 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 246675, 2024-02-19T16:33:28.196 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 649055, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 37, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.070325481, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.028848518, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.041512689, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.003497966, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 863, 2024-02-19T16:33:28.197 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.031845129, 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000036900 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1498, 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000257386, 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000171 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-19T16:33:28.198 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:28.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 146, 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.010949679, 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000074997 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:28.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 15, 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000496703, 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000033113 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 34, 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 21, 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-19T16:33:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 21, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.017361889, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000826756 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 13, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-19T16:33:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 11, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 27, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 27, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 26, 2024-02-19T16:33:28.203 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 1, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 2, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 2, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 3, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 52, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 27, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 9, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 4, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-19T16:33:28.204 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 1, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 4, 2024-02-19T16:33:28.205 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 60, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 1, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 3452, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 26460392, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 6, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 60, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-19T16:33:28.206 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 60, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 5, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-19T16:33:28.207 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 9, 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 9, 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 3, 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 6 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-19T16:33:28.208 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 5, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 2, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 2, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-02-19T16:33:28.209 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-02-19T16:33:28.210 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 35, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-19T16:33:28.211 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 35, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 2, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-02-19T16:33:28.212 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4240453, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 20, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.085500299, 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.004275014 2024-02-19T16:33:28.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 29, 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 31, 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 2, 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 21, 2024-02-19T16:33:28.214 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 22, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 1, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 27, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 30, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 3, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 2, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 16, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 14, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 53660, 2024-02-19T16:33:28.215 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 223612 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 43, 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 34, 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 24, 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 2, 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-19T16:33:28.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3, 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.006238348, 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002079449 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4, 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001813226, 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000453306 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.217 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 11, 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.004140718, 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000376428 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000109737, 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000109737 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.221 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-19T16:33:28.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.223 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.223 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.223 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.223 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-19T16:33:28.223 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.224 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.225 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.226 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-19T16:33:28.227 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.228 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.230 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.231 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.232 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.005059860, 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002529930 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-19T16:33:28.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 5, 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 4, 2024-02-19T16:33:28.234 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 13, 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 13, 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 628, 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-19T16:33:28.235 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:28.236 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:28.237 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:28.238 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:28.239 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 300138, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 136, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 264, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 3, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 118720, 2024-02-19T16:33:28.241 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 738, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-19T16:33:28.242 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 144, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 59399, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 33, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 144, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 144, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.513990918, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.003569381 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.243 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 144, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 176 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 144, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 61, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 83, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-19T16:33:28.244 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 10, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 1, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 21, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 13, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-19T16:33:28.245 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 16, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 60, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 20, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-19T16:33:28.246 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-02-19T16:33:28.247 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-19T16:33:28.248 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 106, 2024-02-19T16:33:28.249 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 15, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 12, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 12, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 30, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 5 2024-02-19T16:33:28.250 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-19T16:33:28.251 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 4, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 2, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 4, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 2, 2024-02-19T16:33:28.252 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 1295, 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1785398, 2024-02-19T16:33:28.253 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 1295, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 1295, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1785398, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.254 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:28.255 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 144, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 59617, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 144, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 59617, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.256 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:28.257 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 144, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 144, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 144, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 144, 2024-02-19T16:33:28.258 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:28.259 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 218, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 1, 2024-02-19T16:33:28.260 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 218, 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x56253292c6e0": { 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:28.261 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 35, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 46149, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 35, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 20, 2024-02-19T16:33:28.262 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 46149, 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:28.263 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-19T16:33:28.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:28 smithi008 ceph-mon[32573]: pgmap v330: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:33:28.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:28 smithi191 ceph-mon[37203]: pgmap v330: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:33:28.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:28 smithi156 ceph-mon[36502]: pgmap v330: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 99 B/s wr, 6 op/s 2024-02-19T16:33:29.055 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-02-19T16:33:29.055 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-02-19T16:33:29.055 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-02-19T16:33:29.055 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-19T16:33:29.056 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-19T16:33:29.056 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-02-19T16:33:29.056 DEBUG:tasks.check_counter:Getting stats from f 2024-02-19T16:33:29.056 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-19T16:33:29.377 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:33:30.758 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:30 smithi191 ceph-mon[37203]: pgmap v331: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:33:30.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:30 smithi008 ceph-mon[32573]: pgmap v331: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:33:30.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:30 smithi156 ceph-mon[36502]: pgmap v331: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-19T16:33:31.164 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-19T16:33:31.165 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:31.165 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1169, 2024-02-19T16:33:31.165 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1171, 2024-02-19T16:33:31.165 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 21343330, 2024-02-19T16:33:31.165 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 363569, 2024-02-19T16:33:31.165 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 7, 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.085784066, 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.022916050, 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.029953667, 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.025510634, 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:31.166 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1169, 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.031059118, 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000026568 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.167 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 90, 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 252, 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 25192889, 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 472126, 2024-02-19T16:33:31.168 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 8, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 6, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.027522617, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.005888517, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.018276867, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.001698853, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 250, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.015191953, 2024-02-19T16:33:31.169 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000060767 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 445936 2024-02-19T16:33:31.170 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1471, 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1453, 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 10495372, 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 511747, 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 5, 2024-02-19T16:33:31.171 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.108111728, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.030538857, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.054984260, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.029149315, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1452, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.037020150, 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000025495 2024-02-19T16:33:31.172 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1625504, 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 134192 2024-02-19T16:33:31.173 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:31.174 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4070, 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.079907722, 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000019633 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.175 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-19T16:33:31.176 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-19T16:33:31.177 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 9584, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 9584, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 9584, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-02-19T16:33:31.178 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 6, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-19T16:33:31.179 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-19T16:33:31.180 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 3452, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 26460392, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 6, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-19T16:33:31.181 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-02-19T16:33:31.182 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-19T16:33:31.183 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-02-19T16:33:31.184 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-19T16:33:31.185 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-19T16:33:31.186 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 24353, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-19T16:33:31.187 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 31, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 3, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 58893439, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 58893303, 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-19T16:33:31.188 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 24353 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 5284, 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 8514, 2024-02-19T16:33:31.189 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 3230, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 247, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 280, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 33, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 9584, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 9667, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 83, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-02-19T16:33:31.190 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 73820, 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215420 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-02-19T16:33:31.191 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-19T16:33:31.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.193 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.194 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-19T16:33:31.196 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.197 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-19T16:33:31.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.198 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.198 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.198 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-19T16:33:31.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.199 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.199 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.199 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-19T16:33:31.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.202 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-19T16:33:31.203 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.204 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.206 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.207 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-19T16:33:31.209 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 2, 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 7, 2024-02-19T16:33:31.210 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 5, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 4, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 7174, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-19T16:33:31.211 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:31.212 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:31.213 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:31.214 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:31.215 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 420370, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 5397, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:31.216 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 19091816, 2024-02-19T16:33:31.217 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 170140, 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-19T16:33:31.218 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2467, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 26, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 4, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2467, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2467, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.917966478, 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000777448 2024-02-19T16:33:31.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2467, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2468 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2467, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 2467, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-02-19T16:33:31.220 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 1583, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 881, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-19T16:33:31.221 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 2, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-19T16:33:31.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-02-19T16:33:31.223 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-19T16:33:31.224 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 106, 2024-02-19T16:33:31.225 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 15, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 9, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 9, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-02-19T16:33:31.226 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-19T16:33:31.227 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-19T16:33:31.228 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2726, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 56770100, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2726, 2024-02-19T16:33:31.229 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2726, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 56770100, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.230 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2467, 2024-02-19T16:33:31.231 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 54711583, 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2467, 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 54711583, 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.232 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:31.233 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2467, 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2467, 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2467, 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2467, 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.234 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:31.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x5624b9c9a960": { 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:31.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:31.238 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:31.239 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.239 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:31.239 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-19T16:33:31.922 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-19T16:33:31.923 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-19T16:33:31.923 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-02-19T16:33:31.923 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-02-19T16:33:31.923 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-02-19T16:33:31.923 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-02-19T16:33:31.923 DEBUG:tasks.check_counter:Getting stats from i 2024-02-19T16:33:31.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-19T16:33:32.186 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:33:32.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:32 smithi191 ceph-mon[37203]: pgmap v332: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:32.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:32 smithi008 ceph-mon[32573]: pgmap v332: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:32.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:32 smithi156 ceph-mon[36502]: pgmap v332: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:34.094 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-19T16:33:34.094 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:34.094 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 20790, 2024-02-19T16:33:34.094 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 20811, 2024-02-19T16:33:34.094 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 5443739, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 36324564, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 159, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 6, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 1.525528488, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.635332185, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.355383477, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.433359290, 2024-02-19T16:33:34.095 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 20812, 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.021552238, 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000049084 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2594, 2024-02-19T16:33:34.096 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000408777, 2024-02-19T16:33:34.097 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000157 2024-02-19T16:33:34.097 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.097 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 46576, 2024-02-19T16:33:34.097 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 2512 2024-02-19T16:33:34.097 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.098 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 18139, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 18220, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 5079854, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 16831894, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 15, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 1.192845853, 2024-02-19T16:33:34.099 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.434736979, 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.276092626, 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.398278019, 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 18221, 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.131554825, 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000062101 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.100 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1509, 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000253882, 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000168 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 45872, 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 460928 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.101 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 46191, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 64459, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 14570659, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 31676439, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 18, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 9, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 2.536121124, 2024-02-19T16:33:34.102 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 1.028330009, 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 1.584925947, 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.542241289, 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 64452, 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2.047078889, 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000031761 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 37705, 2024-02-19T16:33:34.104 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.009593508, 2024-02-19T16:33:34.104 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000254 2024-02-19T16:33:34.104 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.104 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 2039824, 2024-02-19T16:33:34.104 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 294960 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:34.105 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 34527, 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 20.361233129, 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000589719 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:34.106 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4744, 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.595281789, 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000336273 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 10775, 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 10723, 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-19T16:33:34.107 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 10723, 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 35.227071062, 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.003285188 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 52, 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-19T16:33:34.108 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 202, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 23, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 7054, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 247, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 6756, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 51, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 6699, 2024-02-19T16:33:34.109 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 47, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 3446, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 3446, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 12, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 11083, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 10476, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 52, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-19T16:33:34.110 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 1, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 5, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 37, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 1, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 2, 2024-02-19T16:33:34.111 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 52379, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 3452, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 26460392, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 6, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 52379, 2024-02-19T16:33:34.112 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 1192, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 19, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 1173, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 51187, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 5, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 157, 2024-02-19T16:33:34.113 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 10949, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 6, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 352, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 9461, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 8689, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 741, 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 3481 2024-02-19T16:33:34.114 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 10, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-19T16:33:34.115 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 3201, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 3199, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 2, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-19T16:33:34.116 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 19, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 5, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 23, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-19T16:33:34.117 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 24353, 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-19T16:33:34.118 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 24353, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 31, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 31, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 3, 2024-02-19T16:33:34.119 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 58893439, 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 491, 2024-02-19T16:33:34.120 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 14.650629301, 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.029838348 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 5286, 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 8535, 2024-02-19T16:33:34.121 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 3249, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 249, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 282, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 33, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 7054, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 10118, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 3064, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 3446, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 6689, 2024-02-19T16:33:34.122 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 3243, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 100392, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 231804 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 11099, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 43, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 10775, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 45, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 88, 2024-02-19T16:33:34.123 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6593, 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 22.977207682, 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.003485091 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 186, 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.263738273, 2024-02-19T16:33:34.124 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001417947 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-02-19T16:33:34.125 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.003597634, 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001798817 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 36, 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.018431045, 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000511973 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-19T16:33:34.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000115734, 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000115734 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.127 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.128 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-19T16:33:34.129 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 11, 2024-02-19T16:33:34.130 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001588161, 2024-02-19T16:33:34.130 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000144378 2024-02-19T16:33:34.130 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.130 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-19T16:33:34.130 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 84, 2024-02-19T16:33:34.130 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.411367950, 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.016801999 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-19T16:33:34.131 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 11, 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.057894448, 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.005263131 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 429, 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.092600665, 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000215852 2024-02-19T16:33:34.132 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 102, 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.601790020, 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.005899902 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 68, 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.144989004, 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002132191 2024-02-19T16:33:34.133 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-19T16:33:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 22, 2024-02-19T16:33:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.418691897, 2024-02-19T16:33:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.064485995 2024-02-19T16:33:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 5, 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.018651953, 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.003730390 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-19T16:33:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 8, 2024-02-19T16:33:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.038443983, 2024-02-19T16:33:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.004805497 2024-02-19T16:33:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-19T16:33:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-19T16:33:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 8, 2024-02-19T16:33:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.002577785, 2024-02-19T16:33:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000322223 2024-02-19T16:33:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-19T16:33:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 9, 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.021545773, 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002393974 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6, 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.004131298, 2024-02-19T16:33:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000688549 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3142, 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 8.149707757, 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002593796 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-19T16:33:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 2, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 7, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 5, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 2, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 59, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 29, 2024-02-19T16:33:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 645, 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:34.148 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:34.149 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:34.150 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:34.151 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 426607, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 3579, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 1496, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 17, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:34.152 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 18590193, 2024-02-19T16:33:34.153 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 178573, 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-19T16:33:34.154 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 60270, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 58560888, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 37, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 60270, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 60270, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 68.689796178, 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001139701 2024-02-19T16:33:34.155 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.156 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-19T16:33:34.156 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-19T16:33:34.156 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 60270, 2024-02-19T16:33:34.156 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 60562 2024-02-19T16:33:34.156 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.156 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 60270, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 52468, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 7802, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 11, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 70, 2024-02-19T16:33:34.157 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 542, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 52, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 7, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 6924, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-19T16:33:34.158 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 52398, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 22, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-19T16:33:34.159 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 6, 2024-02-19T16:33:34.160 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-19T16:33:34.161 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-19T16:33:34.162 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 106, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 21, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 12, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 12, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 495, 2024-02-19T16:33:34.163 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 35 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-19T16:33:34.164 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 17, 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 10, 2024-02-19T16:33:34.165 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 6383, 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 3199, 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:34.166 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 85116, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 17706130, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 85116, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 85116, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 17706130, 2024-02-19T16:33:34.167 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:34.168 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 60270, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 58909459, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 60270, 2024-02-19T16:33:34.169 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 58909459, 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-19T16:33:34.170 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 60270, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 60270, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 60270, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 60270, 2024-02-19T16:33:34.171 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:34.172 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 3199, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 348571, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 3199, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:34.173 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 51, 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 348571, 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:34.174 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55c91e6de6e0": { 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 24353, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 54699135, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 24353, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:34.175 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 491, 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 54699135, 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:34.177 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-19T16:33:34.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:34 smithi008 ceph-mon[32573]: pgmap v333: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:34.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:34 smithi191 ceph-mon[37203]: pgmap v333: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:34.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:34 smithi156 ceph-mon[36502]: pgmap v333: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:34.935 INFO:tasks.check_counter:Daemon mds.i mds.exported=5 2024-02-19T16:33:34.935 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-02-19T16:33:34.935 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-02-19T16:33:34.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2024-02-19T16:33:34.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=5 2024-02-19T16:33:34.935 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=23 2024-02-19T16:33:34.936 DEBUG:tasks.check_counter:Getting stats from l 2024-02-19T16:33:34.936 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee shell --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-19T16:33:35.187 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/mon.c/config 2024-02-19T16:33:36.678 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:36 smithi191 ceph-mon[37203]: pgmap v334: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:33:36.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:36 smithi008 ceph-mon[32573]: pgmap v334: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:33:36.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:36 smithi156 ceph-mon[36502]: pgmap v334: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:33:36.973 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-19T16:33:36.973 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 793, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 795, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 254314, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 246687, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.046420949, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.014424095, 2024-02-19T16:33:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.012510735, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.014365970, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 793, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.018176103, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000022920 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 713, 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 879, 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 177288, 2024-02-19T16:33:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 668122, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 7, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.052866893, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.016684397, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.012401092, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.018633941, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 877, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.027070065, 2024-02-19T16:33:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000030866 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448240 2024-02-19T16:33:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1036, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1018, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1887104, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 378177, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 3, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.091371918, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.025113751, 2024-02-19T16:33:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.050812524, 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.022533196, 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1017, 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.029598950, 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000029104 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-19T16:33:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1626704, 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 137152 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-19T16:33:36.981 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3723, 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.076625000, 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000020581 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-02-19T16:33:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-02-19T16:33:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 20, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 1, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 19, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 19, 2024-02-19T16:33:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 3, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-19T16:33:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-02-19T16:33:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-02-19T16:33:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-02-19T16:33:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-02-19T16:33:36.990 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-19T16:33:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-19T16:33:36.992 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 35, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-19T16:33:36.993 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4240453, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 4239879, 2024-02-19T16:33:36.994 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 35 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 20, 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 22, 2024-02-19T16:33:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 2, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 12, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 12, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 0, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 20, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 20, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 0, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-02-19T16:33:36.996 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 48128, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215420 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-02-19T16:33:36.997 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.998 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:36.999 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.000 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.001 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-19T16:33:37.002 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-19T16:33:37.003 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.004 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.004 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.004 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.004 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.005 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.006 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.007 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.008 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.009 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-19T16:33:37.010 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.011 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.012 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.013 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.014 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 5, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 4, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 3, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 5567, 2024-02-19T16:33:37.015 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-19T16:33:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-19T16:33:37.017 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-19T16:33:37.018 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-19T16:33:37.019 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 190709, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 66, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-19T16:33:37.020 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 81411, 2024-02-19T16:33:37.021 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 514, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-19T16:33:37.022 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2275, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 21, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 3, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2275, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2275, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.183979543, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000520430 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.023 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2275, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2276 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2275, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 2275, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-19T16:33:37.024 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 1480, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 793, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-02-19T16:33:37.025 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 1, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-19T16:33:37.026 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-19T16:33:37.027 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-19T16:33:37.028 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 106, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 15, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 5, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 5, 2024-02-19T16:33:37.029 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-19T16:33:37.030 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-02-19T16:33:37.031 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2538, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 2073921, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2538, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2538, 2024-02-19T16:33:37.033 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2073921, 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-19T16:33:37.034 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2275, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 46154, 2024-02-19T16:33:37.035 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2275, 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 46154, 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.036 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2275, 2024-02-19T16:33:37.037 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2275, 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2275, 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2275, 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.038 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:37.039 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.040 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x564e1ebf2960": { 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-19T16:33:37.041 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.042 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-19T16:33:37.043 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-19T16:33:37.805 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-02-19T16:33:37.805 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-02-19T16:33:37.805 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-02-19T16:33:37.805 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-02-19T16:33:37.805 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-02-19T16:33:37.805 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-02-19T16:33:37.806 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-19T16:33:37.816 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-19T16:33:37.816 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:33:37.816 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T16:33:37.838 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-02-19T16:33:37.838 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T16:33:37.896 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:33:38.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 -- 172.21.15.8:0/583449054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100fc0 msgr2=0x7f5568105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 --2- 172.21.15.8:0/583449054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100fc0 0x7f5568105000 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5558009930 tx=0x7f555802bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:38.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 -- 172.21.15.8:0/583449054 shutdown_connections 2024-02-19T16:33:38.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 --2- 172.21.15.8:0/583449054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55681056e0 0x7f556810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 --2- 172.21.15.8:0/583449054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100fc0 0x7f5568105000 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 --2- 172.21.15.8:0/583449054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5568100610 0x7f55681009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 -- 172.21.15.8:0/583449054 >> 172.21.15.8:0/583449054 conn(0x7f5568076770 msgr2=0x7f5568076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 -- 172.21.15.8:0/583449054 shutdown_connections 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.049+0000 7f556fa74640 1 -- 172.21.15.8:0/583449054 wait complete. 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 Processor -- start 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 -- start start 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 0x7f5568198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55681056e0 0x7f5568199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55681995d0 0x7f55681a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681a4220 con 0x7f5568100610 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681a4390 con 0x7f55681995d0 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556fa74640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681a4550 con 0x7f55681056e0 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 0x7f5568198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 0x7f5568198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60480/0 (socket says 172.21.15.8:60480) 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 -- 172.21.15.8:0/3455705246 learned_addr learned my addr 172.21.15.8:0/3455705246 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d00f640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55681056e0 0x7f5568199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556e011640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55681995d0 0x7f55681a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 -- 172.21.15.8:0/3455705246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55681056e0 msgr2=0x7f5568199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55681056e0 0x7f5568199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 -- 172.21.15.8:0/3455705246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55681995d0 msgr2=0x7f55681a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55681995d0 0x7f55681a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 -- 172.21.15.8:0/3455705246 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558009560 con 0x7f5568100610 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.050+0000 7f556d810640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 0x7f5568198b00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f556400ba50 tx=0x7f556400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.051+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400cbe0 con 0x7f5568100610 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.051+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f556400cd40 con 0x7f5568100610 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.051+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55681a4830 con 0x7f5568100610 2024-02-19T16:33:38.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.051+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640077d0 con 0x7f5568100610 2024-02-19T16:33:38.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.051+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55681a4cc0 con 0x7f5568100610 2024-02-19T16:33:38.058 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.052+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5568101ae0 con 0x7f5568100610 2024-02-19T16:33:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.053+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f5564014440 con 0x7f5568100610 2024-02-19T16:33:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.053+0000 7f555effd640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f55540a2230 0x7f55540a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.054+0000 7f556d00f640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f55540a2230 0x7f55540a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.054+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f55640cdcd0 con 0x7f5568100610 2024-02-19T16:33:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.054+0000 7f556d00f640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f55540a2230 0x7f55540a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5558000c00 tx=0x7f5558005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.058+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5564096720 con 0x7f5568100610 2024-02-19T16:33:38.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.245+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5568068730 con 0x7f5568100610 2024-02-19T16:33:38.249 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.247+0000 7f555effd640 1 -- 172.21.15.8:0/3455705246 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+25301 (secure 0 0 0) 0x7f5564099de0 con 0x7f5568100610 2024-02-19T16:33:38.249 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:33:38.250 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":106,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:33:16.478604+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/1905912286":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/543580195":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2140457421":"2024-02-20T16:22:52.539698+0000","172.21.15.8:6801/354789085":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/354789085":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2267597929":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/3384401098":"2024-02-20T16:22:52.539698+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:33:38.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f55540a2230 msgr2=0x7f55540a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f55540a2230 0x7f55540a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5558000c00 tx=0x7f5558005fd0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 msgr2=0x7f5568198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 0x7f5568198b00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f556400ba50 tx=0x7f556400bf10 comp rx=0 tx=0).stop 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 shutdown_connections 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55681995d0 0x7f55681a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f55540a2230 0x7f55540a46f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55681056e0 0x7f5568199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 --2- 172.21.15.8:0/3455705246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5568100610 0x7f5568198b00 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 >> 172.21.15.8:0/3455705246 conn(0x7f5568076770 msgr2=0x7f55680fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 shutdown_connections 2024-02-19T16:33:38.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.249+0000 7f556fa74640 1 -- 172.21.15.8:0/3455705246 wait complete. 2024-02-19T16:33:38.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T16:15:46.974010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-19T16:20:19.402493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-19T16:20:20.183329+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-19T16:20:23.990295+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-19T16:33:38.263 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T16:33:38.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 -- 172.21.15.8:0/2372709123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f810bb00 msgr2=0x7fb5f810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/2372709123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f810bb00 0x7fb5f810df30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009be0 tx=0x7fb5f002bf80 comp rx=0 tx=0).stop 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 -- 172.21.15.8:0/2372709123 shutdown_connections 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/2372709123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f810bb00 0x7fb5f810df30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/2372709123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 0x7fb5f810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/2372709123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f8102730 0x7fb5f8102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 -- 172.21.15.8:0/2372709123 >> 172.21.15.8:0/2372709123 conn(0x7fb5f8076770 msgr2=0x7fb5f8076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 -- 172.21.15.8:0/2372709123 shutdown_connections 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.366+0000 7fb5ffb25640 1 -- 172.21.15.8:0/2372709123 wait complete. 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 Processor -- start 2024-02-19T16:33:38.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 -- start start 2024-02-19T16:33:38.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f8102730 0x7fb5f819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 0x7fb5f819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 0x7fb5f8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8197fa0 con 0x7fb5f810bb00 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8198110 con 0x7fb5f8103050 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5ffb25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81982d0 con 0x7fb5f8102730 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5fe0c2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 0x7fb5f8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5fd0c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 0x7fb5f819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5fe0c2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 0x7fb5f8197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60504/0 (socket says 172.21.15.8:60504) 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5fe0c2640 1 -- 172.21.15.8:0/1725409701 learned_addr learned my addr 172.21.15.8:0/1725409701 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.367+0000 7fb5fd8c1640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f8102730 0x7fb5f819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fd0c0640 1 -- 172.21.15.8:0/1725409701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f8102730 msgr2=0x7fb5f819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fd0c0640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f8102730 0x7fb5f819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fd0c0640 1 -- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 msgr2=0x7fb5f8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fd0c0640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 0x7fb5f8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fd0c0640 1 -- 172.21.15.8:0/1725409701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009560 con 0x7fb5f8103050 2024-02-19T16:33:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fe0c2640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 0x7fb5f8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5fd0c0640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 0x7fb5f819dc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9f0 tx=0x7fb5e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800ccc0 con 0x7fb5f8103050 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0009840 con 0x7fb5f8103050 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e800ce20 con 0x7fb5f8103050 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8004a00 con 0x7fb5f8103050 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.368+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81a4610 con 0x7fb5f8103050 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.369+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f806a300 con 0x7fb5f8103050 2024-02-19T16:33:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.370+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fb5e80050b0 con 0x7fb5f8103050 2024-02-19T16:33:38.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.371+0000 7fb5eeffd640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5e40a2230 0x7fb5e40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.371+0000 7fb5fd8c1640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5e40a2230 0x7fb5e40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.371+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb5e8014070 con 0x7fb5f8103050 2024-02-19T16:33:38.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.371+0000 7fb5fd8c1640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5e40a2230 0x7fb5e40a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80ff680 tx=0x7fb5f4009260 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.374+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fb5e8096c30 con 0x7fb5f8103050 2024-02-19T16:33:38.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.557+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5f8198e30 con 0x7fb5f8103050 2024-02-19T16:33:38.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.558+0000 7fb5eeffd640 1 -- 172.21.15.8:0/1725409701 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5e809a2f0 con 0x7fb5f8103050 2024-02-19T16:33:38.560 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-02-19T16:33:38.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5e40a2230 msgr2=0x7fb5e40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5e40a2230 0x7fb5e40a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80ff680 tx=0x7fb5f4009260 comp rx=0 tx=0).stop 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 msgr2=0x7fb5f819dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 0x7fb5f819dc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9f0 tx=0x7fb5e800eeb0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 shutdown_connections 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5f810bb00 0x7fb5f8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f8103050 0x7fb5f819dc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fb5e40a2230 0x7fb5e40a46f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 --2- 172.21.15.8:0/1725409701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f8102730 0x7fb5f819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 >> 172.21.15.8:0/1725409701 conn(0x7fb5f8076770 msgr2=0x7fb5f80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 shutdown_connections 2024-02-19T16:33:38.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.560+0000 7fb5ffb25640 1 -- 172.21.15.8:0/1725409701 wait complete. 2024-02-19T16:33:38.573 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-19T16:33:38.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 -- 172.21.15.8:0/128533635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68281064b0 msgr2=0x7f6828106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 --2- 172.21.15.8:0/128533635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68281064b0 0x7f6828106890 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6824009930 tx=0x7f682402bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 -- 172.21.15.8:0/128533635 shutdown_connections 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 --2- 172.21.15.8:0/128533635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6828100dd0 0x7f682810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 --2- 172.21.15.8:0/128533635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6828100410 0x7f6828100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 --2- 172.21.15.8:0/128533635 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68281064b0 0x7f6828106890 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 -- 172.21.15.8:0/128533635 >> 172.21.15.8:0/128533635 conn(0x7f6828077550 msgr2=0x7f6828077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 -- 172.21.15.8:0/128533635 shutdown_connections 2024-02-19T16:33:38.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.677+0000 7f682ff0d640 1 -- 172.21.15.8:0/128533635 wait complete. 2024-02-19T16:33:38.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 Processor -- start 2024-02-19T16:33:38.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 -- start start 2024-02-19T16:33:38.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 0x7f682810dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6828100dd0 0x7f682810e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68281064b0 0x7f682810e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281a5f00 con 0x7f6828100410 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281a6070 con 0x7f68281064b0 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682ff0d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281a6230 con 0x7f6828100dd0 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 0x7f682810dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 0x7f682810dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60520/0 (socket says 172.21.15.8:60520) 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 -- 172.21.15.8:0/4147796978 learned_addr learned my addr 172.21.15.8:0/4147796978 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682d4a8640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6828100dd0 0x7f682810e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682e4aa640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68281064b0 0x7f682810e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 -- 172.21.15.8:0/4147796978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6828100dd0 msgr2=0x7f682810e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6828100dd0 0x7f682810e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 -- 172.21.15.8:0/4147796978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68281064b0 msgr2=0x7f682810e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68281064b0 0x7f682810e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 -- 172.21.15.8:0/4147796978 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820009840 con 0x7f6828100410 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f682dca9640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 0x7f682810dcb0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6824009230 tx=0x7f682402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.678+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824004350 con 0x7f6828100410 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.679+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68240044b0 con 0x7f6828100410 2024-02-19T16:33:38.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.679+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6824009560 con 0x7f6828100410 2024-02-19T16:33:38.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.679+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824003bf0 con 0x7f6828100410 2024-02-19T16:33:38.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.679+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68281a6880 con 0x7f6828100410 2024-02-19T16:33:38.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.680+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6828068730 con 0x7f6828100410 2024-02-19T16:33:38.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.681+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f6824003d50 con 0x7f6828100410 2024-02-19T16:33:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.681+0000 7f681effd640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f68140a22e0 0x7f68140a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.681+0000 7f682d4a8640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f68140a22e0 0x7f68140a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.681+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f68240f02a0 con 0x7f6828100410 2024-02-19T16:33:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.682+0000 7f682d4a8640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f68140a22e0 0x7f68140a47a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f682810f6f0 tx=0x7f6818005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.686+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f68240b8c40 con 0x7f6828100410 2024-02-19T16:33:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:38 smithi008 ceph-mon[32573]: pgmap v335: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:38.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:38 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3455705246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:38 smithi191 ceph-mon[37203]: pgmap v335: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:38.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:38 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3455705246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:38.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:38 smithi156 ceph-mon[36502]: pgmap v335: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:38.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:38 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3455705246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:38.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.868+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6828108980 con 0x7f6828100410 2024-02-19T16:33:38.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.869+0000 7f681effd640 1 -- 172.21.15.8:0/4147796978 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68240bc300 con 0x7f6828100410 2024-02-19T16:33:38.871 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:33:38.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f68140a22e0 msgr2=0x7f68140a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f68140a22e0 0x7f68140a47a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f682810f6f0 tx=0x7f6818005f90 comp rx=0 tx=0).stop 2024-02-19T16:33:38.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 msgr2=0x7f682810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 0x7f682810dcb0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6824009230 tx=0x7f682402c650 comp rx=0 tx=0).stop 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 shutdown_connections 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68281064b0 0x7f682810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f68140a22e0 0x7f68140a47a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6828100dd0 0x7f682810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 --2- 172.21.15.8:0/4147796978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6828100410 0x7f682810dcb0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.870+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 >> 172.21.15.8:0/4147796978 conn(0x7f6828077550 msgr2=0x7f68281030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.871+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 shutdown_connections 2024-02-19T16:33:38.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.871+0000 7f682ff0d640 1 -- 172.21.15.8:0/4147796978 wait complete. 2024-02-19T16:33:38.884 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 -- 172.21.15.8:0/2235558450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100fc0 msgr2=0x7f10bc105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 --2- 172.21.15.8:0/2235558450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100fc0 0x7f10bc105000 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009930 tx=0x7f10b802bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 -- 172.21.15.8:0/2235558450 shutdown_connections 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 --2- 172.21.15.8:0/2235558450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10bc1056e0 0x7f10bc10d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 --2- 172.21.15.8:0/2235558450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100fc0 0x7f10bc105000 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 --2- 172.21.15.8:0/2235558450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10bc100610 0x7f10bc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 -- 172.21.15.8:0/2235558450 >> 172.21.15.8:0/2235558450 conn(0x7f10bc076770 msgr2=0x7f10bc076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 -- 172.21.15.8:0/2235558450 shutdown_connections 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.987+0000 7f10c32d4640 1 -- 172.21.15.8:0/2235558450 wait complete. 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 Processor -- start 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 -- start start 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 0x7f10bc19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10bc1056e0 0x7f10bc19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10bc197a30 0x7f10bc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc198420 con 0x7f10bc100610 2024-02-19T16:33:38.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc198590 con 0x7f10bc197a30 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c32d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc198750 con 0x7f10bc1056e0 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 0x7f10bc19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 0x7f10bc19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60530/0 (socket says 172.21.15.8:60530) 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 -- 172.21.15.8:0/734494950 learned_addr learned my addr 172.21.15.8:0/734494950 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c086f640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10bc1056e0 0x7f10bc19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1871640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10bc197a30 0x7f10bc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 -- 172.21.15.8:0/734494950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10bc1056e0 msgr2=0x7f10bc19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10bc1056e0 0x7f10bc19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 -- 172.21.15.8:0/734494950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10bc197a30 msgr2=0x7f10bc197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10bc197a30 0x7f10bc197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.988+0000 7f10c1070640 1 -- 172.21.15.8:0/734494950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8009560 con 0x7f10bc100610 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.989+0000 7f10c1070640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 0x7f10bc19d880 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e9f0 tx=0x7f10ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.989+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac009720 con 0x7f10bc100610 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.989+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc1a4740 con 0x7f10bc100610 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.989+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10ac00cd80 con 0x7f10bc100610 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.989+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac010690 con 0x7f10bc100610 2024-02-19T16:33:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.989+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc1a4bd0 con 0x7f10bc100610 2024-02-19T16:33:38.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.990+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc101ae0 con 0x7f10bc100610 2024-02-19T16:33:38.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.991+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f10ac0046c0 con 0x7f10bc100610 2024-02-19T16:33:38.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.991+0000 7f10b27fc640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f10a80a2230 0x7f10a80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:38.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.992+0000 7f10c086f640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f10a80a2230 0x7f10a80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:38.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.992+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f10ac014070 con 0x7f10bc100610 2024-02-19T16:33:38.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.992+0000 7f10c086f640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f10a80a2230 0x7f10a80a46f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10b8000c00 tx=0x7f10b8005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:38.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:38.995+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f10ac0972e0 con 0x7f10bc100610 2024-02-19T16:33:39.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.179+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10bc068730 con 0x7f10bc100610 2024-02-19T16:33:39.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.179+0000 7f10b27fc640 1 -- 172.21.15.8:0/734494950 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10ac09a9a0 con 0x7f10bc100610 2024-02-19T16:33:39.182 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2024-02-19T16:33:39.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f10a80a2230 msgr2=0x7f10a80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f10a80a2230 0x7f10a80a46f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10b8000c00 tx=0x7f10b8005fd0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 msgr2=0x7f10bc19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 0x7f10bc19d880 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e9f0 tx=0x7f10ac00edf0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 shutdown_connections 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10bc197a30 0x7f10bc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f10a80a2230 0x7f10a80a46f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10bc1056e0 0x7f10bc19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 --2- 172.21.15.8:0/734494950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10bc100610 0x7f10bc19d880 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.181+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 >> 172.21.15.8:0/734494950 conn(0x7f10bc076770 msgr2=0x7f10bc1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:39.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.182+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 shutdown_connections 2024-02-19T16:33:39.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.182+0000 7f10c32d4640 1 -- 172.21.15.8:0/734494950 wait complete. 2024-02-19T16:33:39.195 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 -- 172.21.15.8:0/3402659472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0444100610 msgr2=0x7f04441009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 --2- 172.21.15.8:0/3402659472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0444100610 0x7f04441009f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0434009930 tx=0x7f043402bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 -- 172.21.15.8:0/3402659472 shutdown_connections 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 --2- 172.21.15.8:0/3402659472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04441056e0 0x7f044410d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 --2- 172.21.15.8:0/3402659472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100fc0 0x7f0444105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 --2- 172.21.15.8:0/3402659472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0444100610 0x7f04441009f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 -- 172.21.15.8:0/3402659472 >> 172.21.15.8:0/3402659472 conn(0x7f0444076770 msgr2=0x7f0444076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:39.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 -- 172.21.15.8:0/3402659472 shutdown_connections 2024-02-19T16:33:39.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 -- 172.21.15.8:0/3402659472 wait complete. 2024-02-19T16:33:39.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 Processor -- start 2024-02-19T16:33:39.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.298+0000 7f044a5c7640 1 -- start start 2024-02-19T16:33:39.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100610 0x7f044419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 0x7f044419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444197a20 0x7f0444197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444198410 con 0x7f04441056e0 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444198580 con 0x7f0444197a20 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444198740 con 0x7f0444100610 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 0x7f044419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 0x7f044419dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60548/0 (socket says 172.21.15.8:60548) 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 -- 172.21.15.8:0/2712608494 learned_addr learned my addr 172.21.15.8:0/2712608494 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f0443fff640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100610 0x7f044419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f0448b64640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444197a20 0x7f0444197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 -- 172.21.15.8:0/2712608494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100610 msgr2=0x7f044419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100610 0x7f044419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 -- 172.21.15.8:0/2712608494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444197a20 msgr2=0x7f0444197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444197a20 0x7f0444197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 -- 172.21.15.8:0/2712608494 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434009560 con 0x7f04441056e0 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f0443fff640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100610 0x7f044419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04437fe640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 0x7f044419dd60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f042c00e9c0 tx=0x7f042c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c009700 con 0x7f04441056e0 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.299+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04441a46e0 con 0x7f04441056e0 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.300+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f042c00cd80 con 0x7f04441056e0 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.300+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c010430 con 0x7f04441056e0 2024-02-19T16:33:39.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.300+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04441a4c10 con 0x7f04441056e0 2024-02-19T16:33:39.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.301+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0444101ae0 con 0x7f04441056e0 2024-02-19T16:33:39.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.302+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f042c005000 con 0x7f04441056e0 2024-02-19T16:33:39.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.302+0000 7f04417fa640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04300a2230 0x7f04300a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.302+0000 7f0443fff640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04300a2230 0x7f04300a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.302+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f042c014070 con 0x7f04441056e0 2024-02-19T16:33:39.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.303+0000 7f0443fff640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04300a2230 0x7f04300a46f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0434005ab0 tx=0x7f0434005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:39.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.306+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f042c097380 con 0x7f04441056e0 2024-02-19T16:33:39.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.487+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0444100fc0 con 0x7f04441056e0 2024-02-19T16:33:39.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.488+0000 7f04417fa640 1 -- 172.21.15.8:0/2712608494 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7f042c09aa40 con 0x7f04441056e0 2024-02-19T16:33:39.490 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-02-19T16:33:39.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.489+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04300a2230 msgr2=0x7f04300a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.489+0000 7f044a5c7640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04300a2230 0x7f04300a46f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0434005ab0 tx=0x7f0434005fd0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.489+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 msgr2=0x7f044419dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.489+0000 7f044a5c7640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 0x7f044419dd60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f042c00e9c0 tx=0x7f042c00ee80 comp rx=0 tx=0).stop 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.489+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 shutdown_connections 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0444197a20 0x7f0444197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04441056e0 0x7f044419dd60 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f04300a2230 0x7f04300a46f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 --2- 172.21.15.8:0/2712608494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0444100610 0x7f044419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 >> 172.21.15.8:0/2712608494 conn(0x7f0444076770 msgr2=0x7f04441043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 shutdown_connections 2024-02-19T16:33:39.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.490+0000 7f044a5c7640 1 -- 172.21.15.8:0/2712608494 wait complete. 2024-02-19T16:33:39.503 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 -- 172.21.15.8:0/3222113094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc100060 msgr2=0x7f81bc100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/3222113094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc100060 0x7f81bc100440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009930 tx=0x7f81ac02bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 -- 172.21.15.8:0/3222113094 shutdown_connections 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/3222113094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81bc106820 0x7f81bc10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/3222113094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81bc100a10 0x7f81bc106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/3222113094 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc100060 0x7f81bc100440 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 -- 172.21.15.8:0/3222113094 >> 172.21.15.8:0/3222113094 conn(0x7f81bc075650 msgr2=0x7f81bc075a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:39.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 -- 172.21.15.8:0/3222113094 shutdown_connections 2024-02-19T16:33:39.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.604+0000 7f81c1ecd640 1 -- 172.21.15.8:0/3222113094 wait complete. 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 Processor -- start 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 -- start start 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81bc100060 0x7f81bc19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81bc106820 0x7f81bc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 0x7f81bc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc198410 con 0x7f81bc197a20 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc198580 con 0x7f81bc106820 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81c1ecd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc198740 con 0x7f81bc100060 2024-02-19T16:33:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81bbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 0x7f81bc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81bc100060 0x7f81bc19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81bbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 0x7f81bc197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60576/0 (socket says 172.21.15.8:60576) 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81bbfff640 1 -- 172.21.15.8:0/2567706101 learned_addr learned my addr 172.21.15.8:0/2567706101 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.605+0000 7f81baffd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81bc106820 0x7f81bc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81bbfff640 1 -- 172.21.15.8:0/2567706101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81bc100060 msgr2=0x7f81bc19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81bbfff640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81bc100060 0x7f81bc19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81bbfff640 1 -- 172.21.15.8:0/2567706101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81bc106820 msgr2=0x7f81bc19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81bbfff640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81bc106820 0x7f81bc19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81bbfff640 1 -- 172.21.15.8:0/2567706101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac009560 con 0x7f81bc197a20 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81bbfff640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 0x7f81bc197ed0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f81b400eb10 tx=0x7f81b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4004b20 con 0x7f81bc197a20 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc1a4730 con 0x7f81bc197a20 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b4004c80 con 0x7f81bc197a20 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4010650 con 0x7f81bc197a20 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.606+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1a4c10 con 0x7f81bc197a20 2024-02-19T16:33:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.607+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc068730 con 0x7f81bc197a20 2024-02-19T16:33:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.608+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f81b4010830 con 0x7f81bc197a20 2024-02-19T16:33:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.609+0000 7f81b8ff9640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f81a40a2230 0x7f81a40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.609+0000 7f81bb7fe640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f81a40a2230 0x7f81a40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:39.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.609+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f81b4014070 con 0x7f81bc197a20 2024-02-19T16:33:39.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.609+0000 7f81bb7fe640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f81a40a2230 0x7f81a40a46f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005ab0 tx=0x7f81ac005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:39.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.612+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f81b4097470 con 0x7f81bc197a20 2024-02-19T16:33:39.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1725409701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4147796978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/734494950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:39 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2712608494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1725409701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4147796978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/734494950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:39 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2712608494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.808+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f81bc198f50 con 0x7f81bc197a20 2024-02-19T16:33:39.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.809+0000 7f81b8ff9640 1 -- 172.21.15.8:0/2567706101 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v106) v1 ==== 81+0+981 (secure 0 0 0) 0x7f81b409ab30 con 0x7f81bc197a20 2024-02-19T16:33:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1905912286 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/4121837085 2024-02-20T16:13:20.630755+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/543580195 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2746045673 2024-02-20T16:12:53.197791+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2140457421 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/354789085 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2946360420 2024-02-20T16:12:53.197791+0000 2024-02-19T16:33:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1092270301 2024-02-20T16:13:20.630755+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/354789085 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/414318988 2024-02-20T16:12:53.197791+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3125167978 2024-02-20T16:12:53.197791+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/886492473 2024-02-20T16:13:20.630755+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/886492473 2024-02-20T16:13:20.630755+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3227434625 2024-02-20T16:13:20.630755+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3125167978 2024-02-20T16:12:53.197791+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2267597929 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3384401098 2024-02-20T16:22:52.539698+0000 2024-02-19T16:33:39.814 INFO:teuthology.orchestra.run.smithi008.stderr:listed 17 entries 2024-02-19T16:33:39.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f81a40a2230 msgr2=0x7f81a40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f81a40a2230 0x7f81a40a46f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005ab0 tx=0x7f81ac005fd0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 msgr2=0x7f81bc197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 0x7f81bc197ed0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f81b400eb10 tx=0x7f81b400c2d0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 shutdown_connections 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81bc197a20 0x7f81bc197ed0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81bc106820 0x7f81bc19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f81a40a2230 0x7f81a40a46f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 --2- 172.21.15.8:0/2567706101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81bc100060 0x7f81bc19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:39.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 >> 172.21.15.8:0/2567706101 conn(0x7f81bc075650 msgr2=0x7f81bc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:39.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 shutdown_connections 2024-02-19T16:33:39.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:39.811+0000 7f81c1ecd640 1 -- 172.21.15.8:0/2567706101 wait complete. 2024-02-19T16:33:39.824 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-19T16:33:39.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:33:39.825 DEBUG:teuthology.orchestra.run.smithi008:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-19T16:33:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1725409701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4147796978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/734494950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:39 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2712608494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-19T16:33:39.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:33:39.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:33:39.970 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-19T16:33:39.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:33:39.985 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2024-02-19T16:33:40.040 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-19T16:33:40.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:33:40.041 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T16:33:40.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T16:33:40.109 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2024-02-19T16:33:40.219 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-19T16:33:40.232 INFO:tasks.cephadm:Teardown begin 2024-02-19T16:33:40.232 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-19T16:33:40.233 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:33:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 -- 172.21.15.8:0/1841603027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4101e60 msgr2=0x7f47d410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 --2- 172.21.15.8:0/1841603027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4101e60 0x7f47d410e320 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009ca0 tx=0x7f47cc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 -- 172.21.15.8:0/1841603027 shutdown_connections 2024-02-19T16:33:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 --2- 172.21.15.8:0/1841603027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4101e60 0x7f47d410e320 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 --2- 172.21.15.8:0/1841603027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d41014a0 0x7f47d4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 --2- 172.21.15.8:0/1841603027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d4107560 0x7f47d4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.334+0000 7f47db85c640 1 -- 172.21.15.8:0/1841603027 >> 172.21.15.8:0/1841603027 conn(0x7f47d40fcf90 msgr2=0x7f47d40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 -- 172.21.15.8:0/1841603027 shutdown_connections 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 -- 172.21.15.8:0/1841603027 wait complete. 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 Processor -- start 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 -- start start 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d41014a0 0x7f47d419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d4101e60 0x7f47d419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 0x7f47d4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4197eb0 con 0x7f47d4107560 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4198020 con 0x7f47d41014a0 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47db85c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41981e0 con 0x7f47d4101e60 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47d9df9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 0x7f47d4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47d9df9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 0x7f47d4197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60598/0 (socket says 172.21.15.8:60598) 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47d95f8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d41014a0 0x7f47d419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.335+0000 7f47d9df9640 1 -- 172.21.15.8:0/4280383665 learned_addr learned my addr 172.21.15.8:0/4280383665 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:40.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d8df7640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d4101e60 0x7f47d419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d9df9640 1 -- 172.21.15.8:0/4280383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d4101e60 msgr2=0x7f47d419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d9df9640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d4101e60 0x7f47d419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d9df9640 1 -- 172.21.15.8:0/4280383665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d41014a0 msgr2=0x7f47d419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d9df9640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d41014a0 0x7f47d419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d9df9640 1 -- 172.21.15.8:0/4280383665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009900 con 0x7f47d4107560 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d9df9640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 0x7f47d4197860 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f47cc005e30 tx=0x7f47cc035aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc035ea0 con 0x7f47d4107560 2024-02-19T16:33:40.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47cc003e50 con 0x7f47d4107560 2024-02-19T16:33:40.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d4198460 con 0x7f47d4107560 2024-02-19T16:33:40.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc004830 con 0x7f47d4107560 2024-02-19T16:33:40.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.336+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d41a4750 con 0x7f47d4107560 2024-02-19T16:33:40.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.337+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d4068730 con 0x7f47d4107560 2024-02-19T16:33:40.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.339+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f47cc03b040 con 0x7f47d4107560 2024-02-19T16:33:40.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.340+0000 7f47d27fc640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f47bc0a2230 0x7f47bc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.340+0000 7f47d95f8640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f47bc0a2230 0x7f47bc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.340+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f47cc0f10b0 con 0x7f47d4107560 2024-02-19T16:33:40.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.340+0000 7f47d95f8640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f47bc0a2230 0x7f47bc0a46f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f47d41989a0 tx=0x7f47c4005f90 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:40.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.343+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f47cc0b99f0 con 0x7f47d4107560 2024-02-19T16:33:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.566+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d4109950 con 0x7f47d4107560 2024-02-19T16:33:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.567+0000 7f47d27fc640 1 -- 172.21.15.8:0/4280383665 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f47cc0bd0b0 con 0x7f47d4107560 2024-02-19T16:33:40.570 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:33:40.570 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:33:40.570 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f47bc0a2230 msgr2=0x7f47bc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f47bc0a2230 0x7f47bc0a46f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f47d41989a0 tx=0x7f47c4005f90 comp rx=0 tx=0).stop 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 msgr2=0x7f47d4197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 0x7f47d4197860 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f47cc005e30 tx=0x7f47cc035aa0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 shutdown_connections 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47d4107560 0x7f47d4197860 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f47bc0a2230 0x7f47bc0a46f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d4101e60 0x7f47d419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 --2- 172.21.15.8:0/4280383665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d41014a0 0x7f47d419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 >> 172.21.15.8:0/4280383665 conn(0x7f47d40fcf90 msgr2=0x7f47d40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 shutdown_connections 2024-02-19T16:33:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.569+0000 7f47db85c640 1 -- 172.21.15.8:0/4280383665 wait complete. 2024-02-19T16:33:40.582 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T16:33:40.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 -- 172.21.15.8:0/2178375294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 msgr2=0x7fd4a010e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 --2- 172.21.15.8:0/2178375294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a010e420 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ca0 tx=0x7fd49802c0f0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 -- 172.21.15.8:0/2178375294 shutdown_connections 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 --2- 172.21.15.8:0/2178375294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a010e420 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 --2- 172.21.15.8:0/2178375294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a00ffe80 0x7fd4a0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 --2- 172.21.15.8:0/2178375294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4a0106580 0x7fd4a0106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 -- 172.21.15.8:0/2178375294 >> 172.21.15.8:0/2178375294 conn(0x7fd4a0075770 msgr2=0x7fd4a0075b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 -- 172.21.15.8:0/2178375294 shutdown_connections 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.683+0000 7fd4a7581640 1 -- 172.21.15.8:0/2178375294 wait complete. 2024-02-19T16:33:40.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 Processor -- start 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 -- start start 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4a00ffe80 0x7fd4a0101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a01015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a0106580 0x7fd4a0101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01a5e30 con 0x7fd4a0100840 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01a5fa0 con 0x7fd4a0106580 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a7581640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01a6160 con 0x7fd4a00ffe80 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a01015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a01015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60612/0 (socket says 172.21.15.8:60612) 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 -- 172.21.15.8:0/705484921 learned_addr learned my addr 172.21.15.8:0/705484921 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a5b1e640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a0106580 0x7fd4a0101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a531d640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4a00ffe80 0x7fd4a0101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 -- 172.21.15.8:0/705484921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4a00ffe80 msgr2=0x7fd4a0101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4a00ffe80 0x7fd4a0101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 -- 172.21.15.8:0/705484921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a0106580 msgr2=0x7fd4a0101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a0106580 0x7fd4a0101ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 -- 172.21.15.8:0/705484921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498009900 con 0x7fd4a0100840 2024-02-19T16:33:40.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.684+0000 7fd4a4b1c640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a01015a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd49000e9f0 tx=0x7fd49000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:40.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.685+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490009700 con 0x7fd4a0100840 2024-02-19T16:33:40.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.685+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a01a6440 con 0x7fd4a0100840 2024-02-19T16:33:40.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.685+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd49000cd80 con 0x7fd4a0100840 2024-02-19T16:33:40.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.685+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490018920 con 0x7fd4a0100840 2024-02-19T16:33:40.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.685+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a01a6950 con 0x7fd4a0100840 2024-02-19T16:33:40.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.686+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a0068730 con 0x7fd4a0100840 2024-02-19T16:33:40.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.687+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fd490018a80 con 0x7fd4a0100840 2024-02-19T16:33:40.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.687+0000 7fd4967fc640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd48c0a2260 0x7fd48c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:40.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.687+0000 7fd4a531d640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd48c0a2260 0x7fd48c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:40.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.688+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fd490014070 con 0x7fd4a0100840 2024-02-19T16:33:40.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.688+0000 7fd4a531d640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd48c0a2260 0x7fd48c0a4720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd49c005d00 tx=0x7fd49c00b410 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:40.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.692+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fd490053230 con 0x7fd4a0100840 2024-02-19T16:33:40.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:40 smithi008 ceph-mon[32573]: pgmap v336: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:33:40.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:40 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2567706101' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:33:40.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:40 smithi191 ceph-mon[37203]: pgmap v336: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:33:40.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:40 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2567706101' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:33:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:40 smithi156 ceph-mon[36502]: pgmap v336: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-19T16:33:40.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:40 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2567706101' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T16:33:40.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.915+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a0109a50 con 0x7fd4a0100840 2024-02-19T16:33:40.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.916+0000 7fd4967fc640 1 -- 172.21.15.8:0/705484921 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fd490097150 con 0x7fd4a0100840 2024-02-19T16:33:40.919 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:33:40.919 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/3977271875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":3977271875},{"type":"v1","addr":"172.21.15.8:6835","nonce":3977271875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1583103315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":1583103315},{"type":"v1","addr":"172.21.15.8:6837","nonce":1583103315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/1717311742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":1717311742},{"type":"v1","addr":"172.21.15.8:6839","nonce":1717311742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/1752496516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":1752496516},{"type":"v1","addr":"172.21.15.8:6841","nonce":1752496516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2706765134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2706765134},{"type":"v1","addr":"172.21.15.156:6833","nonce":2706765134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4229713412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4229713412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4229713412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-19T16:20:21.002558+0000","modified":"2024-02-19T16:27:03.990719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24502,"mds_1":24488,"mds_2":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2702528663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2702528663},{"type":"v1","addr":"172.21.15.191:6833","nonce":2702528663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/3993132637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3993132637},{"type":"v1","addr":"172.21.15.156:6837","nonce":3993132637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/562441805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":562441805},{"type":"v1","addr":"172.21.15.191:6835","nonce":562441805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6837/1523568014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1523568014},{"type":"v1","addr":"172.21.15.191:6837","nonce":1523568014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/3057541491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":3057541491},{"type":"v1","addr":"172.21.15.156:6839","nonce":3057541491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2229091821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2229091821},{"type":"v1","addr":"172.21.15.191:6839","nonce":2229091821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-19T16:33:40.919 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-02-19T16:33:40.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd48c0a2260 msgr2=0x7fd48c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd48c0a2260 0x7fd48c0a4720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd49c005d00 tx=0x7fd49c00b410 comp rx=0 tx=0).stop 2024-02-19T16:33:40.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 msgr2=0x7fd4a01015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:40.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a01015a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd49000e9f0 tx=0x7fd49000eeb0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 shutdown_connections 2024-02-19T16:33:40.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4a0106580 0x7fd4a0101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd4a0100840 0x7fd4a01015a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fd48c0a2260 0x7fd48c0a4720 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 --2- 172.21.15.8:0/705484921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4a00ffe80 0x7fd4a0101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:40.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 >> 172.21.15.8:0/705484921 conn(0x7fd4a0075770 msgr2=0x7fd4a00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:40.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 shutdown_connections 2024-02-19T16:33:40.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:40.918+0000 7fd4a7581640 1 -- 172.21.15.8:0/705484921 wait complete. 2024-02-19T16:33:40.932 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T16:33:41.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 -- 172.21.15.8:0/66683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46000ff9f0 msgr2=0x7f46000ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 --2- 172.21.15.8:0/66683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46000ff9f0 0x7f46000ffe70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009900 tx=0x7f45f002bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:41.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 -- 172.21.15.8:0/66683 shutdown_connections 2024-02-19T16:33:41.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 --2- 172.21.15.8:0/66683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46001003b0 0x7f460010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 --2- 172.21.15.8:0/66683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46000ff9f0 0x7f46000ffe70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 --2- 172.21.15.8:0/66683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4600107580 0x7f4600107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 -- 172.21.15.8:0/66683 >> 172.21.15.8:0/66683 conn(0x7f46000769b0 msgr2=0x7f46000fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:41.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 -- 172.21.15.8:0/66683 shutdown_connections 2024-02-19T16:33:41.038 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.032+0000 7f460736b640 1 -- 172.21.15.8:0/66683 wait complete. 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 Processor -- start 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 -- start start 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46000ff9f0 0x7f460019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 0x7f460019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4600107580 0x7f46001978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600197ef0 con 0x7f46001003b0 2024-02-19T16:33:41.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600198060 con 0x7f46000ff9f0 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f460736b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600198220 con 0x7f4600107580 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 0x7f460019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 0x7f460019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60634/0 (socket says 172.21.15.8:60634) 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 -- 172.21.15.8:0/3116482863 learned_addr learned my addr 172.21.15.8:0/3116482863 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4605107640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46000ff9f0 0x7f460019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4605908640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4600107580 0x7f46001978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.040 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 -- 172.21.15.8:0/3116482863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4600107580 msgr2=0x7f46001978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4600107580 0x7f46001978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 -- 172.21.15.8:0/3116482863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46000ff9f0 msgr2=0x7f460019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46000ff9f0 0x7f460019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 -- 172.21.15.8:0/3116482863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0009560 con 0x7f46001003b0 2024-02-19T16:33:41.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.033+0000 7f4604906640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 0x7f460019dc30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005940 tx=0x7f45f002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:41.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.034+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f00042d0 con 0x7f46001003b0 2024-02-19T16:33:41.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.034+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45f0004430 con 0x7f46001003b0 2024-02-19T16:33:41.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.034+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46001984a0 con 0x7f46001003b0 2024-02-19T16:33:41.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.034+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0003c50 con 0x7f46001003b0 2024-02-19T16:33:41.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.034+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46001a4790 con 0x7f46001003b0 2024-02-19T16:33:41.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.034+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4600068730 con 0x7f46001003b0 2024-02-19T16:33:41.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.036+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f45f00054f0 con 0x7f46001003b0 2024-02-19T16:33:41.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.036+0000 7f45f67fc640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f45ec0a2230 0x7f45ec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.036+0000 7f4605107640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f45ec0a2230 0x7f45ec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.036+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f45f00f0a50 con 0x7f46001003b0 2024-02-19T16:33:41.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.037+0000 7f4605107640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f45ec0a2230 0x7f45ec0a46f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46001989e0 tx=0x7f45f8009500 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:41.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.040+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f45f00b93f0 con 0x7f46001003b0 2024-02-19T16:33:41.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.222+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46001099d0 con 0x7f46001003b0 2024-02-19T16:33:41.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.223+0000 7f45f67fc640 1 -- 172.21.15.8:0/3116482863 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+25301 (secure 0 0 0) 0x7f45f00bcab0 con 0x7f46001003b0 2024-02-19T16:33:41.226 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-19T16:33:41.226 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":106,"fsid":"81b9a938-cf41-11ee-95bb-87774f69a715","created":"2024-02-19T16:12:25.223147+0000","modified":"2024-02-19T16:33:16.478604+0000","last_up_change":"2024-02-19T16:19:33.874739+0000","last_in_change":"2024-02-19T16:19:13.663928+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T16:15:46.974010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-19T16:20:19.402493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-19T16:20:20.183329+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-19T16:20:23.990295+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bb7652b6-3998-473c-bd04-422f89e95197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6803","nonce":94559077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6805","nonce":94559077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6809","nonce":94559077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":94559077},{"type":"v1","addr":"172.21.15.8:6807","nonce":94559077}]},"public_addr":"172.21.15.8:6803/94559077","cluster_addr":"172.21.15.8:6805/94559077","heartbeat_back_addr":"172.21.15.8:6809/94559077","heartbeat_front_addr":"172.21.15.8:6807/94559077","state":["exists","up"]},{"osd":1,"uuid":"eaca12a9-0ccf-4c84-b071-32648d5dcca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6801","nonce":295266432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6803","nonce":295266432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6807","nonce":295266432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":295266432},{"type":"v1","addr":"172.21.15.156:6805","nonce":295266432}]},"public_addr":"172.21.15.156:6801/295266432","cluster_addr":"172.21.15.156:6803/295266432","heartbeat_back_addr":"172.21.15.156:6807/295266432","heartbeat_front_addr":"172.21.15.156:6805/295266432","state":["exists","up"]},{"osd":2,"uuid":"c5bc2023-4102-4aec-a85d-4c8f175a3926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6801","nonce":3756928102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6803","nonce":3756928102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6807","nonce":3756928102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3756928102},{"type":"v1","addr":"172.21.15.191:6805","nonce":3756928102}]},"public_addr":"172.21.15.191:6801/3756928102","cluster_addr":"172.21.15.191:6803/3756928102","heartbeat_back_addr":"172.21.15.191:6807/3756928102","heartbeat_front_addr":"172.21.15.191:6805/3756928102","state":["exists","up"]},{"osd":3,"uuid":"c8fecdd8-fbd2-4e72-8607-dedc9f0ff018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6811","nonce":1491647664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6813","nonce":1491647664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6817","nonce":1491647664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":1491647664},{"type":"v1","addr":"172.21.15.8:6815","nonce":1491647664}]},"public_addr":"172.21.15.8:6811/1491647664","cluster_addr":"172.21.15.8:6813/1491647664","heartbeat_back_addr":"172.21.15.8:6817/1491647664","heartbeat_front_addr":"172.21.15.8:6815/1491647664","state":["exists","up"]},{"osd":4,"uuid":"356b4cc1-0311-4770-9e1f-02809310c1ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6809","nonce":3469489229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6811","nonce":3469489229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6815","nonce":3469489229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3469489229},{"type":"v1","addr":"172.21.15.156:6813","nonce":3469489229}]},"public_addr":"172.21.15.156:6809/3469489229","cluster_addr":"172.21.15.156:6811/3469489229","heartbeat_back_addr":"172.21.15.156:6815/3469489229","heartbeat_front_addr":"172.21.15.156:6813/3469489229","state":["exists","up"]},{"osd":5,"uuid":"ca5e23ef-499f-4cca-ac7a-d85f36ffbedb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6809","nonce":499228077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6811","nonce":499228077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6815","nonce":499228077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":499228077},{"type":"v1","addr":"172.21.15.191:6813","nonce":499228077}]},"public_addr":"172.21.15.191:6809/499228077","cluster_addr":"172.21.15.191:6811/499228077","heartbeat_back_addr":"172.21.15.191:6815/499228077","heartbeat_front_addr":"172.21.15.191:6813/499228077","state":["exists","up"]},{"osd":6,"uuid":"84d121f0-81a1-4702-bd47-4c34f40b5b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6819","nonce":2596853517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6821","nonce":2596853517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6825","nonce":2596853517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2596853517},{"type":"v1","addr":"172.21.15.8:6823","nonce":2596853517}]},"public_addr":"172.21.15.8:6819/2596853517","cluster_addr":"172.21.15.8:6821/2596853517","heartbeat_back_addr":"172.21.15.8:6825/2596853517","heartbeat_front_addr":"172.21.15.8:6823/2596853517","state":["exists","up"]},{"osd":7,"uuid":"20a26fbd-a93b-472b-a894-7693384be848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6817","nonce":3871421578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6819","nonce":3871421578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6823","nonce":3871421578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3871421578},{"type":"v1","addr":"172.21.15.156:6821","nonce":3871421578}]},"public_addr":"172.21.15.156:6817/3871421578","cluster_addr":"172.21.15.156:6819/3871421578","heartbeat_back_addr":"172.21.15.156:6823/3871421578","heartbeat_front_addr":"172.21.15.156:6821/3871421578","state":["exists","up"]},{"osd":8,"uuid":"0c6e14c6-7bb9-46da-9e3a-1581f86e1764","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6817","nonce":2037482563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6819","nonce":2037482563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6823","nonce":2037482563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2037482563},{"type":"v1","addr":"172.21.15.191:6821","nonce":2037482563}]},"public_addr":"172.21.15.191:6817/2037482563","cluster_addr":"172.21.15.191:6819/2037482563","heartbeat_back_addr":"172.21.15.191:6823/2037482563","heartbeat_front_addr":"172.21.15.191:6821/2037482563","state":["exists","up"]},{"osd":9,"uuid":"00fb3fac-7fc8-4d1f-8bae-dae54e114041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6827","nonce":1070903169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6829","nonce":1070903169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6833","nonce":1070903169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":1070903169},{"type":"v1","addr":"172.21.15.8:6831","nonce":1070903169}]},"public_addr":"172.21.15.8:6827/1070903169","cluster_addr":"172.21.15.8:6829/1070903169","heartbeat_back_addr":"172.21.15.8:6833/1070903169","heartbeat_front_addr":"172.21.15.8:6831/1070903169","state":["exists","up"]},{"osd":10,"uuid":"816f1015-93ed-443a-9ad8-e0291b9970cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6825","nonce":1977244529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6827","nonce":1977244529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6831","nonce":1977244529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1977244529},{"type":"v1","addr":"172.21.15.156:6829","nonce":1977244529}]},"public_addr":"172.21.15.156:6825/1977244529","cluster_addr":"172.21.15.156:6827/1977244529","heartbeat_back_addr":"172.21.15.156:6831/1977244529","heartbeat_front_addr":"172.21.15.156:6829/1977244529","state":["exists","up"]},{"osd":11,"uuid":"a1b3c59f-d69e-4a0a-a285-6dd5b20b738a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6825","nonce":759584656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6827","nonce":759584656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6831","nonce":759584656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":759584656},{"type":"v1","addr":"172.21.15.191:6829","nonce":759584656}]},"public_addr":"172.21.15.191:6825/759584656","cluster_addr":"172.21.15.191:6827/759584656","heartbeat_back_addr":"172.21.15.191:6831/759584656","heartbeat_front_addr":"172.21.15.191:6829/759584656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:14:55.065814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:18.683595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:15:42.587742+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:08.245486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:32.844978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:16:58.369540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:24.341291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:17:49.140420+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:14.011497+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:18:40.130888+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:06.176433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T16:19:32.207970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/1905912286":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/4121837085":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/543580195":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/2746045673":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2140457421":"2024-02-20T16:22:52.539698+0000","172.21.15.8:6801/354789085":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/2946360420":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/1092270301":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/354789085":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/414318988":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6800/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:6801/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6800/886492473":"2024-02-20T16:13:20.630755+0000","172.21.15.8:0/3227434625":"2024-02-20T16:13:20.630755+0000","172.21.15.8:6801/3125167978":"2024-02-20T16:12:53.197791+0000","172.21.15.8:0/2267597929":"2024-02-20T16:22:52.539698+0000","172.21.15.8:0/3384401098":"2024-02-20T16:22:52.539698+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T16:33:41.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f45ec0a2230 msgr2=0x7f45ec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f45ec0a2230 0x7f45ec0a46f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46001989e0 tx=0x7f45f8009500 comp rx=0 tx=0).stop 2024-02-19T16:33:41.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 msgr2=0x7f460019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 0x7f460019dc30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005940 tx=0x7f45f002c670 comp rx=0 tx=0).stop 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 shutdown_connections 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f45ec0a2230 0x7f45ec0a46f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4600107580 0x7f46001978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46001003b0 0x7f460019dc30 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 --2- 172.21.15.8:0/3116482863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46000ff9f0 0x7f460019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 >> 172.21.15.8:0/3116482863 conn(0x7f46000769b0 msgr2=0x7f46000fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 shutdown_connections 2024-02-19T16:33:41.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.225+0000 7f460736b640 1 -- 172.21.15.8:0/3116482863 wait complete. 2024-02-19T16:33:41.239 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-19T16:33:41.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 -- 172.21.15.8:0/4023336511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740ff9f0 msgr2=0x7f53740ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 --2- 172.21.15.8:0/4023336511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740ff9f0 0x7f53740ffe70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5368009930 tx=0x7f536802bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:41.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 -- 172.21.15.8:0/4023336511 shutdown_connections 2024-02-19T16:33:41.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 --2- 172.21.15.8:0/4023336511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53741003b0 0x7f537410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 --2- 172.21.15.8:0/4023336511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740ff9f0 0x7f53740ffe70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 --2- 172.21.15.8:0/4023336511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 0x7f5374107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 -- 172.21.15.8:0/4023336511 >> 172.21.15.8:0/4023336511 conn(0x7f53740769b0 msgr2=0x7f53740fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 -- 172.21.15.8:0/4023336511 shutdown_connections 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.340+0000 7f537880e640 1 -- 172.21.15.8:0/4023336511 wait complete. 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 Processor -- start 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 -- start start 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 0x7f5374100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.346 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741003b0 0x7f5374101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 0x7f5374101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53741a5e50 con 0x7f53740ff9f0 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53741a5fc0 con 0x7f5374107580 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537880e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53741a6180 con 0x7f53741003b0 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 0x7f5374100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 0x7f5374100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60640/0 (socket says 172.21.15.8:60640) 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 -- 172.21.15.8:0/1296029341 learned_addr learned my addr 172.21.15.8:0/1296029341 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:41.347 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f5372d9d640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 0x7f5374101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f5371d9b640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741003b0 0x7f5374101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 -- 172.21.15.8:0/1296029341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741003b0 msgr2=0x7f5374101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741003b0 0x7f5374101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 -- 172.21.15.8:0/1296029341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 msgr2=0x7f5374101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 0x7f5374101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f537259c640 1 -- 172.21.15.8:0/1296029341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009560 con 0x7f53740ff9f0 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f5371d9b640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741003b0 0x7f5374101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.341+0000 7f5372d9d640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 0x7f5374101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:33:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.342+0000 7f537259c640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 0x7f5374100bd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9f0 tx=0x7f536400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:41.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.342+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364009720 con 0x7f53740ff9f0 2024-02-19T16:33:41.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.342+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f536400cd80 con 0x7f53740ff9f0 2024-02-19T16:33:41.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.342+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53741a6460 con 0x7f53740ff9f0 2024-02-19T16:33:41.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.342+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364010690 con 0x7f53740ff9f0 2024-02-19T16:33:41.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.342+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53741a68f0 con 0x7f53740ff9f0 2024-02-19T16:33:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.343+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5374068730 con 0x7f53740ff9f0 2024-02-19T16:33:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.344+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f53640046c0 con 0x7f53740ff9f0 2024-02-19T16:33:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.344+0000 7f53637fe640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f535c0a2260 0x7f535c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.345+0000 7f5371d9b640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f535c0a2260 0x7f535c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.345+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f5364014070 con 0x7f53740ff9f0 2024-02-19T16:33:41.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.345+0000 7f5371d9b640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f535c0a2260 0x7f535c0a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5368006010 tx=0x7f53680047a0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:41.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.349+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f5364097150 con 0x7f53740ff9f0 2024-02-19T16:33:41.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.574+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f53741020b0 con 0x7f53740ff9f0 2024-02-19T16:33:41.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.575+0000 7f53637fe640 1 -- 172.21.15.8:0/1296029341 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f536409a810 con 0x7f53740ff9f0 2024-02-19T16:33:41.578 INFO:teuthology.orchestra.run.smithi008.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-19T16:33:41.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f535c0a2260 msgr2=0x7f535c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f535c0a2260 0x7f535c0a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5368006010 tx=0x7f53680047a0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 msgr2=0x7f5374100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 0x7f5374100bd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9f0 tx=0x7f536400edf0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 shutdown_connections 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5374107580 0x7f5374101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f535c0a2260 0x7f535c0a4720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741003b0 0x7f5374101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 --2- 172.21.15.8:0/1296029341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53740ff9f0 0x7f5374100bd0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.579+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 >> 172.21.15.8:0/1296029341 conn(0x7f53740769b0 msgr2=0x7f53740fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:41.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.580+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 shutdown_connections 2024-02-19T16:33:41.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.580+0000 7f537880e640 1 -- 172.21.15.8:0/1296029341 wait complete. 2024-02-19T16:33:41.593 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-19T16:33:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- 172.21.15.8:0/2149615941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c100fa0 msgr2=0x7f406c104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- 172.21.15.8:0/2149615941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c100fa0 0x7f406c104fe0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4060009930 tx=0x7f406002bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- 172.21.15.8:0/2149615941 shutdown_connections 2024-02-19T16:33:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- 172.21.15.8:0/2149615941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f406c1056c0 0x7f406c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- 172.21.15.8:0/2149615941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c100fa0 0x7f406c104fe0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- 172.21.15.8:0/2149615941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c1005f0 0x7f406c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- 172.21.15.8:0/2149615941 >> 172.21.15.8:0/2149615941 conn(0x7f406c0774c0 msgr2=0x7f406c0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- 172.21.15.8:0/2149615941 shutdown_connections 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- 172.21.15.8:0/2149615941 wait complete. 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 Processor -- start 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- start start 2024-02-19T16:33:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 0x7f406c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f406c1056c0 0x7f406c19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197a20 0x7f406c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c198410 con 0x7f406c1005f0 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c198580 con 0x7f406c197a20 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.699+0000 7f407408f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c198740 con 0x7f406c1056c0 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 0x7f406c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 0x7f406c19d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60652/0 (socket says 172.21.15.8:60652) 2024-02-19T16:33:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 -- 172.21.15.8:0/2922305862 learned_addr learned my addr 172.21.15.8:0/2922305862 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f407262c640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197a20 0x7f406c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f407162a640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f406c1056c0 0x7f406c19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 -- 172.21.15.8:0/2922305862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f406c1056c0 msgr2=0x7f406c19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f406c1056c0 0x7f406c19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 -- 172.21.15.8:0/2922305862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197a20 msgr2=0x7f406c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197a20 0x7f406c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 -- 172.21.15.8:0/2922305862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4060009560 con 0x7f406c1005f0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f4071e2b640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 0x7f406c19d870 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f406400e9f0 tx=0x7f406400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064009700 con 0x7f406c1005f0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f406400cd80 con 0x7f406c1005f0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c1a4730 con 0x7f406c1005f0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064018920 con 0x7f406c1005f0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.700+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c1a4c60 con 0x7f406c1005f0 2024-02-19T16:33:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.701+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c101ac0 con 0x7f406c1005f0 2024-02-19T16:33:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.702+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f4064018a80 con 0x7f406c1005f0 2024-02-19T16:33:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.703+0000 7f406affd640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f40540a22b0 0x7f40540a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.703+0000 7f407162a640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f40540a22b0 0x7f40540a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.703+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9717+0+0 (secure 0 0 0) 0x7f4064014070 con 0x7f406c1005f0 2024-02-19T16:33:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.703+0000 7f407162a640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f40540a22b0 0x7f40540a4770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4060000c00 tx=0x7f4060005fd0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.707+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f4064053230 con 0x7f406c1005f0 2024-02-19T16:33:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/4280383665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:33:41.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/705484921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:33:41.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/3116482863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:41.778 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f[67202]: ignoring --setuser ceph since I am not root 2024-02-19T16:33:41.778 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f[67202]: ignoring --setgroup ceph since I am not root 2024-02-19T16:33:41.778 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mds[67206]: -- 172.21.15.191:0/2759007033 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b8f0481a0 con 0x563b8f01e800 2024-02-19T16:33:41.778 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f[67202]: starting mds.f at 2024-02-19T16:33:41.779 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: ignoring --setuser ceph since I am not root 2024-02-19T16:33:41.779 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: ignoring --setgroup ceph since I am not root 2024-02-19T16:33:41.779 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mds[67846]: -- 172.21.15.191:0/2466862984 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591836cc1a0 con 0x5591836a3000 2024-02-19T16:33:41.779 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: starting mds.i at 2024-02-19T16:33:41.780 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: ignoring --setuser ceph since I am not root 2024-02-19T16:33:41.780 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: ignoring --setgroup ceph since I am not root 2024-02-19T16:33:41.780 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mds[66898]: -- 172.21.15.191:0/4146238304 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56157c0aa1a0 con 0x56157c080800 2024-02-19T16:33:41.780 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: starting mds.l at 2024-02-19T16:33:41.780 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: 2024-02-19T16:33:41.627+0000 7f41db9bf700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-02-19T16:33:41.781 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: ignoring --setuser ceph since I am not root 2024-02-19T16:33:41.781 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: ignoring --setgroup ceph since I am not root 2024-02-19T16:33:41.781 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-mds[67521]: -- 172.21.15.191:0/3118246816 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a460d5a1a0 con 0x55a460d31000 2024-02-19T16:33:41.781 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: starting mds.c at 2024-02-19T16:33:41.781 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: 2024-02-19T16:33:41.627+0000 7f248fbb9700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-02-19T16:33:41.781 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:33:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: 2024-02-19T16:33:41.627+0000 7f248fbb9700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-02-19T16:33:41.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/4280383665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:33:41.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/705484921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:33:41.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/3116482863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:41.831 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b[66973]: ignoring --setuser ceph since I am not root 2024-02-19T16:33:41.831 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b[66973]: ignoring --setgroup ceph since I am not root 2024-02-19T16:33:41.832 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-mds[66977]: -- 172.21.15.156:0/501810656 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6856b61a0 con 0x55b68568d000 2024-02-19T16:33:41.832 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b[66973]: starting mds.b at 2024-02-19T16:33:41.832 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h[66662]: ignoring --setuser ceph since I am not root 2024-02-19T16:33:41.832 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h[66662]: ignoring --setgroup ceph since I am not root 2024-02-19T16:33:41.832 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-mds[66666]: -- 172.21.15.156:0/3801536160 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b042a1e1a0 con 0x55b0429f5000 2024-02-19T16:33:41.832 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:33:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h[66662]: starting mds.h at 2024-02-19T16:33:41.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:41.929+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f406c198f50 con 0x7f406c1005f0 2024-02-19T16:33:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/4280383665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:33:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/705484921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T16:33:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:41 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/3116482863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T16:33:42.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a[32568]: 2024-02-19T16:33:41.574+0000 7f77fe49f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T16:33:42.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.578+0000 7f406affd640 1 -- 172.21.15.8:0/2922305862 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f40640970f0 con 0x7f406c1005f0 2024-02-19T16:33:42.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.581+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f40540a22b0 msgr2=0x7f40540a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:42.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.581+0000 7f407408f640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f40540a22b0 0x7f40540a4770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4060000c00 tx=0x7f4060005fd0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.581+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 msgr2=0x7f406c19d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:42.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.581+0000 7f407408f640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 0x7f406c19d870 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f406400e9f0 tx=0x7f406400eeb0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 shutdown_connections 2024-02-19T16:33:42.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c197a20 0x7f406c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f40540a22b0 0x7f40540a4770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f406c1056c0 0x7f406c19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 --2- 172.21.15.8:0/2922305862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f406c1005f0 0x7f406c19d870 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 >> 172.21.15.8:0/2922305862 conn(0x7f406c0774c0 msgr2=0x7f406c069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:42.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 shutdown_connections 2024-02-19T16:33:42.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.582+0000 7f407408f640 1 -- 172.21.15.8:0/2922305862 wait complete. 2024-02-19T16:33:42.599 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-19T16:33:42.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 -- 172.21.15.8:0/3970607689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4069a60 msgr2=0x7ff5e4069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:42.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 --2- 172.21.15.8:0/3970607689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4069a60 0x7ff5e4069ee0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009900 tx=0x7ff5d802bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 -- 172.21.15.8:0/3970607689 shutdown_connections 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 --2- 172.21.15.8:0/3970607689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e406a420 0x7ff5e410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 --2- 172.21.15.8:0/3970607689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4069a60 0x7ff5e4069ee0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 --2- 172.21.15.8:0/3970607689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e4107560 0x7ff5e4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 -- 172.21.15.8:0/3970607689 >> 172.21.15.8:0/3970607689 conn(0x7ff5e40773d0 msgr2=0x7ff5e40777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 -- 172.21.15.8:0/3970607689 shutdown_connections 2024-02-19T16:33:42.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.710+0000 7ff5ea823640 1 -- 172.21.15.8:0/3970607689 wait complete. 2024-02-19T16:33:42.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.711+0000 7ff5ea823640 1 Processor -- start 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.711+0000 7ff5ea823640 1 -- start start 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5ea823640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e4069a60 0x7ff5e419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5ea823640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e406a420 0x7ff5e419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5ea823640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 0x7ff5e4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5ea823640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4197eb0 con 0x7ff5e4107560 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5ea823640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4198020 con 0x7ff5e406a420 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5ea823640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e41981e0 con 0x7ff5e4069a60 2024-02-19T16:33:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 0x7ff5e4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 0x7ff5e4197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60684/0 (socket says 172.21.15.8:60684) 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 -- 172.21.15.8:0/2544722219 learned_addr learned my addr 172.21.15.8:0/2544722219 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e3fff640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e4069a60 0x7ff5e419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e37fe640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e406a420 0x7ff5e419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 -- 172.21.15.8:0/2544722219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e4069a60 msgr2=0x7ff5e419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e4069a60 0x7ff5e419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 -- 172.21.15.8:0/2544722219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e406a420 msgr2=0x7ff5e419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e406a420 0x7ff5e419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 -- 172.21.15.8:0/2544722219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d8009560 con 0x7ff5e4107560 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e3fff640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e4069a60 0x7ff5e419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e37fe640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e406a420 0x7ff5e419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e8dc0640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 0x7ff5e4197860 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00eaa0 tx=0x7ff5dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.712+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00cdb0 con 0x7ff5e4107560 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.713+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e41984c0 con 0x7ff5e4107560 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.713+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc005020 con 0x7ff5e4107560 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.713+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc0116b0 con 0x7ff5e4107560 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.713+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41a4750 con 0x7ff5e4107560 2024-02-19T16:33:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.714+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e4102c00 con 0x7ff5e4107560 2024-02-19T16:33:42.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.715+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7ff5dc005520 con 0x7ff5e4107560 2024-02-19T16:33:42.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.715+0000 7ff5e17fa640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5cc0a2230 0x7ff5cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:42.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.716+0000 7ff5e3fff640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5cc0a2230 0x7ff5cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:42.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.716+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9717+0+0 (secure 0 0 0) 0x7ff5dc0ce800 con 0x7ff5e4107560 2024-02-19T16:33:42.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.716+0000 7ff5e3fff640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5cc0a2230 0x7ff5cc0a46f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005fd0 tx=0x7ff5d4005850 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:42.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.720+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7ff5dc096fe0 con 0x7ff5e4107560 2024-02-19T16:33:42.777 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: pgmap v337: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:42.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1296029341' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-19T16:33:42.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T16:33:42.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T16:33:42.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1296029341' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-19T16:33:42.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: osdmap e107: 12 total, 12 up, 12 in 2024-02-19T16:33:42.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-19T16:33:42.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:42 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2922305862' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:33:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: pgmap v337: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1296029341' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-19T16:33:42.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T16:33:42.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T16:33:42.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1296029341' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-19T16:33:42.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: osdmap e107: 12 total, 12 up, 12 in 2024-02-19T16:33:42.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-19T16:33:42.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:42 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2922305862' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:33:42.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: pgmap v337: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-19T16:33:42.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1296029341' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-19T16:33:42.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T16:33:42.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T16:33:42.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1296029341' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-19T16:33:42.883 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: osdmap e107: 12 total, 12 up, 12 in 2024-02-19T16:33:42.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-19T16:33:42.884 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:42 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2922305862' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T16:33:42.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:42.905+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff5e41098b0 con 0x7ff5e4107560 2024-02-19T16:33:43.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.584+0000 7ff5e17fa640 1 -- 172.21.15.8:0/2544722219 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff5dc01f020 con 0x7ff5e4107560 2024-02-19T16:33:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_metadata' removed 2024-02-19T16:33:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.586+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5cc0a2230 msgr2=0x7ff5cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.586+0000 7ff5ea823640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5cc0a2230 0x7ff5cc0a46f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005fd0 tx=0x7ff5d4005850 comp rx=0 tx=0).stop 2024-02-19T16:33:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 msgr2=0x7ff5e4197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:43.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 0x7ff5e4197860 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00eaa0 tx=0x7ff5dc00ef60 comp rx=0 tx=0).stop 2024-02-19T16:33:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 shutdown_connections 2024-02-19T16:33:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4107560 0x7ff5e4197860 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e406a420 0x7ff5e419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7ff5cc0a2230 0x7ff5cc0a46f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 --2- 172.21.15.8:0/2544722219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5e4069a60 0x7ff5e419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 >> 172.21.15.8:0/2544722219 conn(0x7ff5e40773d0 msgr2=0x7ff5e40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 shutdown_connections 2024-02-19T16:33:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.587+0000 7ff5ea823640 1 -- 172.21.15.8:0/2544722219 wait complete. 2024-02-19T16:33:43.602 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-19T16:33:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 -- 172.21.15.8:0/3974346365 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041056a0 msgr2=0x7fce0410d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 --2- 172.21.15.8:0/3974346365 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041056a0 0x7fce0410d270 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009cd0 tx=0x7fcdfc02c0f0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 -- 172.21.15.8:0/3974346365 shutdown_connections 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 --2- 172.21.15.8:0/3974346365 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041056a0 0x7fce0410d270 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 --2- 172.21.15.8:0/3974346365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce04100f80 0x7fce04104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 --2- 172.21.15.8:0/3974346365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce041005d0 0x7fce041009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 -- 172.21.15.8:0/3974346365 >> 172.21.15.8:0/3974346365 conn(0x7fce040774f0 msgr2=0x7fce04077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 -- 172.21.15.8:0/3974346365 shutdown_connections 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.708+0000 7fce0bd83640 1 -- 172.21.15.8:0/3974346365 wait complete. 2024-02-19T16:33:43.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 Processor -- start 2024-02-19T16:33:43.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 -- start start 2024-02-19T16:33:43.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce041005d0 0x7fce0419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:43.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce041056a0 0x7fce0419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:43.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 0x7fce04196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:43.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce04197270 con 0x7fce041967c0 2024-02-19T16:33:43.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce041973e0 con 0x7fce041056a0 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0bd83640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce04197550 con 0x7fce041005d0 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0a320640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 0x7fce04196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0a320640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 0x7fce04196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60700/0 (socket says 172.21.15.8:60700) 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0a320640 1 -- 172.21.15.8:0/1230981150 learned_addr learned my addr 172.21.15.8:0/1230981150 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.709+0000 7fce0931e640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce041056a0 0x7fce0419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce09b1f640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce041005d0 0x7fce0419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0a320640 1 -- 172.21.15.8:0/1230981150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce041005d0 msgr2=0x7fce0419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:43.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0a320640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce041005d0 0x7fce0419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0a320640 1 -- 172.21.15.8:0/1230981150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce041056a0 msgr2=0x7fce0419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0a320640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce041056a0 0x7fce0419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0a320640 1 -- 172.21.15.8:0/1230981150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009900 con 0x7fce041967c0 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0931e640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce041056a0 0x7fce0419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0a320640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 0x7fce04196c70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc037040 tx=0x7fcdfc034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0328c0 con 0x7fce041967c0 2024-02-19T16:33:43.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc003e50 con 0x7fce041967c0 2024-02-19T16:33:43.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce04068650 con 0x7fce041967c0 2024-02-19T16:33:43.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004860 con 0x7fce041967c0 2024-02-19T16:33:43.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.710+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce04068b00 con 0x7fce041967c0 2024-02-19T16:33:43.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.711+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04101aa0 con 0x7fce041967c0 2024-02-19T16:33:43.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.712+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7fcdfc00a040 con 0x7fce041967c0 2024-02-19T16:33:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.713+0000 7fcdfaffd640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fcdf00a22b0 0x7fcdf00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.713+0000 7fce09b1f640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fcdf00a22b0 0x7fcdf00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.713+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9155+0+0 (secure 0 0 0) 0x7fcdfc0f2b40 con 0x7fce041967c0 2024-02-19T16:33:43.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.714+0000 7fce09b1f640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fcdf00a22b0 0x7fcdf00a4770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce00009890 tx=0x7fce00008040 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:43.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.717+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7fcdfc0bb600 con 0x7fce041967c0 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: Cluster is now healthy 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2922305862' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6833/696841610,v1:172.21.15.191:6832/696841610] up:boot 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6836/932855892,v1:172.21.15.156:6838/932855892] up:boot 2024-02-19T16:33:43.778 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.156:6837/1877220798,v1:172.21.15.156:6839/1877220798] up:boot 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6834/139361389,v1:172.21.15.191:6835/139361389] up:boot 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6836/239908262,v1:172.21.15.191:6837/239908262] up:boot 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: mds.? [v2:172.21.15.191:6838/2749354335,v1:172.21.15.191:6839/2749354335] up:boot 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: fsmap 12 up:standby 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:33:43.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:33:43.780 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:33:43.780 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:33:43.780 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:33:43.780 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:43 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2544722219' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:43.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: Cluster is now healthy 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2922305862' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6833/696841610,v1:172.21.15.191:6832/696841610] up:boot 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6836/932855892,v1:172.21.15.156:6838/932855892] up:boot 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.156:6837/1877220798,v1:172.21.15.156:6839/1877220798] up:boot 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6834/139361389,v1:172.21.15.191:6835/139361389] up:boot 2024-02-19T16:33:43.831 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6836/239908262,v1:172.21.15.191:6837/239908262] up:boot 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: mds.? [v2:172.21.15.191:6838/2749354335,v1:172.21.15.191:6839/2749354335] up:boot 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: fsmap 12 up:standby 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:33:43.832 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:33:43.833 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:33:43.833 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:43 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2544722219' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:43.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T16:33:43.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T16:33:43.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: Cluster is now healthy 2024-02-19T16:33:43.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2922305862' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-19T16:33:43.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6833/696841610,v1:172.21.15.191:6832/696841610] up:boot 2024-02-19T16:33:43.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6836/932855892,v1:172.21.15.156:6838/932855892] up:boot 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.156:6837/1877220798,v1:172.21.15.156:6839/1877220798] up:boot 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6834/139361389,v1:172.21.15.191:6835/139361389] up:boot 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6836/239908262,v1:172.21.15.191:6837/239908262] up:boot 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: mds.? [v2:172.21.15.191:6838/2749354335,v1:172.21.15.191:6839/2749354335] up:boot 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: fsmap 12 up:standby 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-19T16:33:43.848 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-19T16:33:43.849 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-19T16:33:43.849 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-19T16:33:43.849 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='mgr.15633 172.21.15.8:0/3843760173' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-19T16:33:43.849 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:43 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2544722219' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:43.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:43.900+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce040694e0 con 0x7fce041967c0 2024-02-19T16:33:44.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.591+0000 7fcdfaffd640 1 -- 172.21.15.8:0/1230981150 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcdfc0becc0 con 0x7fce041967c0 2024-02-19T16:33:44.595 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data' removed 2024-02-19T16:33:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fcdf00a22b0 msgr2=0x7fcdf00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fcdf00a22b0 0x7fcdf00a4770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce00009890 tx=0x7fce00008040 comp rx=0 tx=0).stop 2024-02-19T16:33:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 msgr2=0x7fce04196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 0x7fce04196c70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc037040 tx=0x7fcdfc034cc0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 shutdown_connections 2024-02-19T16:33:44.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fce041967c0 0x7fce04196c70 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce041056a0 0x7fce0419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7fcdf00a22b0 0x7fcdf00a4770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 --2- 172.21.15.8:0/1230981150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce041005d0 0x7fce0419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 >> 172.21.15.8:0/1230981150 conn(0x7fce040774f0 msgr2=0x7fce040fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:44.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.596+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 shutdown_connections 2024-02-19T16:33:44.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.597+0000 7fce0bd83640 1 -- 172.21.15.8:0/1230981150 wait complete. 2024-02-19T16:33:44.617 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-19T16:33:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 -- 172.21.15.8:0/3036533009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 msgr2=0x7f60d80ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 --2- 172.21.15.8:0/3036533009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d80ff9f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009900 tx=0x7f60d002bc30 comp rx=0 tx=0).stop 2024-02-19T16:33:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 -- 172.21.15.8:0/3036533009 shutdown_connections 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 --2- 172.21.15.8:0/3036533009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60d810e170 0x7f60d8110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 --2- 172.21.15.8:0/3036533009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d8102910 0x7f60d80fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 --2- 172.21.15.8:0/3036533009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d80ff9f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 -- 172.21.15.8:0/3036533009 >> 172.21.15.8:0/3036533009 conn(0x7f60d80769b0 msgr2=0x7f60d80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 -- 172.21.15.8:0/3036533009 shutdown_connections 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.726+0000 7f60e01d7640 1 -- 172.21.15.8:0/3036533009 wait complete. 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 Processor -- start 2024-02-19T16:33:44.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 -- start start 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d8102910 0x7f60d819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60d810e170 0x7f60d819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d819ec30 con 0x7f60d81024a0 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d819eda0 con 0x7f60d8102910 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60e01d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d819ef60 con 0x7f60d810e170 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d8199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60706/0 (socket says 172.21.15.8:60706) 2024-02-19T16:33:44.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 -- 172.21.15.8:0/236412369 learned_addr learned my addr 172.21.15.8:0/236412369 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 -- 172.21.15.8:0/236412369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60d810e170 msgr2=0x7f60d819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60d810e170 0x7f60d819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 -- 172.21.15.8:0/236412369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d8102910 msgr2=0x7f60d819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d8102910 0x7f60d819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.727+0000 7f60ddf73640 1 -- 172.21.15.8:0/236412369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d0009560 con 0x7f60d81024a0 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.728+0000 7f60ddf73640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d8199d70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005940 tx=0x7f60d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.728+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004440 con 0x7f60d81024a0 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.728+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d819f1e0 con 0x7f60d81024a0 2024-02-19T16:33:44.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.728+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d00052d0 con 0x7f60d81024a0 2024-02-19T16:33:44.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.728+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0003da0 con 0x7f60d81024a0 2024-02-19T16:33:44.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.728+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81a6ac0 con 0x7f60d81024a0 2024-02-19T16:33:44.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.729+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8068730 con 0x7f60d81024a0 2024-02-19T16:33:44.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.730+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 26) v1 ==== 136898+0+0 (secure 0 0 0) 0x7f60d0005440 con 0x7f60d81024a0 2024-02-19T16:33:44.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.731+0000 7f60ceffd640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f60c40a2230 0x7f60c40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T16:33:44.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.731+0000 7f60dd772640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f60c40a2230 0x7f60c40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T16:33:44.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.731+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8625+0+0 (secure 0 0 0) 0x7f60d00f0970 con 0x7f60d81024a0 2024-02-19T16:33:44.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.731+0000 7f60dd772640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f60c40a2230 0x7f60c40a46f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60d80fe710 tx=0x7f60c8005bf0 comp rx=0 tx=0).ready entity=mgr.15633 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T16:33:44.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.734+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184849 (secure 0 0 0) 0x7f60d00b9640 con 0x7f60d81024a0 2024-02-19T16:33:44.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:44.918+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60d8113940 con 0x7f60d81024a0 2024-02-19T16:33:45.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:44 smithi008 ceph-mon[32573]: pgmap v339: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-02-19T16:33:45.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/2544722219' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-19T16:33:45.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:44 smithi008 ceph-mon[32573]: osdmap e108: 12 total, 12 up, 12 in 2024-02-19T16:33:45.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:44 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1230981150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:45.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:44 smithi191 ceph-mon[37203]: pgmap v339: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-02-19T16:33:45.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/2544722219' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-19T16:33:45.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:44 smithi191 ceph-mon[37203]: osdmap e108: 12 total, 12 up, 12 in 2024-02-19T16:33:45.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:44 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1230981150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:44 smithi156 ceph-mon[36502]: pgmap v339: 129 pgs: 129 active+clean; 79 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-02-19T16:33:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/2544722219' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-19T16:33:45.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:44 smithi156 ceph-mon[36502]: osdmap e108: 12 total, 12 up, 12 in 2024-02-19T16:33:45.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:44 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1230981150' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:45.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.597+0000 7f60ceffd640 1 -- 172.21.15.8:0/236412369 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v110) v1 ==== 178+0+0 (secure 0 0 0) 0x7f60d00bcd00 con 0x7f60d81024a0 2024-02-19T16:33:45.600 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data_ec' removed 2024-02-19T16:33:45.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.601+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f60c40a2230 msgr2=0x7f60c40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:45.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.601+0000 7f60e01d7640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f60c40a2230 0x7f60c40a46f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60d80fe710 tx=0x7f60c8005bf0 comp rx=0 tx=0).stop 2024-02-19T16:33:45.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.601+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 msgr2=0x7f60d8199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.601+0000 7f60e01d7640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d8199d70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005940 tx=0x7f60d0004040 comp rx=0 tx=0).stop 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 shutdown_connections 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:6800/4279763273,v1:172.21.15.8:6801/4279763273] conn(0x7f60c40a2230 0x7f60c40a46f0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60d80fe710 tx=0x7f60c8005bf0 comp rx=0 tx=0).stop 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60d810e170 0x7f60d819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60d8102910 0x7f60d819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 --2- 172.21.15.8:0/236412369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60d81024a0 0x7f60d8199d70 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T16:33:45.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 >> 172.21.15.8:0/236412369 conn(0x7f60d80769b0 msgr2=0x7f60d810f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T16:33:45.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.602+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 shutdown_connections 2024-02-19T16:33:45.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-02-19T16:33:45.603+0000 7f60e01d7640 1 -- 172.21.15.8:0/236412369 wait complete. 2024-02-19T16:33:45.623 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:33:45.658 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:33:45.695 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:33:45.730 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-19T16:33:45.730 DEBUG:teuthology.orchestra.run.smithi008:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-19T16:33:45.746 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-19T16:33:45.762 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-19T16:33:45.785 INFO:tasks.cephadm:Stopping all daemons... 2024-02-19T16:33:45.786 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-19T16:33:45.786 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a 2024-02-19T16:33:46.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:45 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/1230981150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-19T16:33:46.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:45 smithi008 ceph-mon[32573]: osdmap e109: 12 total, 12 up, 12 in 2024-02-19T16:33:46.020 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:45 smithi008 ceph-mon[32573]: pgmap v342: 33 pgs: 33 active+clean; 26 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:33:46.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:45 smithi008 ceph-mon[32573]: from='client.? 172.21.15.8:0/236412369' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:46.021 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:45 smithi008 systemd[1]: Stopping Ceph mon.a for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:45 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/1230981150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-19T16:33:46.027 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:45 smithi191 ceph-mon[37203]: osdmap e109: 12 total, 12 up, 12 in 2024-02-19T16:33:46.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:45 smithi191 ceph-mon[37203]: pgmap v342: 33 pgs: 33 active+clean; 26 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:33:46.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:45 smithi191 ceph-mon[37203]: from='client.? 172.21.15.8:0/236412369' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:46.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:45 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/1230981150' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-19T16:33:46.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:45 smithi156 ceph-mon[36502]: osdmap e109: 12 total, 12 up, 12 in 2024-02-19T16:33:46.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:45 smithi156 ceph-mon[36502]: pgmap v342: 33 pgs: 33 active+clean; 26 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-19T16:33:46.081 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:45 smithi156 ceph-mon[36502]: from='client.? 172.21.15.8:0/236412369' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-19T16:33:46.520 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a[32568]: 2024-02-19T16:33:46.099+0000 7f7803caa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:33:46.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a[32568]: 2024-02-19T16:33:46.099+0000 7f7803caa700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-19T16:33:46.521 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:46 smithi008 podman[105671]: 2024-02-19 16:33:46.284324146 +0000 UTC m=+0.305695250 container died 92f87135603b00c0d0773d90476f5eee1b9b9f9b299f458507af3a5fc2b10655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-19T16:33:46.970 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:46 smithi008 podman[105671]: 2024-02-19 16:33:46.650734058 +0000 UTC m=+0.672105161 container cleanup 92f87135603b00c0d0773d90476f5eee1b9b9f9b299f458507af3a5fc2b10655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:33:46.970 INFO:journalctl@ceph.mon.a.smithi008.stdout:Feb 19 16:33:46 smithi008 bash[105671]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-a 2024-02-19T16:33:47.132 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.a.service' 2024-02-19T16:33:47.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:33:47.177 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-19T16:33:47.178 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-19T16:33:47.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.b 2024-02-19T16:33:47.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:47 smithi156 systemd[1]: Stopping Ceph mon.b for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:47.820 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-b[36498]: 2024-02-19T16:33:47.501+0000 7fa4cf307700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:33:47.820 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-b[36498]: 2024-02-19T16:33:47.501+0000 7fa4cf307700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-19T16:33:47.821 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:47 smithi156 podman[72272]: 2024-02-19 16:33:47.542822163 +0000 UTC m=+0.165339944 container died 134876071ffd34bf7c0c9d4d5b1060b36696ed2b19021b3c5cc53091dc521ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-b, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-19T16:33:48.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:47 smithi156 podman[72272]: 2024-02-19 16:33:47.927413452 +0000 UTC m=+0.549931231 container cleanup 134876071ffd34bf7c0c9d4d5b1060b36696ed2b19021b3c5cc53091dc521ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:33:48.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 19 16:33:47 smithi156 bash[72272]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-b 2024-02-19T16:33:48.351 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.b.service' 2024-02-19T16:33:48.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:33:48.395 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-19T16:33:48.395 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-19T16:33:48.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.c 2024-02-19T16:33:48.715 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:48 smithi191 systemd[1]: Stopping Ceph mon.c for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-c[37185]: 2024-02-19T16:33:48.713+0000 7f71d2075700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:33:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-c[37185]: 2024-02-19T16:33:48.713+0000 7f71d2075700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-19T16:33:49.028 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:48 smithi191 podman[73387]: 2024-02-19 16:33:48.775254351 +0000 UTC m=+0.186534245 container died 29df719b1c8917a7f9df89dbc077fba793c9f61031045d641a1479b461999ff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:33:49.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:49 smithi191 podman[73387]: 2024-02-19 16:33:49.156501448 +0000 UTC m=+0.567781359 container cleanup 29df719b1c8917a7f9df89dbc077fba793c9f61031045d641a1479b461999ff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=) 2024-02-19T16:33:49.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 19 16:33:49 smithi191 bash[73387]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mon-c 2024-02-19T16:33:49.527 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mon.c.service' 2024-02-19T16:33:49.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:33:49.591 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-19T16:33:49.592 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-19T16:33:49.592 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x 2024-02-19T16:33:49.915 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:33:49 smithi008 systemd[1]: Stopping Ceph mgr.x for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:50.249 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:33:49 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x[32803]: 2024-02-19T16:33:49.913+0000 7f05d3914700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:33:50.250 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:33:49 smithi008 podman[105794]: 2024-02-19 16:33:49.944834823 +0000 UTC m=+0.155005327 container died 8c5a5d54627cf92bf0b4dc70df1b9991c38bd6292ddfffbe3516ea09876bb0b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-19T16:33:50.520 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:33:50 smithi008 podman[105794]: 2024-02-19 16:33:50.348120452 +0000 UTC m=+0.558290958 container cleanup 8c5a5d54627cf92bf0b4dc70df1b9991c38bd6292ddfffbe3516ea09876bb0b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9f8908eed2b2d7e8c316bd5be7ed6e49024931ee, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-19T16:33:50.520 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Feb 19 16:33:50 smithi008 bash[105794]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-x 2024-02-19T16:33:50.770 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.x.service' 2024-02-19T16:33:50.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:33:50.808 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-19T16:33:50.808 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-19T16:33:50.808 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.y 2024-02-19T16:33:51.145 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:33:50 smithi156 systemd[1]: Stopping Ceph mgr.y for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:51.442 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:33:51 smithi156 podman[72397]: 2024-02-19 16:33:51.14418767 +0000 UTC m=+0.142892902 container died 71c335939279572bbc545b5fc8affb44f241c135b363fb93c4d2d99e3108ff66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-19T16:33:51.768 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:33:51 smithi156 podman[72397]: 2024-02-19 16:33:51.533280697 +0000 UTC m=+0.531985879 container cleanup 71c335939279572bbc545b5fc8affb44f241c135b363fb93c4d2d99e3108ff66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-19T16:33:51.768 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 19 16:33:51 smithi156 bash[72397]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-y 2024-02-19T16:33:51.942 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.y.service' 2024-02-19T16:33:51.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:33:51.980 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-19T16:33:51.980 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-19T16:33:51.980 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.z 2024-02-19T16:33:52.278 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:52 smithi191 systemd[1]: Stopping Ceph mgr.z for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:52.596 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:52 smithi191 podman[73506]: 2024-02-19 16:33:52.298287988 +0000 UTC m=+0.130866541 container died 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-19T16:33:53.028 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:52 smithi191 podman[73506]: 2024-02-19 16:33:52.828121826 +0000 UTC m=+0.660700372 container cleanup 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-02-19T16:33:53.028 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:52 smithi191 bash[73506]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z 2024-02-19T16:33:53.291 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.z.service' 2024-02-19T16:33:53.315 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:53 smithi191 podman[73520]: 2024-02-19 16:33:53.048020263 +0000 UTC m=+0.746608629 container remove 36a6644c1d973970dfc9a25d921518d4bc05ad0d647946d56414795c99df2151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mgr-z, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:33:53.315 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:53 smithi191 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-02-19T16:33:53.315 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:53 smithi191 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-02-19T16:33:53.315 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:53 smithi191 systemd[1]: Stopped Ceph mgr.z for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:33:53.315 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 19 16:33:53 smithi191 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mgr.z.service: Consumed 19.497s CPU time. 2024-02-19T16:33:53.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:33:53.328 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-19T16:33:53.328 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-19T16:33:53.328 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.0 2024-02-19T16:33:53.670 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:53 smithi008 systemd[1]: Stopping Ceph osd.0 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:33:54.020 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0[45163]: 2024-02-19T16:33:53.668+0000 7feb241e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:33:54.020 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0[45163]: 2024-02-19T16:33:53.668+0000 7feb241e3700 -1 osd.0 110 *** Got signal Terminated *** 2024-02-19T16:33:54.020 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0[45163]: 2024-02-19T16:33:53.668+0000 7feb241e3700 -1 osd.0 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:33:59.513 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:59 smithi008 podman[105914]: 2024-02-19 16:33:59.193204006 +0000 UTC m=+5.678566931 container died e3c2247787911c1fd5a846b4fb41a579901cc2a9f6d180e829c98bdf0a227f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-19T16:33:59.770 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:59 smithi008 podman[105914]: 2024-02-19 16:33:59.611253254 +0000 UTC m=+6.096616219 container cleanup e3c2247787911c1fd5a846b4fb41a579901cc2a9f6d180e829c98bdf0a227f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:33:59.770 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:59 smithi008 bash[105914]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0 2024-02-19T16:34:00.189 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:33:59 smithi008 podman[105929]: 2024-02-19 16:33:59.86302542 +0000 UTC m=+0.659998494 container remove e3c2247787911c1fd5a846b4fb41a579901cc2a9f6d180e829c98bdf0a227f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-19T16:34:00.492 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:34:00 smithi008 podman[106029]: 2024-02-19 16:34:00.091101977 +0000 UTC m=+0.018336914 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:00.493 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:34:00 smithi008 podman[106029]: 2024-02-19 16:34:00.230641723 +0000 UTC m=+0.157876694 container create 8ac4eb721443d78bb8c081b69974e84671ec0562f835555ace3074fed848129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T16:34:00.770 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:34:00 smithi008 podman[106029]: 2024-02-19 16:34:00.491795434 +0000 UTC m=+0.419030366 container init 8ac4eb721443d78bb8c081b69974e84671ec0562f835555ace3074fed848129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux ) 2024-02-19T16:34:00.770 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:34:00 smithi008 podman[106029]: 2024-02-19 16:34:00.494967107 +0000 UTC m=+0.422202037 container start 8ac4eb721443d78bb8c081b69974e84671ec0562f835555ace3074fed848129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-19T16:34:00.770 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:34:00 smithi008 podman[106029]: 2024-02-19 16:34:00.591703964 +0000 UTC m=+0.518938905 container attach 8ac4eb721443d78bb8c081b69974e84671ec0562f835555ace3074fed848129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:34:01.096 INFO:journalctl@ceph.osd.0.smithi008.stdout:Feb 19 16:34:00 smithi008 podman[106029]: 2024-02-19 16:34:00.789291121 +0000 UTC m=+0.716526051 container died 8ac4eb721443d78bb8c081b69974e84671ec0562f835555ace3074fed848129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-19T16:34:01.403 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.0.service' 2024-02-19T16:34:01.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:01.446 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-19T16:34:01.446 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-19T16:34:01.446 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.1 2024-02-19T16:34:01.755 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:01 smithi156 systemd[1]: Stopping Ceph osd.1 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:02.080 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1[41099]: 2024-02-19T16:34:01.753+0000 7f376df59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:02.080 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1[41099]: 2024-02-19T16:34:01.753+0000 7f376df59700 -1 osd.1 110 *** Got signal Terminated *** 2024-02-19T16:34:02.080 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1[41099]: 2024-02-19T16:34:01.753+0000 7f376df59700 -1 osd.1 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:07.388 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:07 smithi156 podman[72521]: 2024-02-19 16:34:07.110107279 +0000 UTC m=+5.476565659 container died aa06416c100aafb01481017e4b92655362b44464bf0ec6db26d62e5d7195f7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-19T16:34:07.781 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:07 smithi156 podman[72521]: 2024-02-19 16:34:07.494830855 +0000 UTC m=+5.861289257 container cleanup aa06416c100aafb01481017e4b92655362b44464bf0ec6db26d62e5d7195f7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-19T16:34:07.781 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:07 smithi156 bash[72521]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1 2024-02-19T16:34:08.080 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:07 smithi156 podman[72538]: 2024-02-19 16:34:07.779657555 +0000 UTC m=+0.666176752 container remove aa06416c100aafb01481017e4b92655362b44464bf0ec6db26d62e5d7195f7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:34:08.338 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:08 smithi156 podman[72632]: 2024-02-19 16:34:07.988729395 +0000 UTC m=+0.017380505 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:08.339 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:08 smithi156 podman[72632]: 2024-02-19 16:34:08.113892099 +0000 UTC m=+0.142543213 container create f69ac8cb6118d018f96442939efcc348d8a4e4376701e7e114f6ea4ec1b317f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:34:08.339 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:08 smithi156 podman[72632]: 2024-02-19 16:34:08.333351482 +0000 UTC m=+0.362002581 container init f69ac8cb6118d018f96442939efcc348d8a4e4376701e7e114f6ea4ec1b317f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1-deactivate, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-19T16:34:08.339 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:08 smithi156 podman[72632]: 2024-02-19 16:34:08.337828204 +0000 UTC m=+0.366479306 container start f69ac8cb6118d018f96442939efcc348d8a4e4376701e7e114f6ea4ec1b317f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-19T16:34:08.624 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:08 smithi156 podman[72632]: 2024-02-19 16:34:08.441816082 +0000 UTC m=+0.470467192 container attach f69ac8cb6118d018f96442939efcc348d8a4e4376701e7e114f6ea4ec1b317f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-02-19T16:34:08.987 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 19 16:34:08 smithi156 podman[72632]: 2024-02-19 16:34:08.623604832 +0000 UTC m=+0.652255926 container died f69ac8cb6118d018f96442939efcc348d8a4e4376701e7e114f6ea4ec1b317f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-1-deactivate, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T16:34:09.327 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.1.service' 2024-02-19T16:34:09.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:09.374 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-19T16:34:09.374 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-19T16:34:09.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.2 2024-02-19T16:34:09.683 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:09 smithi191 systemd[1]: Stopping Ceph osd.2 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:10.027 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2[42376]: 2024-02-19T16:34:09.681+0000 7fbb36f0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:10.027 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2[42376]: 2024-02-19T16:34:09.681+0000 7fbb36f0c700 -1 osd.2 110 *** Got signal Terminated *** 2024-02-19T16:34:10.028 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2[42376]: 2024-02-19T16:34:09.681+0000 7fbb36f0c700 -1 osd.2 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:15.426 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:15 smithi191 podman[73630]: 2024-02-19 16:34:15.158390721 +0000 UTC m=+5.592727951 container died f96940a104cd50008f2fb354fe8400a61ea379d9358680d574e3a965fdc45da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-19T16:34:15.777 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:15 smithi191 podman[73630]: 2024-02-19 16:34:15.523663985 +0000 UTC m=+5.958001210 container cleanup f96940a104cd50008f2fb354fe8400a61ea379d9358680d574e3a965fdc45da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-02-19T16:34:15.778 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:15 smithi191 bash[73630]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2 2024-02-19T16:34:16.132 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:15 smithi191 podman[73647]: 2024-02-19 16:34:15.817191738 +0000 UTC m=+0.661375257 container remove f96940a104cd50008f2fb354fe8400a61ea379d9358680d574e3a965fdc45da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-19T16:34:16.423 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:16 smithi191 podman[73737]: 2024-02-19 16:34:16.033966202 +0000 UTC m=+0.017815473 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:16.424 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:16 smithi191 podman[73737]: 2024-02-19 16:34:16.151823505 +0000 UTC m=+0.135672756 container create ce1b85b75ba6d99314d57a4e16fba1770ee5e52a5d20de4d77363dc6a603d2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:34:16.424 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:16 smithi191 podman[73737]: 2024-02-19 16:34:16.422708448 +0000 UTC m=+0.406557721 container init ce1b85b75ba6d99314d57a4e16fba1770ee5e52a5d20de4d77363dc6a603d2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:34:16.714 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:16 smithi191 podman[73737]: 2024-02-19 16:34:16.425868267 +0000 UTC m=+0.409717517 container start ce1b85b75ba6d99314d57a4e16fba1770ee5e52a5d20de4d77363dc6a603d2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:34:16.715 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:16 smithi191 podman[73737]: 2024-02-19 16:34:16.523255467 +0000 UTC m=+0.507104731 container attach ce1b85b75ba6d99314d57a4e16fba1770ee5e52a5d20de4d77363dc6a603d2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-19T16:34:17.027 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 19 16:34:16 smithi191 podman[73737]: 2024-02-19 16:34:16.713836303 +0000 UTC m=+0.697685569 container died ce1b85b75ba6d99314d57a4e16fba1770ee5e52a5d20de4d77363dc6a603d2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-19T16:34:17.404 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.2.service' 2024-02-19T16:34:17.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:17.445 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-19T16:34:17.445 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-19T16:34:17.445 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.3 2024-02-19T16:34:17.757 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:17 smithi008 systemd[1]: Stopping Ceph osd.3 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:18.020 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3[52114]: 2024-02-19T16:34:17.755+0000 7f841d914700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:18.020 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3[52114]: 2024-02-19T16:34:17.755+0000 7f841d914700 -1 osd.3 110 *** Got signal Terminated *** 2024-02-19T16:34:18.020 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:17 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3[52114]: 2024-02-19T16:34:17.755+0000 7f841d914700 -1 osd.3 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:20.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:19.927+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:21.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:20.879+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:22.183 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:21 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:21.801+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:22.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:21.912+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:22.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:22 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:22.060+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:22.520 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:22 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3[52114]: 2024-02-19T16:34:22.180+0000 7f8419f08700 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.423404+0000 front 2024-02-19T16:33:55.423349+0000 (oldest deadline 2024-02-19T16:34:21.323452+0000) 2024-02-19T16:34:23.184 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:22 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:22.819+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:23.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:22.878+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:23.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:23 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:23.058+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:23.491 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:23 smithi008 podman[106133]: 2024-02-19 16:34:23.183415297 +0000 UTC m=+5.541647678 container died f14561727a20b9737e8e65941d46a027aba9b21ce110028b88995557dcfe635b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-19T16:34:23.770 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:23 smithi008 podman[106133]: 2024-02-19 16:34:23.598104552 +0000 UTC m=+5.956336941 container cleanup f14561727a20b9737e8e65941d46a027aba9b21ce110028b88995557dcfe635b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True) 2024-02-19T16:34:23.770 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:23 smithi008 bash[106133]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3 2024-02-19T16:34:24.270 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:23 smithi008 podman[106162]: 2024-02-19 16:34:23.858007843 +0000 UTC m=+0.671221471 container remove f14561727a20b9737e8e65941d46a027aba9b21ce110028b88995557dcfe635b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:34:24.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:23 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:23.822+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:24.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:23.889+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:24.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:24 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:24.071+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:24.529 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:24 smithi008 podman[106252]: 2024-02-19 16:34:24.175494439 +0000 UTC m=+0.018413957 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:24.530 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:24 smithi008 podman[106252]: 2024-02-19 16:34:24.281116072 +0000 UTC m=+0.124035588 container create e264b7a0d24b83827210eb3b990aafef78599a5bc4de4262eac31d992de89f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-19T16:34:24.815 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:24 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:24.813+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:24.816 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:24 smithi008 podman[106252]: 2024-02-19 16:34:24.528733653 +0000 UTC m=+0.371653175 container init e264b7a0d24b83827210eb3b990aafef78599a5bc4de4262eac31d992de89f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-19T16:34:24.816 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:24 smithi008 podman[106252]: 2024-02-19 16:34:24.532270726 +0000 UTC m=+0.375190240 container start e264b7a0d24b83827210eb3b990aafef78599a5bc4de4262eac31d992de89f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3-deactivate, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-19T16:34:24.816 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:24 smithi008 podman[106252]: 2024-02-19 16:34:24.611778641 +0000 UTC m=+0.454698169 container attach e264b7a0d24b83827210eb3b990aafef78599a5bc4de4262eac31d992de89f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T16:34:25.123 INFO:journalctl@ceph.osd.3.smithi008.stdout:Feb 19 16:34:24 smithi008 podman[106252]: 2024-02-19 16:34:24.828280622 +0000 UTC m=+0.671200136 container died e264b7a0d24b83827210eb3b990aafef78599a5bc4de4262eac31d992de89f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-19T16:34:25.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:24.932+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:25.463 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.3.service' 2024-02-19T16:34:25.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:25.508 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-19T16:34:25.508 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-19T16:34:25.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.4 2024-02-19T16:34:25.537 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:25 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:25.119+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:25.808 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:25 smithi156 systemd[1]: Stopping Ceph osd.4 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:25.907 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:25.647+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:26.080 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:25 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:34:25.805+0000 7fc2f34ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:26.080 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:25 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:34:25.805+0000 7fc2f34ee700 -1 osd.4 110 *** Got signal Terminated *** 2024-02-19T16:34:26.080 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:25 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:34:25.805+0000 7fc2f34ee700 -1 osd.4 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:26.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:25 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:25.794+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:26.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:25.906+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:26.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:26 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:26.148+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:27.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:26 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:26.755+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:27.027 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:26.884+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:27.027 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:26.648+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:27.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:27 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:27.126+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:28.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:27 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:27.746+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:28.020 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:27 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:27.868+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:28.027 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:27 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:27.651+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:28.028 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:27 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:27.868+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:28.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:27 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:27.888+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:28.331 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:27 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:34:27.877+0000 7fc2ebada700 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.114847+0000 front 2024-02-19T16:34:05.114787+0000 (oldest deadline 2024-02-19T16:34:27.414982+0000) 2024-02-19T16:34:28.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:28 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:28.080+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:28.603 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:28 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:28.313+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:28.873 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:28 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:28.601+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:29.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:28 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:28.905+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:29.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:28 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:28.796+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:29.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:28 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:28.871+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:29.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:28 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:28.918+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:29.330 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:28 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:34:28.881+0000 7fc2ebada700 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.114847+0000 front 2024-02-19T16:34:05.114787+0000 (oldest deadline 2024-02-19T16:34:27.414982+0000) 2024-02-19T16:34:29.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:29 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:29.099+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:29.778 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:29 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:29.335+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:29.778 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:29 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:29.585+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:30.138 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:29 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:29.882+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:30.138 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:29 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4[47626]: 2024-02-19T16:34:29.867+0000 7fc2ebada700 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.114847+0000 front 2024-02-19T16:34:05.114787+0000 (oldest deadline 2024-02-19T16:34:27.414982+0000) 2024-02-19T16:34:30.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:29 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:29.874+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:30.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:29 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:29.805+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:30.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:29 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:29.882+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:30.564 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:30 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:30.305+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:30.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:30 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:30.135+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:30.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:30 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:30.135+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:30.863 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:30 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:30.562+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:30.863 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:30 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:30.562+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:31.145 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:30 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:30.880+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:31.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:30 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:30.904+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:31.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:30 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:30.815+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:31.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:30 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:30.861+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:31.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:30 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:30.861+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:31.569 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:31.296+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:31.580 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:31 smithi156 podman[72749]: 2024-02-19 16:34:31.333947384 +0000 UTC m=+5.635932776 container died 09cf8c743316311d1b295c86efaf9a31ac7b0beca6ec130a8d0006a1e6176d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-19T16:34:31.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:31 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:31.143+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:31.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:31 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:31.143+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:31.868 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:31 smithi156 podman[72749]: 2024-02-19 16:34:31.698028527 +0000 UTC m=+6.000013910 container cleanup 09cf8c743316311d1b295c86efaf9a31ac7b0beca6ec130a8d0006a1e6176d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-19T16:34:31.868 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:31 smithi156 bash[72749]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4 2024-02-19T16:34:31.911 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:31.568+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:31.912 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:31.568+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:32.263 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:31.910+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:32.263 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:31 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:31.910+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:32.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:31 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:31.932+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:32.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:31 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:31.827+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:32.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:31 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:31.827+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:32.301 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:31 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:31.865+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:32.301 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:31 smithi156 podman[72768]: 2024-02-19 16:34:31.95766388 +0000 UTC m=+0.623860492 container remove 09cf8c743316311d1b295c86efaf9a31ac7b0beca6ec130a8d0006a1e6176d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:34:32.301 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:32 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:32.100+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:32.302 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:32 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:32.100+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:32.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:32.262+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:32.580 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:32 smithi156 podman[72860]: 2024-02-19 16:34:32.300132943 +0000 UTC m=+0.101725249 container create 14062a43c1d1d7fe916daf44f400fee3c756af7563fd5e0f05ec3df3b2293295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-19T16:34:32.580 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:32 smithi156 podman[72860]: 2024-02-19 16:34:32.216110583 +0000 UTC m=+0.017702867 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:32.580 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:32 smithi156 podman[72860]: 2024-02-19 16:34:32.511764054 +0000 UTC m=+0.313356329 container init 14062a43c1d1d7fe916daf44f400fee3c756af7563fd5e0f05ec3df3b2293295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-02-19T16:34:32.581 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:32 smithi156 podman[72860]: 2024-02-19 16:34:32.515110412 +0000 UTC m=+0.316702686 container start 14062a43c1d1d7fe916daf44f400fee3c756af7563fd5e0f05ec3df3b2293295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4-deactivate, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-19T16:34:32.847 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:32 smithi156 podman[72860]: 2024-02-19 16:34:32.594286178 +0000 UTC m=+0.395878454 container attach 14062a43c1d1d7fe916daf44f400fee3c756af7563fd5e0f05ec3df3b2293295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS) 2024-02-19T16:34:32.847 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 19 16:34:32 smithi156 podman[72860]: 2024-02-19 16:34:32.803239779 +0000 UTC m=+0.604832045 container died 14062a43c1d1d7fe916daf44f400fee3c756af7563fd5e0f05ec3df3b2293295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:34:32.951 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:32.540+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:32.952 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:32.541+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:33.134 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:32 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:32.843+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:33.245 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:32.950+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:33.245 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:32 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:32.950+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:33.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:32 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:32.927+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:33.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:32 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:32.819+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:33.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:32 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:32.819+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:33.474 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:33 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:33.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:33.475 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:33 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:33.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:33.488 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.4.service' 2024-02-19T16:34:33.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:33.243+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:33.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:33.568 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-19T16:34:33.568 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-19T16:34:33.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.5 2024-02-19T16:34:33.880 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:33.542+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:33.881 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:33.542+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:33.881 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:33 smithi191 systemd[1]: Stopping Ceph osd.5 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:34.183 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:33 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:33.846+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:34.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:33 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:33.910+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:34.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:33 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:33.802+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:34.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:33 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:33.802+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:34.271 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:33.999+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:34.271 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:34 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:33.999+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:34.271 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:33.878+0000 7f0d8a5e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:34.271 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:33.878+0000 7f0d8a5e3700 -1 osd.5 110 *** Got signal Terminated *** 2024-02-19T16:34:34.272 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:33 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:33.878+0000 7f0d8a5e3700 -1 osd.5 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:34.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:34 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:34.269+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:34.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:34.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:34.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:34.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:35.003 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:34 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:34.568+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:35.003 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:34 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:34.568+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:35.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:34 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:34.815+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:35.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:34.905+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:35.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:34.816+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:35.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:34 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:34.816+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:35.277 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:35.238+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:35.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:35.001+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:35.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:35.001+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:35.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:35.225+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:35.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:35.225+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:35.959 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:35.582+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:35.959 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:35.582+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:36.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:35 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:35.769+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:36.249 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:35.957+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:36.250 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:35 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:35.957+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:36.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:35.917+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:36.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:35.917+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:36.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:35.795+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:36.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:35 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:35.795+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:36.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:36.248+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:36.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:36 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:36.193+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:36.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:36 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:36.193+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:36.921 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:36.576+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:36.921 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:36.576+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:37.020 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:36.956+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:37.020 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:36.956+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:37.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:36.768+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:37.021 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:36.768+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:37.021 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:36.768+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:37.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:36 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:36.790+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:37.241 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:36.920+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:37.242 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:36.920+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:37.242 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:36 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:36.920+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:37.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:37.240+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:37.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:37.214+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:37.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:37.214+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:37.883 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:37.551+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:37.883 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:37.551+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:37.883 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:37.551+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.230504+0000 front 2024-02-19T16:34:13.230526+0000 (oldest deadline 2024-02-19T16:34:37.330728+0000) 2024-02-19T16:34:38.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:37.795+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:38.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:37 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:37.795+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:38.257 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:37.881+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:38.257 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:37.881+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:38.257 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:37 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:37.881+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:38.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:37.968+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:38.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:37.968+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:38.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:37.785+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:38.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:37.785+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:38.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:37 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:37.785+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:38.524 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:38.255+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:38.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:38.173+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:38.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:38.173+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:38.777 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:38.523+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:54.428879+0000 front 2024-02-19T16:33:54.428918+0000 (oldest deadline 2024-02-19T16:34:25.029128+0000) 2024-02-19T16:34:38.778 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:38.523+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.029946+0000 front 2024-02-19T16:34:05.029706+0000 (oldest deadline 2024-02-19T16:34:30.329623+0000) 2024-02-19T16:34:38.778 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5[48932]: 2024-02-19T16:34:38.523+0000 7f0d86bd7700 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.230504+0000 front 2024-02-19T16:34:13.230526+0000 (oldest deadline 2024-02-19T16:34:37.330728+0000) 2024-02-19T16:34:39.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:38.743+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:39.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:38.743+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:39.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:38.743+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:39.021 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:38.971+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:39.021 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:38 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:38.971+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:39.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:38.752+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:39.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:38 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:38.752+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:39.198 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:38.872+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:39.198 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:38.872+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:39.198 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:38 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:38.872+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:39.528 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:39 smithi191 podman[73844]: 2024-02-19 16:34:39.197460589 +0000 UTC m=+5.441788635 container died 85c0f47f4ab89ea6fb4390844754c5a994528eab8afcafa18a10f6adf0bf418c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-19T16:34:39.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:39 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:39.233+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:39.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:39 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:39.233+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:39.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:39 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:39.221+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:39.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:39 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:39.221+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:39.916 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:39 smithi191 podman[73844]: 2024-02-19 16:34:39.670904602 +0000 UTC m=+5.915232648 container cleanup 85c0f47f4ab89ea6fb4390844754c5a994528eab8afcafa18a10f6adf0bf418c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:34:39.917 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:39 smithi191 bash[73844]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5 2024-02-19T16:34:40.020 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:39.936+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:40.020 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:39.936+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:40.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:39.758+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:40.021 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:39.758+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:40.021 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:39 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:39.758+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:40.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:39 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:39.751+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:40.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:39 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:39.751+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:40.196 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73861]: 2024-02-19 16:34:40.005996551 +0000 UTC m=+0.810596926 container remove 85c0f47f4ab89ea6fb4390844754c5a994528eab8afcafa18a10f6adf0bf418c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T16:34:40.196 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:39 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:39.914+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:40.196 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:39 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:39.914+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:40.196 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:39 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:39.914+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:40.359 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:40.189+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:40.359 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:40.189+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:40.360 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:40.189+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:40.507 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73955]: 2024-02-19 16:34:40.231931431 +0000 UTC m=+0.017777938 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:40.507 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73955]: 2024-02-19 16:34:40.382559755 +0000 UTC m=+0.168406295 container create ef933f9b8c7afb393a57f46f92febf93a92d271c4051bf87409de3924c023122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-19T16:34:40.507 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:40 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:40.194+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:40.507 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:40 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:40.194+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:40.777 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73955]: 2024-02-19 16:34:40.627549466 +0000 UTC m=+0.413395964 container init ef933f9b8c7afb393a57f46f92febf93a92d271c4051bf87409de3924c023122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-19T16:34:40.778 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73955]: 2024-02-19 16:34:40.630734365 +0000 UTC m=+0.416580857 container start ef933f9b8c7afb393a57f46f92febf93a92d271c4051bf87409de3924c023122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-19T16:34:40.778 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73955]: 2024-02-19 16:34:40.727450535 +0000 UTC m=+0.513297032 container attach ef933f9b8c7afb393a57f46f92febf93a92d271c4051bf87409de3924c023122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:34:41.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:40.715+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:41.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:40 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:40.715+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:41.228 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 19 16:34:40 smithi191 podman[73955]: 2024-02-19 16:34:40.918458991 +0000 UTC m=+0.704305488 container died ef933f9b8c7afb393a57f46f92febf93a92d271c4051bf87409de3924c023122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:34:41.228 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:40 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:40.873+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:41.228 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:40 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:40.873+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:41.228 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:40 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:40.873+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:41.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:40.974+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:41.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:40.974+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:41.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:40.798+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:41.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:40.798+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:41.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:40 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:40.798+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:41.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:41.226+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:41.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:41.226+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:41.570 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.5.service' 2024-02-19T16:34:41.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:41.144+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:41.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:41.144+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:41.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:41.144+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:41.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:41.613 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-19T16:34:41.613 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-19T16:34:41.614 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.6 2024-02-19T16:34:41.919 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:41 smithi008 systemd[1]: Stopping Ceph osd.6 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:41.920 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:41.917+0000 7f1b80c7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:41.920 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:41.917+0000 7f1b80c7f700 -1 osd.6 110 *** Got signal Terminated *** 2024-02-19T16:34:41.920 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:41.917+0000 7f1b80c7f700 -1 osd.6 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:41.920 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:41.828+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:41.920 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:41.828+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:41.920 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:41.828+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:42.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:41.760+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:42.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:41 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:41.760+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:42.193 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:41.903+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:42.193 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:41.903+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:42.193 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:41 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:41.903+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:42.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:42 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:42.021+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:42.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:42 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:42.021+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:42.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:42 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:42.191+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:42.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:42 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:42.191+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:42.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:42 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:42.187+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:42.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:42 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:42.187+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:42.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:42 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:42.187+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:43.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:42 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:42.809+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:43.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:42 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:42.809+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:43.186 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:42 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:42.905+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:43.186 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:42 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:42.905+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:43.186 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:42 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:42.905+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:43.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:42 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:42.876+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:43.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:42 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:42.876+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:43.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:42 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:42.876+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:43.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:43.035+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:43.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:43.035+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:43.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:43 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:43.185+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:43.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:43 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:43.185+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:43.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:43 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:43.205+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:43.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:43 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:43.205+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:43.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:43 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:43.205+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:44.202 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:43 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:43.855+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:44.202 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:43 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:43.855+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:44.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:43.987+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:44.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:43.987+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:44.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:43.909+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:44.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:43.909+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:44.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:43.909+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:44.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:43 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:43.909+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:44.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:44.148+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:44.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:44.148+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:44.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:43 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:43.953+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:44.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:43 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:43.953+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:44.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:43 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:43.953+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:44.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:44 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:44.200+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:44.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:44 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:44.200+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:44.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:44 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:44.201+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:44.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:44 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:44.201+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:45.233 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:44 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:44.855+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:45.233 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:44 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:44.855+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:45.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:45.231+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:45.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:45.231+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:45.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:45.231+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:45.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:45.231+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:45.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:44 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:44.973+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:45.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:44 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:44.973+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:45.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:44 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:44.924+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:45.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:44 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:44.924+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:45.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:44 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:44.924+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:45.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:44 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:44.924+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:45.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:45.144+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:45.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:45.144+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:45.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:44.920+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:45.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:44.920+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:45.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:44 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:44.920+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:46.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:45.809+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:46.081 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:45.809+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:46.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:45.967+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:03.608511+0000 front 2024-02-19T16:34:03.608274+0000 (oldest deadline 2024-02-19T16:34:27.708177+0000) 2024-02-19T16:34:46.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:45.967+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.209300+0000 front 2024-02-19T16:34:11.209465+0000 (oldest deadline 2024-02-19T16:34:35.309324+0000) 2024-02-19T16:34:46.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6[58970]: 2024-02-19T16:34:45.967+0000 7f1b7926b700 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:20.411741+0000 front 2024-02-19T16:34:20.411668+0000 (oldest deadline 2024-02-19T16:34:45.711780+0000) 2024-02-19T16:34:46.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:45.928+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:46.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:45.928+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:46.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:45.928+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:46.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:45 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:45.928+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:46.277 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:46 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:46.109+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:46.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:46 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:46.109+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:46.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:45.968+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:46.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:45.968+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:46.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:45 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:45.968+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:46.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:46.211+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:46.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:46.211+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:46.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:46.211+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:46.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:46.211+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:47.168 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:46.844+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:47.169 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:46 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:46.844+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:47.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:46.954+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:47.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:46.954+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:47.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:46.954+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:47.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:46 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:46.954+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:47.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:47.143+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:47.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:47.143+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:47.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:46.999+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:47.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:46.999+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:47.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:47 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:46.999+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:47.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:47.167+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:47.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:47.167+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:47.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:47.167+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:47.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:47.167+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:47.695 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:47 smithi008 podman[106357]: 2024-02-19 16:34:47.274243212 +0000 UTC m=+5.466876190 container died a991015ef6b4f1138d6dd5f97207d1802594a44f56be89cf5f9b4a377e07d8a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-19T16:34:47.995 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:47 smithi008 podman[106357]: 2024-02-19 16:34:47.79350642 +0000 UTC m=+5.986139397 container cleanup a991015ef6b4f1138d6dd5f97207d1802594a44f56be89cf5f9b4a377e07d8a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:34:47.995 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:47 smithi008 bash[106357]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6 2024-02-19T16:34:48.170 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:47.843+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:48.171 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:47.843+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:48.171 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:47 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:47.843+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:48.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106374]: 2024-02-19 16:34:48.028154422 +0000 UTC m=+0.746464464 container remove a991015ef6b4f1138d6dd5f97207d1802594a44f56be89cf5f9b4a377e07d8a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-02-19T16:34:48.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:47.994+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:48.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:47.994+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:48.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:47.994+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:48.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:47 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:47.994+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:48.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:48.124+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:48.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:48.124+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:48.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:48.124+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:48.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:48.029+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:48.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:48.029+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:48.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:48 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:48.029+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:48.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:48.169+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:48.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:48.169+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:48.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:48.169+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:48.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:48.169+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:48.649 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106465]: 2024-02-19 16:34:48.257352511 +0000 UTC m=+0.017909482 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:48.649 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106465]: 2024-02-19 16:34:48.404392835 +0000 UTC m=+0.164949787 container create b9bbc59d975aef8614a846ba9420d973de2a105ff79a58c2ab1fa72ebd424e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T16:34:48.958 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106465]: 2024-02-19 16:34:48.648564699 +0000 UTC m=+0.409121651 container init b9bbc59d975aef8614a846ba9420d973de2a105ff79a58c2ab1fa72ebd424e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-19T16:34:48.958 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106465]: 2024-02-19 16:34:48.651788746 +0000 UTC m=+0.412345697 container start b9bbc59d975aef8614a846ba9420d973de2a105ff79a58c2ab1fa72ebd424e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-19T16:34:48.958 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106465]: 2024-02-19 16:34:48.731885357 +0000 UTC m=+0.492442308 container attach b9bbc59d975aef8614a846ba9420d973de2a105ff79a58c2ab1fa72ebd424e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:34:49.138 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:48.880+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:49.138 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:48.880+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:49.138 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:48 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:48.880+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:49.270 INFO:journalctl@ceph.osd.6.smithi008.stdout:Feb 19 16:34:48 smithi008 podman[106465]: 2024-02-19 16:34:48.957626612 +0000 UTC m=+0.718183571 container died b9bbc59d975aef8614a846ba9420d973de2a105ff79a58c2ab1fa72ebd424e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-19T16:34:49.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:48.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:49.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:48.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:49.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:48.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:49.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:48 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:48.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:49.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:49.174+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:49.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:49.174+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:49.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:49.174+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:49.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:49.043+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:49.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:49.043+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:49.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:49 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:49.043+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:49.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:49.136+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:49.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:49.136+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:49.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:49.136+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:49.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:49.136+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:49.706 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.6.service' 2024-02-19T16:34:49.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:49.753 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-19T16:34:49.754 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-19T16:34:49.754 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.7 2024-02-19T16:34:50.069 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:49 smithi156 systemd[1]: Stopping Ceph osd.7 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:50.069 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:49.834+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:50.069 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:49.834+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:50.069 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:49 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:49.834+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:50.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:50.067+0000 7f25418c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:50.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:50.067+0000 7f25418c3700 -1 osd.7 110 *** Got signal Terminated *** 2024-02-19T16:34:50.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:50.067+0000 7f25418c3700 -1 osd.7 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:50.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:50.124+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:50.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:50.124+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:50.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:50.124+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:50.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:50.124+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:50.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.043+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:50.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.043+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:50.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.043+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:50.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:50 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.043+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:50.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:50.175+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:50.527 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:50.175+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:50.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:50.175+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:50.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:50.082+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:50.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:50.082+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:50.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:50 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:50.082+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:51.080 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:50.837+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:51.081 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:50.837+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:51.081 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:50.837+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:51.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.999+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:51.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.999+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:51.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.999+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:51.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:51 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:50.999+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:51.353 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:51.078+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:51.354 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:51.078+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:51.354 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:51.078+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:51.354 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:51.078+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:51.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:51.126+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:51.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:51.126+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:51.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:51.126+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:51.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:51.035+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:51.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:51.035+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:51.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:51 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:51.035+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:52.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:51.848+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:52.331 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:51.848+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:52.331 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:51 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:51.848+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:52.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:52.090+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:52.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:52.090+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:52.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:52.090+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:52.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:52.090+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:52.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:52.030+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:52.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:52.030+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:52.521 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:52.030+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:52.521 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:52 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:52.030+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:52.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:52.036+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:52.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:52.036+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:52.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:52.036+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:52.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:52.132+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:52.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:52.132+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:52.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:52.132+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:52.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:52 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:52.132+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:53.124 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:52.846+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:53.124 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:52.846+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:53.124 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:52.846+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:53.124 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:52 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:52.846+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.649551+0000 front 2024-02-19T16:34:27.649357+0000 (oldest deadline 2024-02-19T16:34:52.349437+0000) 2024-02-19T16:34:53.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:53.002+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:53.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:53.002+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:53.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:53.002+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:53.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:53.094+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:53.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:53.094+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:53.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:53.094+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:53.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:53 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:53.094+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:53.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:53.023+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:53.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:53.023+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:53.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:53.023+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:53.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:53 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:53.023+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:53.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:53.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:53.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:53.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:53.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:53.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:53.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:53.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:54.158 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:53.867+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:54.159 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:53.867+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:54.159 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:53.867+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:54.159 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:53 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:53.868+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.649551+0000 front 2024-02-19T16:34:27.649357+0000 (oldest deadline 2024-02-19T16:34:52.349437+0000) 2024-02-19T16:34:54.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.016+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:54.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.016+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:54.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.016+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:54.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.016+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:54.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:54.129+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:54.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:54.129+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:54.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:54.129+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:54.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:54.129+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:54.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:54.006+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:54.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:54.006+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:54.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:54 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:54.006+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:54.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:54.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:54.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:54.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:54.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:54.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:54.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:54.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:54.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:54.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:34:55.152 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:54.864+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.445798+0000 front 2024-02-19T16:34:05.445820+0000 (oldest deadline 2024-02-19T16:34:27.146008+0000) 2024-02-19T16:34:55.153 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:54.864+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.548046+0000 front 2024-02-19T16:34:13.547900+0000 (oldest deadline 2024-02-19T16:34:37.647881+0000) 2024-02-19T16:34:55.153 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:54.864+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.348941+0000 front 2024-02-19T16:34:22.349081+0000 (oldest deadline 2024-02-19T16:34:47.648858+0000) 2024-02-19T16:34:55.153 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7[54645]: 2024-02-19T16:34:54.864+0000 7f253deb7700 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.649551+0000 front 2024-02-19T16:34:27.649357+0000 (oldest deadline 2024-02-19T16:34:52.349437+0000) 2024-02-19T16:34:55.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:55.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:55.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:55.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:55.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:54 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:54.993+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:34:55.277 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.001+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:55.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.002+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:55.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.002+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:55.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:55.136+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:55.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:55.136+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:55.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:55.136+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:55.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:55.136+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:55.581 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:55 smithi156 podman[72978]: 2024-02-19 16:34:55.398772157 +0000 UTC m=+5.449952891 container died ce88c1be18944d0159a15b3150a4fb61134d042adc3ebd01ad4d794c07f35d0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-19T16:34:55.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:55 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:55.151+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:55.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:55 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:55.151+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:55.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:55 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:55.151+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:55.582 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:55 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:55.151+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:55.582 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:55 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:55.151+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:34:55.935 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:55 smithi156 podman[72978]: 2024-02-19 16:34:55.73305671 +0000 UTC m=+5.784237451 container cleanup ce88c1be18944d0159a15b3150a4fb61134d042adc3ebd01ad4d794c07f35d0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-19T16:34:55.935 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:55 smithi156 bash[72978]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7 2024-02-19T16:34:56.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:55 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:55.973+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:56.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:55 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:55.973+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:56.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:55 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:55.973+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:56.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:55 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:55.973+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:56.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:55 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:55.973+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:34:56.277 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:56.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:56.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:56.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:56.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:56.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:56.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:56.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:56.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.960+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:56.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.960+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:56.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.960+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:56.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:55 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:55.960+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:34:56.330 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:55 smithi156 podman[72995]: 2024-02-19 16:34:55.934619964 +0000 UTC m=+0.537326827 container remove ce88c1be18944d0159a15b3150a4fb61134d042adc3ebd01ad4d794c07f35d0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-19T16:34:56.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:56 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:56.149+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:56.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:56 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:56.149+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:56.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:56 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:56.149+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:56.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:56 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:56.149+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:56.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:56 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:56.149+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:34:56.605 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:56 smithi156 podman[73088]: 2024-02-19 16:34:56.249505571 +0000 UTC m=+0.017465719 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:34:56.606 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:56 smithi156 podman[73088]: 2024-02-19 16:34:56.360972686 +0000 UTC m=+0.128932840 container create b4fc2e641259e28abfd7fdabb03f696c3239f61455e653d1b8c3b99eec12cd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T16:34:56.606 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:56 smithi156 podman[73088]: 2024-02-19 16:34:56.604709462 +0000 UTC m=+0.372669598 container init b4fc2e641259e28abfd7fdabb03f696c3239f61455e653d1b8c3b99eec12cd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T16:34:56.895 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:56 smithi156 podman[73088]: 2024-02-19 16:34:56.607986855 +0000 UTC m=+0.375946993 container start b4fc2e641259e28abfd7fdabb03f696c3239f61455e653d1b8c3b99eec12cd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-19T16:34:56.895 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:56 smithi156 podman[73088]: 2024-02-19 16:34:56.687935613 +0000 UTC m=+0.455895764 container attach b4fc2e641259e28abfd7fdabb03f696c3239f61455e653d1b8c3b99eec12cd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-19T16:34:57.166 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 19 16:34:56 smithi156 podman[73088]: 2024-02-19 16:34:56.894187545 +0000 UTC m=+0.662147680 container died b4fc2e641259e28abfd7fdabb03f696c3239f61455e653d1b8c3b99eec12cd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-19T16:34:57.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:57 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:57.010+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:57.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:57 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:57.010+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:57.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:57 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:57.010+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:57.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:57 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:57.010+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:57.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:57 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:57.010+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:34:57.277 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.141+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:57.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.141+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:57.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.141+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:57.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.141+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:57.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:56.985+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:57.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:56.985+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:57.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:56.985+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:57.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:56 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:56.985+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:34:57.551 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:57 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:57.164+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:57.551 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:57 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:57.164+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:57.551 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:57 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:57.164+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:57.551 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:57 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:57.164+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:57.551 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:57 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:57.164+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:34:57.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.7.service' 2024-02-19T16:34:57.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:34:57.626 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-19T16:34:57.626 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-19T16:34:57.626 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.8 2024-02-19T16:34:57.942 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 systemd[1]: Stopping Ceph osd.8 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:34:58.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.018+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:58.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.018+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:58.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.018+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:58.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.018+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:58.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.018+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:34:58.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:57.951+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:58.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:57.951+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:58.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:57.951+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:58.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:57.951+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:34:58.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.940+0000 7fbf946cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:34:58.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.940+0000 7fbf946cc700 -1 osd.8 110 *** Got signal Terminated *** 2024-02-19T16:34:58.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:57 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:57.940+0000 7fbf946cc700 -1 osd.8 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:34:58.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:58.113+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:58.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:58.113+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:58.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:58.113+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:58.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:58.113+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:58.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:58.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:58.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:58.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:58.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:58.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:58.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:58.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:58.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:58.132+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:34:59.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.974+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:34:59.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.974+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:34:59.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.974+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:34:59.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.974+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:34:59.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:58 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:58.974+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:34:59.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:59 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:59.137+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:34:59.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:59 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:59.137+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:34:59.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:59 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:59.137+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:34:59.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:34:59 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:34:59.137+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:34:59.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:58.991+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:34:59.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:58.991+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:34:59.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:58.991+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:34:59.279 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:34:58 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:34:58.991+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:34:59.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:59 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:59.165+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:34:59.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:59 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:59.165+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:34:59.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:59 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:59.165+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:34:59.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:59 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:59.165+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:34:59.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:34:59 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:34:59.165+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:00.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:59 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:59.995+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:00.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:59 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:59.995+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:00.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:59 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:59.995+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:00.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:59 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:59.995+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:00.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:59 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:59.995+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:00.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:34:59 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:34:59.995+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:00.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:00.037+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:00.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:00.037+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:00.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:00.037+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:00.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:00.037+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:00.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:00.135+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:35:00.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:00.135+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:35:00.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:00.135+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:35:00.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:00.135+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:35:00.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:00 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:00.135+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:37.217718+0000 front 2024-02-19T16:34:37.217749+0000 (oldest deadline 2024-02-19T16:34:59.418539+0000) 2024-02-19T16:35:00.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:00 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:00.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:00.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:00 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:00.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:00.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:00 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:00.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:00.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:00 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:00.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:00.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:00 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:00.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:01.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:01 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:01.017+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:01.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:01 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:01.017+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:01.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:01 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:01.017+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:01.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:01 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:01.017+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:01.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:01 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:01.017+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:01.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:01 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:01.017+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:01.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:01.020+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:01.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:01.020+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:01.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:01.020+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:01.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:01.020+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:01.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:01.020+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:01.278 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:01.103+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:35:01.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:01.103+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:35:01.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:01.103+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:35:01.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:01.103+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:35:01.279 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:01 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:01.103+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:37.217718+0000 front 2024-02-19T16:34:37.217749+0000 (oldest deadline 2024-02-19T16:34:59.418539+0000) 2024-02-19T16:35:01.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:01.183+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:01.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:01.183+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:01.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:01.183+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:01.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:01.183+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:01.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:01 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:01.183+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:02.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.044+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:02.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.044+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:02.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.044+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:02.520 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.044+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:02.521 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.044+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:02.521 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.044+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:02.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:02.114+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:01.712109+0000 front 2024-02-19T16:34:01.712073+0000 (oldest deadline 2024-02-19T16:34:28.112613+0000) 2024-02-19T16:35:02.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:02.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:14.315237+0000 front 2024-02-19T16:34:14.315457+0000 (oldest deadline 2024-02-19T16:34:38.415417+0000) 2024-02-19T16:35:02.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:02.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.516730+0000 front 2024-02-19T16:34:22.516701+0000 (oldest deadline 2024-02-19T16:34:47.816463+0000) 2024-02-19T16:35:02.528 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:02.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:27.817181+0000 front 2024-02-19T16:34:27.816962+0000 (oldest deadline 2024-02-19T16:34:51.916927+0000) 2024-02-19T16:35:02.529 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8[55520]: 2024-02-19T16:35:02.115+0000 7fbf90cc0700 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:37.217718+0000 front 2024-02-19T16:34:37.217749+0000 (oldest deadline 2024-02-19T16:34:59.418539+0000) 2024-02-19T16:35:02.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:02.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:02.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:02.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:02.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:02.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:02.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:02.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:02.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:02.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:02.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:02 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:02.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:02.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:02 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:02.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:02.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:02 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:02.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:02.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:02 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:02.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:02.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:02 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:02.157+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:03.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.996+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:03.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.996+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:03.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.996+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:03.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.996+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:03.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.996+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:03.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:02 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:02.996+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:03.304 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:03 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:03.031+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:03.304 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:03 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:03.031+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:03.304 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:03 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:03.031+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:03.304 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:03 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:03.031+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:03.304 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:03 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:03.031+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:03.578 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:03 smithi191 podman[74062]: 2024-02-19 16:35:03.302825869 +0000 UTC m=+5.485676684 container died 0e13f5ec443c29e5c7fb70ac6f8982b74553d5cf37033548d65ec7d5d9c51331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T16:35:03.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:03 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:03.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:03.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:03 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:03.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:03.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:03 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:03.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:03.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:03 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:03.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:03.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:03 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:03.163+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:03.936 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:03 smithi191 podman[74062]: 2024-02-19 16:35:03.666798115 +0000 UTC m=+5.849648944 container cleanup 0e13f5ec443c29e5c7fb70ac6f8982b74553d5cf37033548d65ec7d5d9c51331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD) 2024-02-19T16:35:03.936 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:03 smithi191 bash[74062]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8 2024-02-19T16:35:04.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:03 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:03.980+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:04.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:03 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:03.980+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:04.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:03 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:03.980+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:04.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:03 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:03.980+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:04.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:03 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:03.980+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:04.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:03 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:03.980+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:04.277 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:03 smithi191 podman[74079]: 2024-02-19 16:35:03.934701449 +0000 UTC m=+0.627959815 container remove 0e13f5ec443c29e5c7fb70ac6f8982b74553d5cf37033548d65ec7d5d9c51331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:35:04.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:04 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:04.075+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:04.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:04 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:04.075+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:04.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:04 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:04.075+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:04.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:04 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:04.075+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:04.278 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:04 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:04.075+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:04.539 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:04 smithi191 podman[74174]: 2024-02-19 16:35:04.18219785 +0000 UTC m=+0.017767311 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:35:04.539 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:04 smithi191 podman[74174]: 2024-02-19 16:35:04.294795218 +0000 UTC m=+0.130364732 container create ab65c9e8934f530e8768e422d18a01dd7f63e05c81904f0ee9b6df522fc81f05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-19T16:35:04.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:04 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:04.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:04.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:04 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:04.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:04.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:04 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:04.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:04.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:04 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:04.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:04.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:04 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:04.182+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:04.834 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:04 smithi191 podman[74174]: 2024-02-19 16:35:04.538354292 +0000 UTC m=+0.373923770 container init ab65c9e8934f530e8768e422d18a01dd7f63e05c81904f0ee9b6df522fc81f05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-19T16:35:04.835 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:04 smithi191 podman[74174]: 2024-02-19 16:35:04.541658051 +0000 UTC m=+0.377227513 container start ab65c9e8934f530e8768e422d18a01dd7f63e05c81904f0ee9b6df522fc81f05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-02-19T16:35:04.835 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:04 smithi191 podman[74174]: 2024-02-19 16:35:04.621447643 +0000 UTC m=+0.457017124 container attach ab65c9e8934f530e8768e422d18a01dd7f63e05c81904f0ee9b6df522fc81f05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:35:04.835 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 19 16:35:04 smithi191 podman[74174]: 2024-02-19 16:35:04.83307826 +0000 UTC m=+0.668647725 container died ab65c9e8934f530e8768e422d18a01dd7f63e05c81904f0ee9b6df522fc81f05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-19T16:35:05.142 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:05 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:05.059+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:05.142 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:05 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:05.059+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:05.142 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:05 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:05.059+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:05.143 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:05 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:05.059+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:05.143 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:05 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:05.059+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:05.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:04 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:04.946+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:05.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:04 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:04.946+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:05.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:04 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:04.946+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:05.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:04 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:04.946+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:05.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:04 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:04.946+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:05.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:04 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:04.946+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:05.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.8.service' 2024-02-19T16:35:05.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:05.502 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-19T16:35:05.502 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-19T16:35:05.502 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.9 2024-02-19T16:35:05.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:05 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:05.202+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:05.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:05 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:05.202+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:05.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:05 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:05.202+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:05.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:05 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:05.202+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:05.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:05 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:05.202+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:05.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:05 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:05.202+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:05.831 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 systemd[1]: Stopping Ceph osd.9 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:06.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.829+0000 7f0d9fd1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:06.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.829+0000 7f0d9fd1f700 -1 osd.9 110 *** Got signal Terminated *** 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.829+0000 7f0d9fd1f700 -1 osd.9 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.962+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.962+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.962+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.962+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:06.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:05 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:05.962+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:06.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:06.094+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:06.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:06.094+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:06.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:06.094+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:06.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:06.094+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:06.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:06.094+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:06.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:06.223+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:06.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:06.223+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:06.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:06.223+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:06.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:06.223+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:06.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:06.223+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:06.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:06 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:06.223+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:07.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.004+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:07.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.004+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:07.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.004+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:07.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.004+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:07.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.004+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:07.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.004+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:07.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:07 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:07.122+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:07.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:07 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:07.122+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:07.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:07 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:07.122+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:07.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:07 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:07.122+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:07.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:07 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:07.122+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:07.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:07 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:07.175+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:07.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:07 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:07.175+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:07.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:07 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:07.175+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:07.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:07 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:07.175+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:07.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:07 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:07.175+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:07.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:07 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:07.175+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:08.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:08.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:08.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:08.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:08.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:08.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:07 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:07.961+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:08.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:08 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:08.074+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:08.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:08 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:08.074+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:08.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:08 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:08.074+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:08.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:08 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:08.074+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:08.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:08 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:08.074+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:08.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:08 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:08.140+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:08.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:08 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:08.140+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:08.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:08 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:08.140+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:08.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:08 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:08.140+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:08.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:08 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:08.140+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:08.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:08 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:08.140+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:09.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:08 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:08.941+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:09.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:08 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:08.941+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:09.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:08 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:08.941+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:09.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:08 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:08.941+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:09.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:08 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:08.941+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:09.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:08 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:08.941+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:09.429 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:09.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:09.429 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:09.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:09.429 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:09.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:09.429 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:09.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:09.429 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:09 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:09.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:09.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:09 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:09.141+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:09.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:09 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:09.141+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:09.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:09 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:09.141+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:09.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:09 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:09.141+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:09.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:09 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:09.141+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:09.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:09 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:09.141+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:10.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:09.967+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.288673+0000 front 2024-02-19T16:33:55.288799+0000 (oldest deadline 2024-02-19T16:34:21.188820+0000) 2024-02-19T16:35:10.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:09.967+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.289781+0000 front 2024-02-19T16:34:05.290056+0000 (oldest deadline 2024-02-19T16:34:31.189811+0000) 2024-02-19T16:35:10.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:09.967+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:11.190593+0000 front 2024-02-19T16:34:11.190471+0000 (oldest deadline 2024-02-19T16:34:35.890440+0000) 2024-02-19T16:35:10.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:09.967+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:22.192276+0000 front 2024-02-19T16:34:22.192320+0000 (oldest deadline 2024-02-19T16:34:43.892454+0000) 2024-02-19T16:35:10.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:09.967+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.193795+0000 front 2024-02-19T16:34:29.193811+0000 (oldest deadline 2024-02-19T16:34:54.493507+0000) 2024-02-19T16:35:10.271 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:09 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9[65985]: 2024-02-19T16:35:09.967+0000 7f0d9c313700 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:36.794545+0000 front 2024-02-19T16:34:36.794380+0000 (oldest deadline 2024-02-19T16:34:59.694263+0000) 2024-02-19T16:35:10.482 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:10.482 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:10.482 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:10.483 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:10.483 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:10.483 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:10.483 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:10 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:10.162+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:10.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:10.112+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:10.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:10.112+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:10.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:10.112+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:10.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:10.112+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:10.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:10.112+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:10.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:10.112+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:11.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:11 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:11.067+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:11.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:11 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:11.067+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:11.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:11 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:11.067+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:11.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:11 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:11.067+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:11.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:11 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:11.067+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:11.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:11 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:11.067+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:11.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:11.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:11.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:11.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:11.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:11.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:11.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:11 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:11.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:11.582 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:11 smithi008 podman[106570]: 2024-02-19 16:35:11.324807262 +0000 UTC m=+5.625109754 container died e24c1bd9e1d4d768385355bdca34f099c4adfcd67fb830caa02205cfe6d0efc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:35:11.965 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:11 smithi008 podman[106570]: 2024-02-19 16:35:11.670939695 +0000 UTC m=+5.971242187 container cleanup e24c1bd9e1d4d768385355bdca34f099c4adfcd67fb830caa02205cfe6d0efc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-19T16:35:11.965 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:11 smithi008 bash[106570]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9 2024-02-19T16:35:12.270 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:11 smithi008 podman[106588]: 2024-02-19 16:35:11.964142311 +0000 UTC m=+0.637870368 container remove e24c1bd9e1d4d768385355bdca34f099c4adfcd67fb830caa02205cfe6d0efc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-19T16:35:12.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:12 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:12.098+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:12.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:12 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:12.098+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:12.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:12 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:12.098+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:12.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:12 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:12.098+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:12.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:12 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:12.098+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:12.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:12 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:12.098+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:12.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:12 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:12.115+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:12.726 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:12 smithi008 podman[106678]: 2024-02-19 16:35:12.343420169 +0000 UTC m=+0.017461859 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:35:12.726 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:12 smithi008 podman[106678]: 2024-02-19 16:35:12.448214146 +0000 UTC m=+0.122255837 container create 874dfeb001c2c8f9bed3135dcf91f416b7290985736f121ada8d34df81d9302b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-19T16:35:12.727 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:12 smithi008 podman[106678]: 2024-02-19 16:35:12.659047411 +0000 UTC m=+0.333089104 container init 874dfeb001c2c8f9bed3135dcf91f416b7290985736f121ada8d34df81d9302b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-19T16:35:12.727 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:12 smithi008 podman[106678]: 2024-02-19 16:35:12.661991486 +0000 UTC m=+0.336033185 container start 874dfeb001c2c8f9bed3135dcf91f416b7290985736f121ada8d34df81d9302b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-19T16:35:12.727 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:12 smithi008 podman[106678]: 2024-02-19 16:35:12.725538825 +0000 UTC m=+0.399580535 container attach 874dfeb001c2c8f9bed3135dcf91f416b7290985736f121ada8d34df81d9302b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-19T16:35:13.020 INFO:journalctl@ceph.osd.9.smithi008.stdout:Feb 19 16:35:12 smithi008 podman[106678]: 2024-02-19 16:35:12.959180487 +0000 UTC m=+0.633222177 container died 874dfeb001c2c8f9bed3135dcf91f416b7290985736f121ada8d34df81d9302b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-02-19T16:35:13.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:13 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:13.083+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:13.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:13 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:13.083+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:13.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:13 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:13.083+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:13.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:13 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:13.083+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:13.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:13 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:13.083+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:13.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:13 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:13.083+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:13.555 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.9.service' 2024-02-19T16:35:13.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:13.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:13.580 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:13.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:13.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:13.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:13.581 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.116+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:13.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:13.599 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-19T16:35:13.599 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-19T16:35:13.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.10 2024-02-19T16:35:13.920 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 systemd[1]: Stopping Ceph osd.10 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:14.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.918+0000 7fcef42c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.918+0000 7fcef42c5700 -1 osd.10 110 *** Got signal Terminated *** 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:13 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:13.918+0000 7fcef42c5700 -1 osd.10 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:14.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:14 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:14.122+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:14.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:14.079+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:14.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:14.079+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:14.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:14.079+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:14.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:14.079+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:14.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:14.079+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:14.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:14.079+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:15.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:15.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:15.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:15.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:15.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:15.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:15.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:15 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:15.072+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:15.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:15 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:15.050+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:15.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:15 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:15.050+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:15.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:15 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:15.050+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:15.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:15 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:15.050+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:15.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:15 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:15.050+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:15.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:15 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:15.050+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:16.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:16.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:16.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:16.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:16.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:16.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:16.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:16 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:16.044+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:16.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:16 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:16.066+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:16.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:16 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:16.066+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:16.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:16 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:16.066+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:16.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:16 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:16.066+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:16.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:16 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:16.066+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:16.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:16 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:16.066+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:17.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:17.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:17.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:17.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:17.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:17.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:17.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:17 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:17.051+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:17.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:17 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:17.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:17.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:17 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:17.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:17.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:17 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:17.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:17.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:17 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:17.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:17.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:17 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:17.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:17.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:17 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:17.069+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:18.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:55.854251+0000 front 2024-02-19T16:33:55.854146+0000 (oldest deadline 2024-02-19T16:34:21.154178+0000) 2024-02-19T16:35:18.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:06.355796+0000 front 2024-02-19T16:34:06.355960+0000 (oldest deadline 2024-02-19T16:34:29.255924+0000) 2024-02-19T16:35:18.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:13.956986+0000 front 2024-02-19T16:34:13.957196+0000 (oldest deadline 2024-02-19T16:34:39.256940+0000) 2024-02-19T16:35:18.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2024-02-19T16:34:19.257760+0000 front 2024-02-19T16:34:19.257664+0000 (oldest deadline 2024-02-19T16:34:43.357540+0000) 2024-02-19T16:35:18.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:29.258466+0000 front 2024-02-19T16:34:29.258432+0000 (oldest deadline 2024-02-19T16:34:53.358515+0000) 2024-02-19T16:35:18.331 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:34.459698+0000 front 2024-02-19T16:34:34.459717+0000 (oldest deadline 2024-02-19T16:35:05.060062+0000) 2024-02-19T16:35:18.332 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:18 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10[61403]: 2024-02-19T16:35:18.003+0000 7fcef08b9700 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.060664+0000 front 2024-02-19T16:34:45.060645+0000 (oldest deadline 2024-02-19T16:35:09.160435+0000) 2024-02-19T16:35:18.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:18.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:18.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:18.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:18.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:18.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:18.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:18 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:18.056+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:19.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:19.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:19.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:19.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:19.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:19.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:19.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:19 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:19.085+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:19.661 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:19 smithi156 podman[73219]: 2024-02-19 16:35:19.401447798 +0000 UTC m=+5.608494037 container died 6d4563213b817b5daf867f8d4bb1f7deb5f7544978c316276df600c64dd17b6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-02-19T16:35:20.053 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:19 smithi156 podman[73219]: 2024-02-19 16:35:19.75972484 +0000 UTC m=+5.966771076 container cleanup 6d4563213b817b5daf867f8d4bb1f7deb5f7544978c316276df600c64dd17b6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-19T16:35:20.053 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:19 smithi156 bash[73219]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10 2024-02-19T16:35:20.330 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73236]: 2024-02-19 16:35:20.052518355 +0000 UTC m=+0.644582639 container remove 6d4563213b817b5daf867f8d4bb1f7deb5f7544978c316276df600c64dd17b6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-19T16:35:20.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:20.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:20.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:20.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:20.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:20.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:20.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:20 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:20.091+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:20.689 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73328]: 2024-02-19 16:35:20.270708645 +0000 UTC m=+0.017581277 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:35:20.690 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73328]: 2024-02-19 16:35:20.378328156 +0000 UTC m=+0.125200789 container create 28d8d8f2432089eecbbe016190f51969bd18762c190cc1e027ddfafe09fe7320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:35:20.690 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73328]: 2024-02-19 16:35:20.605715472 +0000 UTC m=+0.352588104 container init 28d8d8f2432089eecbbe016190f51969bd18762c190cc1e027ddfafe09fe7320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-19T16:35:20.690 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73328]: 2024-02-19 16:35:20.60902715 +0000 UTC m=+0.355899769 container start 28d8d8f2432089eecbbe016190f51969bd18762c190cc1e027ddfafe09fe7320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-19T16:35:20.690 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73328]: 2024-02-19 16:35:20.688814707 +0000 UTC m=+0.435687347 container attach 28d8d8f2432089eecbbe016190f51969bd18762c190cc1e027ddfafe09fe7320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10-deactivate, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-19T16:35:21.080 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 19 16:35:20 smithi156 podman[73328]: 2024-02-19 16:35:20.894671478 +0000 UTC m=+0.641544096 container died 28d8d8f2432089eecbbe016190f51969bd18762c190cc1e027ddfafe09fe7320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-10-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-19T16:35:21.507 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.10.service' 2024-02-19T16:35:21.527 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:21.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:21.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:21.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:21.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:21.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:21.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.120+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:21.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:21.547 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-19T16:35:21.548 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-19T16:35:21.548 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.11 2024-02-19T16:35:21.837 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 systemd[1]: Stopping Ceph osd.11 for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:22.171 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.834+0000 7fdc54ecf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:22.172 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.834+0000 7fdc54ecf700 -1 osd.11 110 *** Got signal Terminated *** 2024-02-19T16:35:22.172 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:21 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:21.834+0000 7fdc54ecf700 -1 osd.11 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T16:35:22.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:22.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:22.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:22.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:22.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:22.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:22.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:22 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:22.169+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:23.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:23 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:23.123+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:24.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:24 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:24.166+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:25.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:25.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:25.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:25.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:25.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:25.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:25.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:25.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:25 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:25.198+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-02-19T16:34:59.888666+0000 front 2024-02-19T16:34:59.888699+0000 (oldest deadline 2024-02-19T16:35:24.588655+0000) 2024-02-19T16:35:26.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-02-19T16:33:58.178266+0000 front 2024-02-19T16:33:58.178460+0000 (oldest deadline 2024-02-19T16:34:19.878406+0000) 2024-02-19T16:35:26.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-19T16:34:05.080389+0000 front 2024-02-19T16:34:05.080202+0000 (oldest deadline 2024-02-19T16:34:30.380388+0000) 2024-02-19T16:35:26.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-19T16:34:12.681383+0000 front 2024-02-19T16:34:12.681418+0000 (oldest deadline 2024-02-19T16:34:36.181579+0000) 2024-02-19T16:35:26.528 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-19T16:34:30.184208+0000 front 2024-02-19T16:34:30.184265+0000 (oldest deadline 2024-02-19T16:34:55.484193+0000) 2024-02-19T16:35:26.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-19T16:34:38.385126+0000 front 2024-02-19T16:34:38.385076+0000 (oldest deadline 2024-02-19T16:35:00.085132+0000) 2024-02-19T16:35:26.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2024-02-19T16:34:45.986337+0000 front 2024-02-19T16:34:45.986273+0000 (oldest deadline 2024-02-19T16:35:09.486145+0000) 2024-02-19T16:35:26.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-19T16:34:52.887861+0000 front 2024-02-19T16:34:52.887915+0000 (oldest deadline 2024-02-19T16:35:17.587716+0000) 2024-02-19T16:35:26.529 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:26 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11[62566]: 2024-02-19T16:35:26.229+0000 7fdc514c3700 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-02-19T16:34:59.888666+0000 front 2024-02-19T16:34:59.888699+0000 (oldest deadline 2024-02-19T16:35:24.588655+0000) 2024-02-19T16:35:27.652 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:27 smithi191 podman[74291]: 2024-02-19 16:35:27.333944719 +0000 UTC m=+5.594806599 container died 0750298011cc3d754007e55377798596db48c02bed83d4f6231ab49dbbd7ff97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-19T16:35:27.652 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:27 smithi191 podman[74291]: 2024-02-19 16:35:27.651153039 +0000 UTC m=+5.912014908 container cleanup 0750298011cc3d754007e55377798596db48c02bed83d4f6231ab49dbbd7ff97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-19T16:35:28.027 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:27 smithi191 bash[74291]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11 2024-02-19T16:35:28.028 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:27 smithi191 podman[74311]: 2024-02-19 16:35:27.860502007 +0000 UTC m=+0.524544752 container remove 0750298011cc3d754007e55377798596db48c02bed83d4f6231ab49dbbd7ff97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T16:35:28.472 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:28 smithi191 podman[74399]: 2024-02-19 16:35:28.060054274 +0000 UTC m=+0.017594068 image pull cec87a2a42a416e1420614505435ae4de282e255ae3705eacc156a98f91169c8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05 2024-02-19T16:35:28.473 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:28 smithi191 podman[74399]: 2024-02-19 16:35:28.177974915 +0000 UTC m=+0.135514713 container create 55362dc66be5300646fb55c546dd9e01a5a0f1f6457537451e4a0d7b30aa92cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:35:28.473 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:28 smithi191 podman[74399]: 2024-02-19 16:35:28.38840045 +0000 UTC m=+0.345940244 container init 55362dc66be5300646fb55c546dd9e01a5a0f1f6457537451e4a0d7b30aa92cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-19T16:35:28.473 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:28 smithi191 podman[74399]: 2024-02-19 16:35:28.391350487 +0000 UTC m=+0.348890265 container start 55362dc66be5300646fb55c546dd9e01a5a0f1f6457537451e4a0d7b30aa92cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-19T16:35:28.473 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:28 smithi191 podman[74399]: 2024-02-19 16:35:28.471547961 +0000 UTC m=+0.429087760 container attach 55362dc66be5300646fb55c546dd9e01a5a0f1f6457537451e4a0d7b30aa92cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-19T16:35:28.778 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 19 16:35:28 smithi191 podman[74399]: 2024-02-19 16:35:28.683551469 +0000 UTC m=+0.641091240 container died 55362dc66be5300646fb55c546dd9e01a5a0f1f6457537451e4a0d7b30aa92cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-19T16:35:29.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@osd.11.service' 2024-02-19T16:35:29.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:29.274 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-19T16:35:29.274 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-19T16:35:29.274 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.a 2024-02-19T16:35:29.683 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:35:29 smithi008 systemd[1]: Stopping Ceph mds.a for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:30.020 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:35:29 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-a[72540]: 2024-02-19T16:35:29.681+0000 7f72eb7e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:30.020 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:35:29 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-a[72540]: 2024-02-19T16:35:29.681+0000 7f72eb7e6700 -1 mds.a *** got signal Terminated *** 2024-02-19T16:35:30.950 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:35:30 smithi008 podman[106799]: 2024-02-19 16:35:30.67522383 +0000 UTC m=+1.208445615 container died 50d1bfbf92cef3fd64f2aea86043870e1d2ef7c13917951f8717862ecdf34fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-a, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-19T16:35:31.270 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:35:31 smithi008 podman[106799]: 2024-02-19 16:35:31.048118825 +0000 UTC m=+1.581340609 container cleanup 50d1bfbf92cef3fd64f2aea86043870e1d2ef7c13917951f8717862ecdf34fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-19T16:35:31.270 INFO:journalctl@ceph.mds.a.smithi008.stdout:Feb 19 16:35:31 smithi008 bash[106799]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-a 2024-02-19T16:35:31.484 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.a.service' 2024-02-19T16:35:31.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:31.521 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-19T16:35:31.521 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-19T16:35:31.521 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.d 2024-02-19T16:35:32.020 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:31 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-d[71582]: 2024-02-19T16:35:31.826+0000 7f5c12426700 -1 mds.d *** got signal Terminated *** 2024-02-19T16:35:35.262 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:34 smithi008 podman[106920]: 2024-02-19 16:35:34.97394801 +0000 UTC m=+3.240090716 container died 2e861e0d5659cee780e98a24b0c195ce154c0e975d693e3d2f1249404b2a9619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-d, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:35:35.520 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:35 smithi008 podman[106920]: 2024-02-19 16:35:35.351955404 +0000 UTC m=+3.618098152 container cleanup 2e861e0d5659cee780e98a24b0c195ce154c0e975d693e3d2f1249404b2a9619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:35:35.520 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:35 smithi008 bash[106920]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-d 2024-02-19T16:35:35.938 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.d.service' 2024-02-19T16:35:35.962 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:35 smithi008 podman[106934]: 2024-02-19 16:35:35.71173294 +0000 UTC m=+0.737710491 container remove 2e861e0d5659cee780e98a24b0c195ce154c0e975d693e3d2f1249404b2a9619 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T16:35:35.963 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:35 smithi008 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.d.service: Deactivated successfully. 2024-02-19T16:35:35.963 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:35 smithi008 systemd[1]: Stopped Ceph mds.d for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:35:35.963 INFO:journalctl@ceph.mds.d.smithi008.stdout:Feb 19 16:35:35 smithi008 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.d.service: Consumed 1.068s CPU time. 2024-02-19T16:35:35.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:35.973 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-19T16:35:35.973 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-19T16:35:35.973 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.g 2024-02-19T16:35:36.270 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:35:36 smithi008 systemd[1]: Stopping Ceph mds.g for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:36.770 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:35:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g[72175]: 2024-02-19T16:35:36.313+0000 7f8071bc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:36.770 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:35:36 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g[72175]: 2024-02-19T16:35:36.313+0000 7f8071bc0700 -1 mds.g *** got signal Terminated *** 2024-02-19T16:35:40.215 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:35:39 smithi008 podman[107041]: 2024-02-19 16:35:39.852791853 +0000 UTC m=+3.655310534 container died 7169860ae5abac0f3f0fb7e073618948060282b75fd5c56cbdd3c723dac15927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T16:35:40.520 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:35:40 smithi008 podman[107041]: 2024-02-19 16:35:40.339615316 +0000 UTC m=+4.142133999 container cleanup 7169860ae5abac0f3f0fb7e073618948060282b75fd5c56cbdd3c723dac15927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-19T16:35:40.520 INFO:journalctl@ceph.mds.g.smithi008.stdout:Feb 19 16:35:40 smithi008 bash[107041]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-g 2024-02-19T16:35:40.721 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.g.service' 2024-02-19T16:35:40.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:40.756 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-19T16:35:40.756 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-19T16:35:40.756 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.j 2024-02-19T16:35:41.020 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:40 smithi008 systemd[1]: Stopping Ceph mds.j for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:41.520 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j[71233]: 2024-02-19T16:35:41.109+0000 7f9eccc6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:41.520 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:41 smithi008 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j[71233]: 2024-02-19T16:35:41.109+0000 7f9eccc6c700 -1 mds.j *** got signal Terminated *** 2024-02-19T16:35:44.352 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:44 smithi008 podman[107160]: 2024-02-19 16:35:44.023376245 +0000 UTC m=+3.050791164 container died 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-02-19T16:35:44.353 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:44 smithi008 podman[107160]: 2024-02-19 16:35:44.351759729 +0000 UTC m=+3.379174656 container cleanup 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-02-19T16:35:44.765 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:44 smithi008 bash[107160]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j 2024-02-19T16:35:44.766 INFO:journalctl@ceph.mds.j.smithi008.stdout:Feb 19 16:35:44 smithi008 podman[107174]: 2024-02-19 16:35:44.527766737 +0000 UTC m=+0.502856563 container remove 664f851e55ed027fe119e992a0ef4ec8e3bdf78f53bb2ec68240cc7af08103b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True) 2024-02-19T16:35:44.781 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.j.service' 2024-02-19T16:35:44.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:44.815 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-19T16:35:44.815 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-19T16:35:44.815 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.b 2024-02-19T16:35:45.102 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:44 smithi156 systemd[1]: Stopping Ceph mds.b for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:45.372 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b[66973]: 2024-02-19T16:35:45.101+0000 7fc292879700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:45.372 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:45 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b[66973]: 2024-02-19T16:35:45.101+0000 7fc292879700 -1 mds.b *** got signal Terminated *** 2024-02-19T16:35:49.418 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:49 smithi156 podman[73432]: 2024-02-19 16:35:49.110825109 +0000 UTC m=+4.104430439 container died 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:35:49.418 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:49 smithi156 podman[73432]: 2024-02-19 16:35:49.417200807 +0000 UTC m=+4.410806139 container cleanup 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:35:49.772 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:49 smithi156 bash[73432]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b 2024-02-19T16:35:49.772 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 19 16:35:49 smithi156 podman[73458]: 2024-02-19 16:35:49.651850659 +0000 UTC m=+0.533624490 container remove 9c1f4ac98c1f2e1668b6f0a2c52c47f0ccef7d800894b942d338c5c5fdebcc80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-b, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:35:49.788 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.b.service' 2024-02-19T16:35:49.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:49.826 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-19T16:35:49.826 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-19T16:35:49.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.e 2024-02-19T16:35:50.080 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:35:49 smithi156 systemd[1]: Stopping Ceph mds.e for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:50.580 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:35:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e[66345]: 2024-02-19T16:35:50.188+0000 7fafa1628700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:50.580 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:35:50 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e[66345]: 2024-02-19T16:35:50.188+0000 7fafa1628700 -1 mds.e *** got signal Terminated *** 2024-02-19T16:35:54.123 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:35:53 smithi156 podman[73568]: 2024-02-19 16:35:53.838163333 +0000 UTC m=+3.795818147 container died 7f17d82d3cf70665c95bc543f72dd71a7d88fa3fc642c6c76f47d9cd4ed7cd30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:35:54.441 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:35:54 smithi156 podman[73568]: 2024-02-19 16:35:54.2214961 +0000 UTC m=+4.179150905 container cleanup 7f17d82d3cf70665c95bc543f72dd71a7d88fa3fc642c6c76f47d9cd4ed7cd30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, CEPH_POINT_RELEASE=, ceph=True) 2024-02-19T16:35:54.442 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 19 16:35:54 smithi156 bash[73568]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-e 2024-02-19T16:35:54.560 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.e.service' 2024-02-19T16:35:54.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:54.596 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-19T16:35:54.597 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-19T16:35:54.597 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.h 2024-02-19T16:35:54.830 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:35:54 smithi156 systemd[1]: Stopping Ceph mds.h for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:55.330 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:35:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h[66662]: 2024-02-19T16:35:54.917+0000 7f1cb015f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:55.330 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:35:54 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h[66662]: 2024-02-19T16:35:54.917+0000 7f1cb015f700 -1 mds.h *** got signal Terminated *** 2024-02-19T16:35:58.062 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:35:57 smithi156 podman[73689]: 2024-02-19 16:35:57.642638099 +0000 UTC m=+2.832176635 container died 297efbd4b3c88532493afd32d56d9cf3f25ac099923aa848da92905030825ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:35:58.330 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:35:58 smithi156 podman[73689]: 2024-02-19 16:35:58.126826982 +0000 UTC m=+3.316365520 container cleanup 297efbd4b3c88532493afd32d56d9cf3f25ac099923aa848da92905030825ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-19T16:35:58.331 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 19 16:35:58 smithi156 bash[73689]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-h 2024-02-19T16:35:58.466 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.h.service' 2024-02-19T16:35:58.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:35:58.501 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-19T16:35:58.501 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-19T16:35:58.501 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.k 2024-02-19T16:35:58.815 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:35:58 smithi156 systemd[1]: Stopping Ceph mds.k for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:35:59.080 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:35:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k[66025]: 2024-02-19T16:35:58.813+0000 7f5fcb790700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:35:59.080 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:35:58 smithi156 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k[66025]: 2024-02-19T16:35:58.813+0000 7f5fcb790700 -1 mds.k *** got signal Terminated *** 2024-02-19T16:36:01.330 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:36:01 smithi156 podman[73809]: 2024-02-19 16:36:01.076688258 +0000 UTC m=+2.361381176 container died 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e) 2024-02-19T16:36:01.765 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:36:01 smithi156 podman[73809]: 2024-02-19 16:36:01.404924526 +0000 UTC m=+2.689617460 container cleanup 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:36:01.765 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:36:01 smithi156 bash[73809]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k 2024-02-19T16:36:01.765 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 19 16:36:01 smithi156 podman[73823]: 2024-02-19 16:36:01.564054578 +0000 UTC m=+0.486904928 container remove 9981358b9c47a9f460454b64b8ceaa5939db01bb9173f01b48708f4554b7ad7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:36:01.780 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.k.service' 2024-02-19T16:36:01.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:36:01.814 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-19T16:36:01.814 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-19T16:36:01.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.c 2024-02-19T16:36:02.108 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:01 smithi191 systemd[1]: Stopping Ceph mds.c for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:36:02.387 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: 2024-02-19T16:36:02.107+0000 7f248dbb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:36:02.387 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:02 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c[67516]: 2024-02-19T16:36:02.107+0000 7f248dbb5700 -1 mds.c *** got signal Terminated *** 2024-02-19T16:36:05.941 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:05 smithi191 podman[74504]: 2024-02-19 16:36:05.641464463 +0000 UTC m=+3.637559756 container died 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True) 2024-02-19T16:36:06.224 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:06 smithi191 podman[74504]: 2024-02-19 16:36:06.039208793 +0000 UTC m=+4.035304058 container cleanup 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-19T16:36:06.225 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:06 smithi191 bash[74504]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c 2024-02-19T16:36:06.465 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.c.service' 2024-02-19T16:36:06.489 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:06 smithi191 podman[74529]: 2024-02-19 16:36:06.22353062 +0000 UTC m=+0.577344261 container remove 7fedf19021a38740fcb0e7bdd59da9a3b92b69a07da87b1030a1d34645205e5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-9f8908e, GIT_CLEAN=True) 2024-02-19T16:36:06.490 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:06 smithi191 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.c.service: Deactivated successfully. 2024-02-19T16:36:06.490 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:06 smithi191 systemd[1]: Stopped Ceph mds.c for 81b9a938-cf41-11ee-95bb-87774f69a715. 2024-02-19T16:36:06.490 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 19 16:36:06 smithi191 systemd[1]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.c.service: Consumed 4min 1.307s CPU time. 2024-02-19T16:36:06.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:36:06.503 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-19T16:36:06.503 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-19T16:36:06.503 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.f 2024-02-19T16:36:06.778 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:06 smithi191 systemd[1]: Stopping Ceph mds.f for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:36:07.277 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f[67202]: 2024-02-19T16:36:06.817+0000 7f0bcd316700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:36:07.278 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:06 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f[67202]: 2024-02-19T16:36:06.817+0000 7f0bcd316700 -1 mds.f *** got signal Terminated *** 2024-02-19T16:36:10.002 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:09 smithi191 podman[74636]: 2024-02-19 16:36:09.666854948 +0000 UTC m=+2.952356243 container died 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T16:36:10.278 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:10 smithi191 podman[74636]: 2024-02-19 16:36:10.000964015 +0000 UTC m=+3.286465326 container cleanup 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T16:36:10.278 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:10 smithi191 bash[74636]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f 2024-02-19T16:36:10.278 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 19 16:36:10 smithi191 podman[74650]: 2024-02-19 16:36:10.185405131 +0000 UTC m=+0.517361569 container remove 8e976bff373870e11d73590557d8b9cf3fe711c0a3b2c36eef4d3b5c4d617e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-19T16:36:10.399 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.f.service' 2024-02-19T16:36:10.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:36:10.435 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-19T16:36:10.435 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-19T16:36:10.435 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.i 2024-02-19T16:36:10.764 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:36:10 smithi191 systemd[1]: Stopping Ceph mds.i for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:36:11.027 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:36:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: 2024-02-19T16:36:10.762+0000 7f55fce34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:36:11.028 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:36:10 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i[67842]: 2024-02-19T16:36:10.762+0000 7f55fce34700 -1 mds.i *** got signal Terminated *** 2024-02-19T16:36:13.947 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:36:13 smithi191 podman[74756]: 2024-02-19 16:36:13.660938946 +0000 UTC m=+3.015584972 container died b6a981c3c635febb50cb37c71a4f109507db3a6f3000c1ba4f23ed0cbb6b8763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T16:36:14.277 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:36:14 smithi191 podman[74756]: 2024-02-19 16:36:14.045823901 +0000 UTC m=+3.400469918 container cleanup b6a981c3c635febb50cb37c71a4f109507db3a6f3000c1ba4f23ed0cbb6b8763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-9f8908e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-19T16:36:14.278 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 19 16:36:14 smithi191 bash[74756]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-i 2024-02-19T16:36:14.461 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.i.service' 2024-02-19T16:36:14.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:36:14.495 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-19T16:36:14.495 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-19T16:36:14.495 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.l 2024-02-19T16:36:14.809 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:36:14 smithi191 systemd[1]: Stopping Ceph mds.l for 81b9a938-cf41-11ee-95bb-87774f69a715... 2024-02-19T16:36:15.277 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:36:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: 2024-02-19T16:36:14.807+0000 7f41d99bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T16:36:15.278 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:36:14 smithi191 ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l[66894]: 2024-02-19T16:36:14.807+0000 7f41d99bb700 -1 mds.l *** got signal Terminated *** 2024-02-19T16:36:17.959 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:36:17 smithi191 podman[74876]: 2024-02-19 16:36:17.645565168 +0000 UTC m=+2.940783338 container died ba5a4dcad17016ea893ea06d267a1c9ef552f868f9d6d820c5389dd1c6395399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-9f8908e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-19T16:36:18.267 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:36:18 smithi191 podman[74876]: 2024-02-19 16:36:18.040929506 +0000 UTC m=+3.336147675 container cleanup ba5a4dcad17016ea893ea06d267a1c9ef552f868f9d6d820c5389dd1c6395399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4d62b0e8d4858350590f414c6e75f5973ed83708481aa1042efce95a35ff3f05, name=ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l, RELEASE=wip-lusov-qdb-9f8908e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-19T16:36:18.267 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 19 16:36:18 smithi191 bash[74876]: ceph-81b9a938-cf41-11ee-95bb-87774f69a715-mds-l 2024-02-19T16:36:18.404 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81b9a938-cf41-11ee-95bb-87774f69a715@mds.l.service' 2024-02-19T16:36:18.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T16:36:18.439 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-19T16:36:18.439 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --force --keep-logs 2024-02-19T16:36:18.641 INFO:teuthology.orchestra.run.smithi008.stdout:Deleting cluster with fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:36:21.031 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --force --keep-logs 2024-02-19T16:36:21.221 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:36:23.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --force --keep-logs 2024-02-19T16:36:23.948 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:36:26.445 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:36:26.481 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:36:26.515 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T16:36:26.544 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-19T16:36:26.546 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi008/crash 2024-02-19T16:36:26.547 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash -- . 2024-02-19T16:36:26.589 INFO:teuthology.orchestra.run.smithi008.stderr:tar: /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash: Cannot open: No such file or directory 2024-02-19T16:36:26.589 INFO:teuthology.orchestra.run.smithi008.stderr:tar: Error is not recoverable: exiting now 2024-02-19T16:36:26.591 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi156/crash 2024-02-19T16:36:26.592 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash -- . 2024-02-19T16:36:26.635 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash: Cannot open: No such file or directory 2024-02-19T16:36:26.635 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2024-02-19T16:36:26.637 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi191/crash 2024-02-19T16:36:26.638 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash -- . 2024-02-19T16:36:26.676 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/crash: Cannot open: No such file or directory 2024-02-19T16:36:26.677 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-02-19T16:36:26.678 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-19T16:36:26.678 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-19T16:36:26.713 INFO:teuthology.orchestra.run.smithi008.stdout:2024-02-19T16:14:19.758079+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-19T16:36:26.713 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-19T16:36:26.714 INFO:tasks.cephadm:Compressing logs... 2024-02-19T16:36:26.714 DEBUG:teuthology.orchestra.run.smithi008:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-19T16:36:26.756 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-19T16:36:26.759 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-19T16:36:26.785 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-19T16:36:26.785 INFO:teuthology.orchestra.run.smithi191.stderr:: No such file or directory 2024-02-19T16:36:26.785 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.l.log 2024-02-19T16:36:26.785 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log 2024-02-19T16:36:26.785 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.i.log 2024-02-19T16:36:26.800 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.11.log 2024-02-19T16:36:26.800 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.z.log 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.8.log 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.5.log 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.f.log 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.f.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.5.log: 90.1% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log.gz 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log.gz 2024-02-19T16:36:26.801 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.2.log 2024-02-19T16:36:26.802 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log: 87.4% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log.gz 2024-02-19T16:36:26.802 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.c.log 2024-02-19T16:36:26.802 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.2.log: 91.8% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.z.log.gz 2024-02-19T16:36:26.802 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-19T16:36:26.806 INFO:teuthology.orchestra.run.smithi008.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-19T16:36:26.806 INFO:teuthology.orchestra.run.smithi008.stderr:: No such file or directory 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.6.log 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.a.log 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.6.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.g.log 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.j.log 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-02-19T16:36:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.3.log 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.x.log 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.0.log 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.x.log: 90.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log.gz 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.0.log: 94.4% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.g.log.gz 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr: 94.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.j.log.gz 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-19T16:36:26.808 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log: 84.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log.gz 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.9.log 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.d.log 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.9.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log: 87.3% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log.gz 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.a.log 2024-02-19T16:36:26.809 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.d.log.gz 2024-02-19T16:36:26.812 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.c.log 2024-02-19T16:36:26.812 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.a.log.gz 2024-02-19T16:36:26.813 INFO:teuthology.orchestra.run.smithi008.stderr: 91.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.10.log 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.b.log 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.4.log 2024-02-19T16:36:26.815 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.1.log 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.b.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.k.log 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.h.log 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.7.log 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.e.log 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.7.log: 90.1% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.audit.log.gz 2024-02-19T16:36:26.816 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.y.log 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.k.log.gz 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.e.log.gz 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.cephadm.log.gz 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log: 87.4% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph.log.gz 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.b.log 2024-02-19T16:36:26.817 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log: 91.8% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.y.log.gz 2024-02-19T16:36:26.818 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-19T16:36:26.835 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log: /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-19T16:36:26.853 INFO:teuthology.orchestra.run.smithi191.stderr: 94.3% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.l.log.gz 2024-02-19T16:36:26.873 INFO:teuthology.orchestra.run.smithi156.stderr: 94.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.h.log.gz 2024-02-19T16:36:26.904 INFO:teuthology.orchestra.run.smithi191.stderr: 92.6% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.c.log.gz 2024-02-19T16:36:27.084 INFO:teuthology.orchestra.run.smithi156.stderr: 93.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.b.log.gz 2024-02-19T16:36:27.128 INFO:teuthology.orchestra.run.smithi008.stderr: 89.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mgr.x.log.gz 2024-02-19T16:36:27.159 INFO:teuthology.orchestra.run.smithi191.stderr: 92.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.c.log.gz 2024-02-19T16:36:27.285 INFO:teuthology.orchestra.run.smithi156.stderr: 92.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.b.log.gz 2024-02-19T16:36:28.073 INFO:teuthology.orchestra.run.smithi008.stderr: 90.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mon.a.log.gz 2024-02-19T16:36:28.495 INFO:teuthology.orchestra.run.smithi191.stderr: 95.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.f.log.gz 2024-02-19T16:36:28.875 INFO:teuthology.orchestra.run.smithi156.stderr: 92.3% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.10.log.gz 2024-02-19T16:36:28.990 INFO:teuthology.orchestra.run.smithi191.stderr: 92.7% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.8.log.gz 2024-02-19T16:36:29.027 INFO:teuthology.orchestra.run.smithi191.stderr: 92.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.11.log.gz 2024-02-19T16:36:29.058 INFO:teuthology.orchestra.run.smithi156.stderr: 92.4% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.7.log.gz 2024-02-19T16:36:29.354 INFO:teuthology.orchestra.run.smithi156.stderr: 92.6% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.1.log.gz 2024-02-19T16:36:29.462 INFO:teuthology.orchestra.run.smithi008.stderr: 92.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.3.log.gz 2024-02-19T16:36:29.563 INFO:teuthology.orchestra.run.smithi008.stderr: 92.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.9.log.gz 2024-02-19T16:36:29.900 INFO:teuthology.orchestra.run.smithi008.stderr: 92.7% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.6.log.gz 2024-02-19T16:36:29.922 INFO:teuthology.orchestra.run.smithi008.stderr: 92.7% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.0.log.gz 2024-02-19T16:36:29.924 INFO:teuthology.orchestra.run.smithi008.stderr: 2024-02-19T16:36:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:real 0m3.153s 2024-02-19T16:36:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:user 0m12.952s 2024-02-19T16:36:29.924 INFO:teuthology.orchestra.run.smithi008.stderr:sys 0m0.430s 2024-02-19T16:36:30.719 INFO:teuthology.orchestra.run.smithi156.stderr: 93.0% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.4.log.gz 2024-02-19T16:36:30.721 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-02-19T16:36:30.721 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m3.948s 2024-02-19T16:36:30.721 INFO:teuthology.orchestra.run.smithi156.stderr:user 0m11.353s 2024-02-19T16:36:30.721 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m0.417s 2024-02-19T16:36:30.742 INFO:teuthology.orchestra.run.smithi191.stderr: 92.9% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.2.log.gz 2024-02-19T16:36:30.903 INFO:teuthology.orchestra.run.smithi191.stderr: 92.6% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-osd.5.log.gz 2024-02-19T16:36:42.045 INFO:teuthology.orchestra.run.smithi191.stderr: 93.5% -- replaced with /var/log/ceph/81b9a938-cf41-11ee-95bb-87774f69a715/ceph-mds.i.log.gz 2024-02-19T16:36:42.047 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-19T16:36:42.047 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m15.275s 2024-02-19T16:36:42.047 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m29.037s 2024-02-19T16:36:42.047 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.912s 2024-02-19T16:36:42.048 INFO:tasks.cephadm:Archiving logs... 2024-02-19T16:36:42.048 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi008/log 2024-02-19T16:36:42.049 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-19T16:36:43.862 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi156/log 2024-02-19T16:36:43.863 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-19T16:36:45.556 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi191/log 2024-02-19T16:36:45.557 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-19T16:36:49.309 INFO:tasks.cephadm:Removing cluster... 2024-02-19T16:36:49.309 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --force 2024-02-19T16:36:49.507 INFO:teuthology.orchestra.run.smithi008.stdout:Deleting cluster with fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:36:49.777 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --force 2024-02-19T16:36:49.970 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:36:50.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81b9a938-cf41-11ee-95bb-87774f69a715 --force 2024-02-19T16:36:50.428 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 81b9a938-cf41-11ee-95bb-87774f69a715 2024-02-19T16:36:50.716 INFO:tasks.cephadm:Removing cephadm ... 2024-02-19T16:36:50.716 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-19T16:36:50.731 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-19T16:36:50.747 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-19T16:36:50.763 INFO:tasks.cephadm:Teardown complete 2024-02-19T16:36:50.763 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-19T16:36:50.775 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-19T16:36:50.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-19T16:36:50.777 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-19T16:36:50.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-19T16:36:50.907 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-19T16:36:50.920 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-19T16:36:50.920 DEBUG:teuthology.orchestra.run.smithi008:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T16:36:50.923 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T16:36:50.925 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T16:36:50.934 INFO:teuthology.orchestra.run.smithi008.stderr:bash: line 1: ntpq: command not found 2024-02-19T16:36:50.937 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-02-19T16:36:50.938 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T16:36:50.938 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2024-02-19T16:36:50.938 INFO:teuthology.orchestra.run.smithi008.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +1433us[+1433us] +/- 89ms 2024-02-19T16:36:50.938 INFO:teuthology.orchestra.run.smithi008.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -282us[ -270us] +/- 60ms 2024-02-19T16:36:50.938 INFO:teuthology.orchestra.run.smithi008.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -794us[ -794us] +/- 73ms 2024-02-19T16:36:50.938 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:36:50.939 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-02-19T16:36:50.941 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T16:36:50.941 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-02-19T16:36:50.942 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 43 +1456us[+1481us] +/- 89ms 2024-02-19T16:36:50.942 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -300us[ -276us] +/- 60ms 2024-02-19T16:36:50.942 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -794us[ -794us] +/- 73ms 2024-02-19T16:36:50.942 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:36:50.943 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T16:36:50.943 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-19T16:36:50.944 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +1429us[+1437us] +/- 89ms 2024-02-19T16:36:50.944 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -291us[ -284us] +/- 60ms 2024-02-19T16:36:50.944 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -801us[ -794us] +/- 73ms 2024-02-19T16:36:50.944 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T16:36:50.944 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-19T16:36:50.956 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-19T16:36:50.957 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-19T16:36:50.969 DEBUG:teuthology.orchestra.run.smithi008:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-19T16:36:51.064 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-19T16:36:51.137 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-19T16:36:51.207 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T16:36:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T16:36:51.238 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T16:36:51.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T16:36:51.268 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T16:36:51.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T16:36:51.298 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-19T16:36:51.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-19T16:36:51.338 INFO:teuthology.task.internal:Duration was 2211.970928 seconds 2024-02-19T16:36:51.338 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-19T16:36:51.351 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-19T16:36:51.351 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-19T16:36:51.353 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-19T16:36:51.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-19T16:36:51.393 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T16:36:51.396 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T16:36:51.398 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T16:36:51.561 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-19T16:36:51.561 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi008.front.sepia.ceph.com 2024-02-19T16:36:51.562 DEBUG:teuthology.orchestra.run.smithi008:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-19T16:36:51.620 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-02-19T16:36:51.620 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-19T16:36:51.648 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-02-19T16:36:51.649 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-19T16:36:51.668 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-19T16:36:51.669 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-19T16:36:51.671 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-19T16:36:51.691 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-19T16:36:51.738 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-19T16:36:51.738 DEBUG:teuthology.orchestra.run.smithi008:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-19T16:36:51.740 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-19T16:36:51.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-19T16:36:52.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-19T16:36:52.172 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-19T16:36:52.173 DEBUG:teuthology.orchestra.run.smithi008:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-19T16:36:52.199 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-19T16:36:52.228 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-19T16:36:52.255 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-19T16:36:52.269 DEBUG:teuthology.orchestra.run.smithi008:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-19T16:36:52.271 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-19T16:36:52.274 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-19T16:36:52.294 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = core 2024-02-19T16:36:52.299 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-02-19T16:36:52.321 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-02-19T16:36:52.351 DEBUG:teuthology.orchestra.run.smithi008:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-19T16:36:52.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T16:36:52.365 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-19T16:36:52.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T16:36:52.380 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-19T16:36:52.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T16:36:52.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-19T16:36:52.416 INFO:teuthology.task.internal:Transferring archived files... 2024-02-19T16:36:52.417 DEBUG:teuthology.misc:Transferring archived files from smithi008:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi008 2024-02-19T16:36:52.417 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-19T16:36:52.462 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi156 2024-02-19T16:36:52.463 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-19T16:36:52.502 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488/remote/smithi191 2024-02-19T16:36:52.502 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-19T16:36:52.540 INFO:teuthology.task.internal:Removing archive directory... 2024-02-19T16:36:52.540 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-19T16:36:52.542 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-19T16:36:52.544 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-19T16:36:52.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-19T16:36:52.604 INFO:teuthology.task.internal:Not uploading archives. 2024-02-19T16:36:52.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-19T16:36:52.623 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-19T16:36:52.623 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-19T16:36:52.625 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-19T16:36:52.627 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-19T16:36:52.637 INFO:teuthology.orchestra.run.smithi008.stdout: 265690 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 16:36 /home/ubuntu/cephtest 2024-02-19T16:36:52.639 INFO:teuthology.orchestra.run.smithi156.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 16:36 /home/ubuntu/cephtest 2024-02-19T16:36:52.644 INFO:teuthology.orchestra.run.smithi191.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 16:36 /home/ubuntu/cephtest 2024-02-19T16:36:52.645 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-19T16:36:52.656 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-19T16:36:52.681 INFO:teuthology.nuke:Checking targets against current locks 2024-02-19T16:36:52.708 DEBUG:teuthology.nuke:shortname: smithi008 2024-02-19T16:36:52.708 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T16:36:52.734 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 15:51:39.462193', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6YLnbOWLAMMgd9y9+tWj00Ho2BFWYD4iRiKLHWDtiZ0W/ccp4KKfErA9licffVLsl1U+QT5YURnNVG80ilreo='} 2024-02-19T16:36:52.742 DEBUG:teuthology.nuke:shortname: smithi156 2024-02-19T16:36:52.743 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T16:36:52.760 DEBUG:teuthology.nuke:shortname: smithi191 2024-02-19T16:36:52.761 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T16:36:52.764 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 15:51:39.463922', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmO4OhXuNsz4g5x92SzAmoBxhaK2ReqT7bjDc5YDCRAi6G39ay7K8y7QhKaGxS7rlc1YwnBammnbEsi+sd24/o='} 2024-02-19T16:36:52.775 INFO:teuthology.orchestra.console.smithi008:Power off 2024-02-19T16:36:52.775 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T16:36:52.787 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-19_14:34:49-fs-wip-lusov-qdb-distro-default-smithi/7566488', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 15:51:39.465061', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHbjSsnTtHJUjPF9JBs62HxTGDNA7R5A4lfvXf5YGv9HA9LovsrHODQDoRmMW2QLrpghe3v74AgEx64XX7d/ojM='} 2024-02-19T16:36:52.800 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2024-02-19T16:36:52.800 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:36:52.820 INFO:teuthology.orchestra.console.smithi156:Power off 2024-02-19T16:36:52.821 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T16:36:52.831 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-19T16:36:52.844 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2024-02-19T16:36:52.844 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:36:52.859 INFO:teuthology.orchestra.console.smithi191:Power off 2024-02-19T16:36:52.859 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T16:36:52.869 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-02-19T16:36:52.882 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-02-19T16:36:52.882 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:36:52.916 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-19T16:36:56.833 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:36:56.870 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:36:56.917 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:36:56.944 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-19T16:36:56.980 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-02-19T16:36:57.026 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-19T16:37:00.949 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:37:00.981 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:37:01.027 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T16:37:01.059 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-19T16:37:01.090 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2024-02-19T16:37:01.135 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-19T16:37:01.161 INFO:teuthology.orchestra.console.smithi008:Power off completed 2024-02-19T16:37:01.191 INFO:teuthology.orchestra.console.smithi156:Power off completed 2024-02-19T16:37:01.236 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-02-19T16:37:01.343 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} duration: 2211.9709281921387 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi008 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9f8908eed2b2d7e8c316bd5be7ed6e49024931ee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f6d8b14e1eb54b03870dc0e937d46b1f status: fail success: false 2024-02-19T16:37:01.343 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T16:37:01.425 INFO:teuthology.run:FAIL